To avoid undesirable content in the search indexes, webmasters can instruct spiders not to crawl certain files or directories through the standard robots.txt file in the root directory of the domain. Additionally, a page can be explicitly excluded from a search engine's database by using a meta tag specific to robots (usually  ). When a search engine visits a site, the robots.txt located in the root directory is the first file crawled. The robots.txt file is then parsed and will instruct the robot as to which pages are not to be crawled. As a search engine crawler may keep a cached copy of this file, it may on occasion crawl pages a webmaster does not wish crawled. Pages typically prevented from being crawled include login specific pages such as shopping carts and user-specific content such as search results from internal searches. In March 2007, Google warned webmasters that they should prevent indexing of internal search results because those pages are considered search spam.[47]

This was very interesting. I run a website that promotes sports entertainment amongst teenagers who are graphic designers or video editors. The foundation is in place (Over 60 contributors) so my only focus is how to blog consistently about what goes on in the sports world with appeal to teenagers. I am confident i took a huge step today after learning these 4 steps!
I really enjoyed your post, im building my own business from the ground up making custom furniture, lighting, and home decor. it took me a year to launch my website and now im trying to invite more traffic and ways for clients and interested parties to share my content and start buying my product. I liked the idea of Share triggers… im going to be incorporating that into my social media strategies. Any advice would go a long way. thanks again Brian
The goal of SEO is to get a web page high search engine ranking. The better a web page's search engine optimization, the higher a ranking it will achieve in search result listings. (Note that SEO is not the only factor that determines search engine page ranks.) This is especially critical because most people who use search engines only look at the first page or two of the search results, so for a page to get high traffic from a search engine, it has to be listed on those first two pages, and the higher the rank, the closer a page is to the number one listing, the better.  And whatever your web page's rank is, you want your website to be listed before your competitor's websites if your business is selling products or services over the internet.

Thanks for the very, very in-depth article. I am a real estate agent in Miami, Florida and have been blogging all-original content for the past 21 months on my website and watched traffic increase over time. I have been trying to grow my readership/leads/clients exponentially and have always heard about standard SEO backlink techniques and writing for my reader, not influencers. Recently, I have had a few of my articles picked up and backlinked by 2 of the largest real estate blogs in the country, which skyrocketed visits to my site. Realizing what I wrote about, that appealed to them, and now reading your article, I am going to continue writing in a way that will leverage those influencers to help me with quality backlinks.


I really enjoyed your post, im building my own business from the ground up making custom furniture, lighting, and home decor. it took me a year to launch my website and now im trying to invite more traffic and ways for clients and interested parties to share my content and start buying my product. I liked the idea of Share triggers… im going to be incorporating that into my social media strategies. Any advice would go a long way. thanks again Brian
I consulted a few years ago before Yahoo and CNET and my clients were all small businesses, even friends' sites.  No matter the size of the project, you can still try to get some insight into your target audiences and what they need or want.  I mentioned in a previous comment I used Search once to determine sentiment on a site vs. it's competitors by searching for a feature the site and its competitors all had, along with "like", "love", "hate", "wish", etc.  I also took note of who the people were who said those things and where they were talking (forums, twitter, etc).  It's a hacked manual approach and although not nearly as quality as a good market research report, at least I have a llittle bit of insight before going out to make site recommendations based solely on tags & links.  If you're recommending the site build things that people want (and fix or remove things that they dont), you're more likely to gain links and traffic naturally.
Black hat SEO attempts to improve rankings in ways that are disapproved of by the search engines, or involve deception. One black hat technique uses hidden text, either as text colored similar to the background, in an invisible div, or positioned off screen. Another method gives a different page depending on whether the page is being requested by a human visitor or a search engine, a technique known as cloaking. Another category sometimes used is grey hat SEO. This is in between black hat and white hat approaches, where the methods employed avoid the site being penalized but do not act in producing the best content for users. Grey hat SEO is entirely focused on improving search engine rankings.
For some reason I had to delete some pages, these pages are using the HTML suffix, so I blocked them in robots.txt use Disallow: /*.html, but it’s been almost a year, I found that google robot often capture these pages, How can I quickly let Google completely remove these pages? And I have removed these URL from google webmaster tool by google index-> remove URLs, but Google still capture these pages.
However I feel that batching all the things influencers share , filter whats relevant from whats not… and ultimately niche it down to identify which exact type of content is hot in order to build our own is a bit fuzzy. Influencers share SO MUCH content on a daily basis – how do you exactly identify the topic base you’ll use build great content that is guaranteed to be shared?
Keep resources crawlable. Blocking page resources can give Google an incomplete picture of your website. This often happens when your robots.txt file is blocking access to some or all of your page resources. If Googlebot doesn't have access to a page's resources, such as CSS, JavaScript, or images, we may not detect that it's built to display and work well on a mobile browser. In other words, we may not detect that the page is "mobile-friendly," and therefore not properly serve it to mobile searchers.
You grant to Us a worldwide, irrevocable, non-exclusive, royalty-free license to use, reproduce, adapt, publish, translate and distribute Your Content in any existing or future media. You also grant to Us the right to sublicense these rights and the right to bring an action for infringement of these rights. If You delete Content, we will use reasonable efforts to remove it from the Service, but You acknowledge that caching or references to the Content may not be made immediately unavailable.
Write a description that would both inform and interest users if they saw your description meta tag as a snippet in a search result. While there's no minimal or maximal length for the text in a description meta tag, we recommend making sure that it's long enough to be fully shown in Search (note that users may see different sized snippets depending on how and where they search), and contains all the relevant information users would need to determine whether the page will be useful and relevant to them.
Brian, I’ve drunk your Kool aid! Thank you for honesty and transparency – it really gives me hope. Quick question: I am beyond passionate about a niche (UFOs, extraterrestrials, free energy) and know in my bones that an authority site is a long term opportunity. The problem today is that not many products are attached to this niche and so it becomes a subscriber / info product play. However, after 25+ years as an entrepreneur with a financial background and marketing MBA, am I Internet naive to believe that my passion and creativity will win profitability in the end? The target audience is highly passionate too. Feedback?
Google Analytics is an invaluable source of data on just about every conceivable aspect of your site, from your most popular pages to visitor demographics. Keep a close eye on your Analytics data, and use this information to inform your promotional and content strategies. Pay attention to what posts and pages are proving the most popular. Inspect visitor data to see how, where and when your site traffic is coming from.

But some schema extensions are targeted at search engines. These code snippets tell Google which elements you would like to display next to your links in the search results. Of course, Google isn’t obliged to follow your instructions, and they can totally ignore the schema you insert in your code. But often, Google honors the schema you insert in your pages.

Black hat SEO attempts to improve rankings in ways that are disapproved of by the search engines, or involve deception. One black hat technique uses hidden text, either as text colored similar to the background, in an invisible div, or positioned off screen. Another method gives a different page depending on whether the page is being requested by a human visitor or a search engine, a technique known as cloaking. Another category sometimes used is grey hat SEO. This is in between black hat and white hat approaches, where the methods employed avoid the site being penalized but do not act in producing the best content for users. Grey hat SEO is entirely focused on improving search engine rankings.
Like many SEOs, I was hired with one vague responsibility: to set up an SEO program and achieve results. Like many SEOs, we jumped right in and started spewing out SEO audits, rewriting title tags, offering up link suggestions, rewriting URLs and so on. And like many SEOs we promised results. But what we didn’t do, until that fateful launch, was develop a comprehensive strategy.
Yes the article was highly informative it\’s very very useful to the newbies like me in the field of digital marketing, It shows instead of creating some website / blog waiting with hope for the traffic to come instead put more logical steps. Promoting at the different sources where usually traffic can be built for the websites excellent insight hope we can use to the fullest extent with your further guidance & support of-course.
Firstly, a disclaimer – don’t spam Reddit and other similar sites hoping to “hit the jackpot” of referral traffic, because it’s not going to happen. Members of communities like Reddit are extraordinarily savvy to spam disguised as legitimate links, but every now and again, it doesn’t hurt to submit links that these audiences will find genuinely useful. Choose a relevant subreddit, submit your content, then watch the traffic pour in.
Hey Ted, thanks for the great questions! The peak times refer to your particular time zone, if you are targeting an audience that resides in the same zone as you. You can also use tools to find out when most of your audience is online. For example, Facebook has this built into their Page Insights. For Twitter, you can use https://followerwonk.com/. Many social posting tools also offer this functionality.
×