Robots.txt is not an appropriate or effective way of blocking sensitive or confidential material. It only instructs well-behaved crawlers that the pages are not for them, but it does not prevent your server from delivering those pages to a browser that requests them. One reason is that search engines could still reference the URLs you block (showing just the URL, no title or snippet) if there happen to be links to those URLs somewhere on the Internet (like referrer logs). Also, non-compliant or rogue search engines that don't acknowledge the Robots Exclusion Standard could disobey the instructions of your robots.txt. Finally, a curious user could examine the directories or subdirectories in your robots.txt file and guess the URL of the content that you don't want seen.


Yes the article was highly informative it\’s very very useful to the newbies like me in the field of digital marketing, It shows instead of creating some website / blog waiting with hope for the traffic to come instead put more logical steps. Promoting at the different sources where usually traffic can be built for the websites excellent insight hope we can use to the fullest extent with your further guidance & support of-course.
All the content published on the MyThemeShop.com domain including images, site content published on the showcase and on the blog, belongs to MyThemeShop and is under copyright. Any reproduction of the site content has to be authorized and distinctly referenced back to the source. Written consent of MyThemeShop is required before the MyThemeShop website is used or exploited for any commercial and non-private purpose. Though the content published on demo sites is non-exclusive and is not copyrighted.​
Excellent post Brian. I think the point about writing content that appeals to influencers in spot on. Could you recommend some good, manual strategies through which I can spot influencers in boring niches *B2B* where influencers are not really talking much online? Is it a good idea to rely on newspaper articles to a feel for what a particular industry is talking about? Would love to hear your thoughts on that.
×