Robots.txt is not an appropriate or effective way of blocking sensitive or confidential material. It only instructs well-behaved crawlers that the pages are not for them, but it does not prevent your server from delivering those pages to a browser that requests them. One reason is that search engines could still reference the URLs you block (showing just the URL, no title or snippet) if there happen to be links to those URLs somewhere on the Internet (like referrer logs). Also, non-compliant or rogue search engines that don't acknowledge the Robots Exclusion Standard could disobey the instructions of your robots.txt. Finally, a curious user could examine the directories or subdirectories in your robots.txt file and guess the URL of the content that you don't want seen.
Search engines may penalize sites they discover using black or grey hat methods, either by reducing their rankings or eliminating their listings from their databases altogether. Such penalties can be applied either automatically by the search engines' algorithms, or by a manual site review. One example was the February 2006 Google removal of both BMW Germany and Ricoh Germany for use of deceptive practices. Both companies, however, quickly apologized, fixed the offending pages, and were restored to Google's search engine results page.
Use social networks to expand your reach. Social networking is hugely important, and ensuring that you have a solid presence will have a large impact on your views. Post compelling content and you’ll soon build a loyal following. Follow and share with other users, who may reciprocate and follow you. There are a variety of ways that you can use social networks to extend your online presence, depending on the needs of your site.
SEO often involves improving the quality of the content, ensuring that it is rich in relevant keywords and organizing it by using subheads, bullet points, and bold and italic characters. SEO also ensures that the site’s HTML is optimized such that a search engine can determine what is on the page and display it as a search result in relevant searches. These standards involve the use of metadata, including the title tag and meta description. Cross linking within the website is also important.
Website owners recognized the value of a high ranking and visibility in search engine results, creating an opportunity for both white hat and black hat SEO practitioners. According to industry analyst Danny Sullivan, the phrase "search engine optimization" probably came into use in 1997. Sullivan credits Bruce Clay as one of the first people to popularize the term. On May 2, 2007, Jason Gambert attempted to trademark the term SEO by convincing the Trademark Office in Arizona that SEO is a "process" involving manipulation of keywords and not a "marketing service."