In order to avoid undesirable content while in the lookup indexes, webmasters can instruct spiders not to crawl selected data files or directories with the normal robots.txt file in the root Listing with the domain. Also, a website page might be explicitly excluded from a internet search engine's database through https://www.facebook.com/acourseinmiraclesdavid/