Robot Directives
Search engine directives (also called "Robot" directives") are snippets of code that give instructions to search engine website crawlers on how to catalog or indexed a websites content. Robots meta tags give the website owner granular control on how their content should be accessed, stored and displayed. It is important to note that these are suggestions only, and that the search engine may choose to ignore these directives.