What is robots.txt and Its Importance in SEO

Robots.txt is a text document put in the root registry of a website that gives directions to search engine crawlers about which pages or segments of the webpage ought to or ought not be slithered and filed. It goes about as a guardian, guiding search engine bots on the most proficient method to explore and collaborate with a website’s content.

What is robots.txt and Its Importance in SEO