+ This Module Allows Users to Customize their robots.txt File, Informing + Search Engine Crawlers About the Accessible URLs on your Site. + Its Primary Purpose is to Prevent Overloading your Site with Requests. +
++
Can Effortlessly Configure the robots.txt File Based on the Different Modes. ++
Available in Odoo 17.0 Community and Enterprise. ++ Go to Website --> Settings --> SEO --> Edit robots.txt File and Choose the Mode from the Wizard. +
++
++
++ Configure robots.txt File Effortlessly.
++ Able to Choose Different Modes to Configure robot.txt File.
++ A robots.txt file is a + text file used by websites + to give instructions to web + crawlers (like Googlebot) + about which pages they can + and cannot crawl and index. +
++ A robots.txt file is necessary + to manage search engine bots' + access to your site's content. + It helps you control which pages + or files are accessible to bots + and which are not, thereby optimizing + your site's search engine performance + and preventing overloading your server + with unnecessary requests. +
+