FAQ / Glossary

robots.txt

‹ Glossary

A text file in the root of a website (/robots.txt) that tells robots where they may and may not go. The command Disallow: / forbids robots from the whole website. It also contains a link to the sitemap.

The file is only a recommendation and does not secure content. Protect sensitive things with a password, not with a robots.txt file.

Still not sure what to do?

Write to us and we will solve it together.

Contact support ›