Add Robots.txt

A robots.txt file tells search engine crawlers which URLs the crawler can access on your site.

This feature is available in paid plans.

To create a Robots.txt for your project/site, follow the below instructions.

1. Click the 'Site Settings' icon from the left navigation bar:

Site Settings

1. Click on Robots.txt and enter your content that you want to display in Robots.txt.

1. Add your content as below and Press 'Save Settings' button to save. You can also click on 'View robots.txt' to make sure you can see the content. Save the Robots.txt URL if needed.

1. The Robots.txt URL is always available at https://domainname.com/robots.txt or https://subdomain.domainname.com/robots.txt based on your site URL.

Once the above steps are done, you can start using the Robots.txt URL at this point.