Does this generator follow robots.txt?

This version does not interpret robots.txt rules. Only crawl sites you own or have permission to crawl, and review their crawl policies first. Recognised noindex directives exclude page entries, and recognised nofollow directives suppress link discovery. This is not a full crawler-policy or indexing audit.

Do not confuse these generator behaviors with the crawl policies of other tools. Review permissions and the target’s own rules yourself, especially before generating a sitemap for a site you do not operate.