CRAWLING AND LIMITS
Why did crawling stop at 1,000 pages?
The safety limit is 1,000 page attempts per run, including failed requests. It is not the XML protocol’s general limit. Results are marked partial when more work remains. For a larger site, use a platform-generated sitemap or a crawler designed for larger inventories. Starting at a section can help focus discovery, but this tool does not enforce a path boundary and may follow links elsewhere on the same origin.
Download partial results before starting again. Increasing the limit is not an option in the interface, and a second run is not automatically merged with the first.