sitemap.download

ERRORS AND TROUBLESHOOTING

Why does crawling take a long time?

Pages are fetched one at a time with a short pause between requests. Response time on the target website, redirects, large HTML responses and timeouts all affect the duration.

Keep the tab open while it works. The 1,000-page limit is a maximum, not a promise of a particular completion time. You can stop and review the eligible pages already collected instead of waiting for every remaining request.

← Search all 59 questions