+- ScrapeBox Forum (https://www.scrapeboxforum.com)
+-- Forum: ScrapeBox Main Discussion (https://www.scrapeboxforum.com/Forum-scrapebox-main-discussion)
+--- Forum: General ScrapeBox Talk (https://www.scrapeboxforum.com/Forum-general-scrapebox-talk)
+--- Thread: Grab links from very large sites (/Thread-grab-links-from-very-large-sites)
Grab links from very large sites - Nosh - 11-27-2018
Hi,
is it possible to scrape URLs of larger sites like Am*zon ?
I tried it with the Grab Links function, but it ends with 168730 URLs.
RE: Grab links from very large sites - loopline - 11-28-2018
Sure. You mean internal or external urls or ? Probably your going to fast and its getting you blocked, or maybe thats the max crawl depth.