+- ScrapeBox Forum (https://www.scrapeboxforum.com)
+-- Forum: ScrapeBox Main Discussion (https://www.scrapeboxforum.com/Forum-scrapebox-main-discussion)
+--- Forum: General ScrapeBox Talk (https://www.scrapeboxforum.com/Forum-general-scrapebox-talk)
+--- Thread: scrape noindex page (/Thread-scrape-noindex-page)
scrape noindex page - daniele - 05-17-2020
hello everyone I would like to know if scrapebox can search within a site and also extrapolate non-indexed pages and return the urls to me. thanks
RE: scrape noindex page - loopline - 05-20-2020
You could use the site crawler to crawl thru all the pages and then you could use the check indexed function to see if those urls are indexed.