+- ScrapeBox Forum (https://www.scrapeboxforum.com)
+-- Forum: ScrapeBox Main Discussion (https://www.scrapeboxforum.com/Forum-scrapebox-main-discussion)
+--- Forum: General ScrapeBox Talk (https://www.scrapeboxforum.com/Forum-general-scrapebox-talk)
+--- Thread: How to scrape all of the URLs listed in one site... ? (/Thread-how-to-scrape-all-of-the-urls-listed-in-one-site)
How to scrape all of the URLs listed in one site... ? - vincentseo - 10-24-2011
Hi, I have a site which is similar to dmoz. It's a directory with thousands of URLs (all different).
Is it possible to scrape all of these URL automatically?
Been trying site:www.url.com, but it only scrapes the directory's URL not the sites present in the directory.
Thanks, V
RE: How to scrape all of the URLs listed in one site... ? - kachaloo.k - 10-24-2011
Hi,
Yes, Its possible.
Send me the site details and let me look at it and I will tell you how to get the details out.
Cheers,
Kachaloo
RE: How to scrape all of the URLs listed in one site... ? - vincentseo - 10-24-2011
Hi Kachaloo.... Basicaly I want to scrape all of the URL mentioned on this site: http://www.web-affiliations.com/forum/
Not just the url web-affilatio.comn/xxx but all the different domains mentioned.
Can Scrapebox do that?
thanks for your help :-)
RE: How to scrape all of the URLs listed in one site... ? - kachaloo.k - 10-24-2011
as far as I know scrapebox cannot do that. You will need to make a custom application to do that for you.
Cheers,
Kachaloo