05-28-2016, 05:30 PM
Ive got a profile site id like to scrape.
The format of the start URL im interested in is : http://example.com/artsits/new-york - on there is a list a artists in new york. 10 per page with pagination (next / prev buttons) between each of the pages, there are 100s of pages.
- Im trying to scrape a list of all the profiles pages URLS from the start URL
- Then once i have a list of their URLs i want to go to each profile page and extract select data (eg: name, email, url) either using the html path or xpath (or something similar)
Previously to do this i was using Kimonify which worked ok, but dosnt support proxies and dosnt support crawl rate which often gets its self banned as it just powers through the crawl to quickly.
Can this be done with Scrape Box ?
The format of the start URL im interested in is : http://example.com/artsits/new-york - on there is a list a artists in new york. 10 per page with pagination (next / prev buttons) between each of the pages, there are 100s of pages.
- Im trying to scrape a list of all the profiles pages URLS from the start URL
- Then once i have a list of their URLs i want to go to each profile page and extract select data (eg: name, email, url) either using the html path or xpath (or something similar)
Previously to do this i was using Kimonify which worked ok, but dosnt support proxies and dosnt support crawl rate which often gets its self banned as it just powers through the crawl to quickly.
Can this be done with Scrape Box ?