ScrapeBox Forum
Grab Links Problem - Printable Version

+- ScrapeBox Forum (https://www.scrapeboxforum.com)
+-- Forum: ScrapeBox Main Discussion (https://www.scrapeboxforum.com/Forum-scrapebox-main-discussion)
+--- Forum: General ScrapeBox Talk (https://www.scrapeboxforum.com/Forum-general-scrapebox-talk)
+--- Thread: Grab Links Problem (/Thread-grab-links-problem)



Grab Links Problem - Nosh - 04-18-2019

Hi,
I want to grab these links with company data: 
https://ranking-empresas.eleconomista.es/
I'm using "Grab Links by crawling a site"

Works fine, but I get only get 2200. Any chance to get the entire 20.000?


RE: Grab Links Problem - loopline - 04-19-2019

Try loading in diffferent pages for starting points. ITs probably just running out of crawl depth. So if you load in several differnet pages, like the first page and last and something in the middle or different sections of the site it will crawl thru different areas, if that makes sense.


RE: Grab Links Problem - Nosh - 04-21-2019

The problem is the pagination. URL stays the same. Do you have a solution for this ?


RE: Grab Links Problem - loopline - 04-23-2019

You can look at the requests in a debugger when you browsing thru the pages in a web browser. Often times you can see there is pagination but its just not displayed in your browser.

Now if its a matter of javascript it won't work, but sometimes you can pick out the pagination structure in debugger and then just fabricate those urls and they will work.

Http debugger pro is the program I use and it has a free fully featured trial.