07-25-2018, 05:55 PM
Hi everyone,
I'm using the URL Generator in the Email Scraper to scrape a directory. The URLs look like this
So I tried
https://domaine.fr/page?keyword&from=1&to={pagenum}
but the website crashes when the pagenum is too high.
I want to do something like
https://domaine.fr/page?keyword&from={pa...o={pagenum}
I tried with
https://domaine.fr/page?keyword&from={ke...o={pagenum}
with 1, 21, 41, 61, etc... as {keyword}, (from an Excel) but I have thousand of useless URLs...
Can someone help me?
Thanks!
I'm using the URL Generator in the Email Scraper to scrape a directory. The URLs look like this
Code:
https://domaine.fr/page?keyword&from=1&to=20
https://domaine.fr/page?keyword&from=21&to=40
https://domaine.fr/page?keyword&from=41&to=60
...
https://domaine.fr/page?keyword&from=1781&to=1800
So I tried
https://domaine.fr/page?keyword&from=1&to={pagenum}
but the website crashes when the pagenum is too high.
I want to do something like
https://domaine.fr/page?keyword&from={pa...o={pagenum}
I tried with
https://domaine.fr/page?keyword&from={ke...o={pagenum}
with 1, 21, 41, 61, etc... as {keyword}, (from an Excel) but I have thousand of useless URLs...
Can someone help me?
Thanks!