I am currently testing some functions using Scrapebox without proxies.
Google and Bing Meta Scraper offer a "delay between requests" option in connection settings (so I can can test a long list with delays of 12 to 30 seconds without getting blocked).
I do not see an option for "delay between requests" for the Page Scanner Addon.
Is there a way to set a delay in Page Scanner Addon?
I am using page scanner to search google results and find which pages have Google My Business, A Video section or Image section
eg https://www.google.com/search?q=ford+mustang
I did a test of 1000 similar URL's and it completed in about 10 minutes but after about 700 results it returned about 300 results with "HTTP/1.1 429 Too Many Requests" in the Status column.
One surprise is that I was able to do a manual search in browser while this was running and after it finished without a Captcha challenge.
Thanks ColinK
Google and Bing Meta Scraper offer a "delay between requests" option in connection settings (so I can can test a long list with delays of 12 to 30 seconds without getting blocked).
I do not see an option for "delay between requests" for the Page Scanner Addon.
Is there a way to set a delay in Page Scanner Addon?
I am using page scanner to search google results and find which pages have Google My Business, A Video section or Image section
eg https://www.google.com/search?q=ford+mustang
I did a test of 1000 similar URL's and it completed in about 10 minutes but after about 700 results it returned about 300 results with "HTTP/1.1 429 Too Many Requests" in the Status column.
One surprise is that I was able to do a manual search in browser while this was running and after it finished without a Captcha challenge.
Thanks ColinK