+- ScrapeBox Forum (https://www.scrapeboxforum.com)
+-- Forum: ScrapeBox Main Discussion (https://www.scrapeboxforum.com/Forum-scrapebox-main-discussion)
+--- Forum: General ScrapeBox Talk (https://www.scrapeboxforum.com/Forum-general-scrapebox-talk)
+--- Thread: harvester delay time over 60 seconds poblem (/Thread-harvester-delay-time-over-60-seconds-poblem)
harvester delay time over 60 seconds poblem - walkallone - 10-28-2022
Hello,
i currently am using scrapebox with 5 proxies. in order to not get banned by google when scraping sites i increased the time between harvestes.
By default, in platform the maximum limit is 60 seconds. Unfortunately, these days 60s is not enough, my proxies get banned after aprox 12-16 scrapes, even if they rotate. to be able to effectively scrape i need about 10 min delay so at leas 600 seconds.
this theoretically IS possible by editing the userengines dat file. I have done this and put a huge number there - over 3000 seconds (just to test). See image.
However, this does not work as intended even it it is showed in scrapebox (the 3000+s delay). the maximum delay is still 60 sec. i believe the upper 60s limit somehow overwrites any number placed there.
I have tested this - watched the scrapebox window with a timer and the delay is 60s
Can this limit be increased?
RE: harvester delay time over 60 seconds poblem - loopline - 10-28-2022
Your using the delay in the engine setup, and that delay is per page.
If you use the detailed harvester, instead of the custom harvester and set a delay on the detailed harvester, that delay is in between each keyword and you can run that delay higher. Plus both delays will fire, so you will could slow it down to a crawl.
RE: harvester delay time over 60 seconds poblem - walkallone - 10-31-2022
Thanks a lot.
works like a charm now.
RE: harvester delay time over 60 seconds poblem - loopline - 11-03-2022
Great, glad its working!