11-22-2016, 01:44 AM
(10-18-2016, 05:45 PM)edge13 Wrote: Hi,
I'm a new user, I have 20 private proxies from squidproxies, and set connections to 1 when I'm using the custom harvester, problem is it can't go too far, mostly after I get around 3000 urls, I start receiving errors all the time. The latest time, I only got less 700 urls.
Anyone can tell me what is the problem please?
Thanks!
Your proxies are all getting banned. Typically you would need to have 50-75 proxies or more before you can run 1 thread. You will need to wait until your proxies are unblocked and then use the detailed harvester and use a delay.
https://www.youtube.com/watch?v=GadX5AXiW34