11-12-2016, 02:01 PM
(11-11-2016, 09:22 PM)loopline Wrote: No, it will not. Captcahs are used when posting to sites, but are not used when scraping.
They did have that once upon a day but it caused way more issue then it solved.
I mean you might get 50 requests and then get your ip blocked, then solve a captcah, then get 2-3 requests, then blocked again. Then you solve a captcah, get 2-3 requests then get permanently blocked or long term blocked for days with no captcha option.
So solving captchas takes a bunch of time and then you get a handful of requests before you long term ban or perma ban your ip. Its better to just slow down a bit and not get the ips blocked in the first place, or if they are public proxies just get more proxies.
I have a video here
https://www.youtube.com/watch?v=GadX5AXiW34
that might be helpful for you
thanks Loopline
I thought it would have been good to make it optional. so for those that would like to solve g captchas when scraping can opt to do so.. there could be some settings passed down to the end user such as max captchas to solve per a proxy in a 5 min period etc ..
Because I use back connect (reverse proxies) I feel this could actually get us more results.. Either way ive upped the wait time - set the retries to max and things are working much slower but its more stable
thanks for your help