+- ScrapeBox Forum (https://www.scrapeboxforum.com)
+-- Forum: ScrapeBox Main Discussion (https://www.scrapeboxforum.com/Forum-scrapebox-main-discussion)
+--- Forum: General ScrapeBox Talk (https://www.scrapeboxforum.com/Forum-general-scrapebox-talk)
+--- Thread: Harvester, No proxies, 1 Thread, 20 sec delay, No results (/Thread-harvester-no-proxies-1-thread-20-sec-delay-no-results)
Harvester, No proxies, 1 Thread, 20 sec delay, No results - colink - 07-28-2021
Scrapebox V 2.1.0.25
My settings for Custom or detailed harvester
1 thread, no proxies, 20 second delay
I have tried with several simple keywords eg Ford Explorer, University Entrance
I have tried Google, Bing and Yahoo.
I get zero results.
Even with just one keyword in the keywords list.
Same searches in my browser return results as expected - no captcha request form google.
Broken link checker with 1 thread returns hundreds of results in seconds
Nothing in this old "no results" video helps https://www.youtube.com/watch?v=2QaLWgTXsRo
Any suggestions?
======
Also tried several times with "use server proxies" which returns ~"no proxies found"
RE: Harvester, No proxies, 1 Thread, 20 sec delay, No results - loopline - 07-31-2021
server proxies are currently offline.
Try going to settings >> harvester engine configureation >> import >>> download default engines
then try it. If it does not work go to help >> show error log >> harvester - what are the errors?
RE: Harvester, No proxies, 1 Thread, 20 sec delay, No results - colink - 08-01-2021
Thanks for your suggestions. - see response at bottom of this post.
Before I read your reply I tried again aver the last couple of days - as follows
It seems likely initially I got blocked using the Custom harvester as the delay set for Google, Bing and Yahoo did not work (I assume I set it incorrectly).
I think I got blocked on two different days (though no Errors showing in the Custom Harvester Error column - just zero results)
*** How would Scrapebox tell me I am blocked? Particularly as I could do exactly the same search on my browser and get the necessary results without Google Captcha
It looks like my original "too fast" searches has matched the Scrapebox browser profile to my IP and blocked it for maybe 24 hours each time.
I tried again today with Detailed Harvester and global delay of 20 seconds and got 9 sets of valid results, before all results became zero - I also tried some of the zero results in my Chrome browser and all OK with no captcha.
I know that proxies are best solution for Scrapebox, but I use it so infrequently and I am in no urgency for results, I assume the next best option is to find a very long delay that works with Google.
======
Error Logs both open as empty files.
I have just downloaded default engines and created a custom settings for Google with 10 result and 30 seconds delay - I will try again tomorrow and update in a new reply.
Would it be useful for Screpebox to pop up a message when Server Proxies are offline?
Thanks ColinK
RE: Harvester, No proxies, 1 Thread, 20 sec delay, No results - loopline - 08-03-2021
Try a massive delay, like 500 as a test. When you harvest with scrapebox its using no javascript results and that gives scrapebox the control and so google blocks differently then when using a browser. When using a browser your using javascript and that gives google control and so they are more lenient on blocking. You can try turning off javascript in your browser and try it.
Maybe on the popup, but server proxies are on the back burner right now. Its just public proxies that you can scrape with scrapebox anyway, and they have bigger things they are working on and dealing with and so few people use them anyway.
At any rate its all about speed. So if you find the right delay of what google thinks looks like a human then you won't get blocked. else you will get blocked.
If that delay is too painful then proxies is the way to go as that effectively lets you go faster.