+- ScrapeBox Forum (https://www.scrapeboxforum.com)
+-- Forum: ScrapeBox Main Discussion (https://www.scrapeboxforum.com/Forum-scrapebox-main-discussion)
+--- Forum: General ScrapeBox Talk (https://www.scrapeboxforum.com/Forum-general-scrapebox-talk)
+--- Thread: Unable To Scrape Google Search Results (/Thread-unable-to-scrape-google-search-results)
Unable To Scrape Google Search Results - mortyzplace - 03-02-2016
Hi,
I have been able to scrape google search results in the past but am having an issue doing so now.
I am using private proxies and when I use the run the proxy checker all of my proxies are passing the anonymous test as well as the google test. I have set my scraping to a 1 to 1 ratio and it does not seem to matter if I am using the keyword "pig" or any other search term I am not getting any results.
Any help would be greatly appreciated. I have been knocking my head against this problem for days now.
I can return bing and yahoo results no problem.
I have also tested with scraped proxies that have passed the google test as well. Nothing seems to be working.
RE: Unable To Scrape Google Search Results - loopline - 03-06-2016
Well you should use a 1 to 50 or more ratio. Meaning 1 connection for every 50 proxies. Else your just guaranteeing a ban on your proxies. For private ones I mean.
Go to settings >> harvester engine configuration >> import >> download default engines. This will make sure you have the latest engine file as they dont' force update these as you can customize them. This could be your issue.
Else try a test on like
cloud
car
green
etc..
with no proxies, does it work?
If you have less then 50 proxies use the detailed harvester with a delay. You can then also see it rotatitg thru proxies and you know if its skipping an ip its banned.
RE: Unable To Scrape Google Search Results - vjbeng - 03-10-2016
(03-06-2016, 02:39 AM)loopline Wrote: Well you should use a 1 to 50 or more ratio. Meaning 1 connection for every 50 proxies. Else your just guaranteeing a ban on your proxies. For private ones I mean.hello,
Go to settings >> harvester engine configuration >> import >> download default engines. This will make sure you have the latest engine file as they dont' force update these as you can customize them. This could be your issue.
Else try a test on like
cloud
car
green
etc..
with no proxies, does it work?
If you have less then 50 proxies use the detailed harvester with a delay. You can then also see it rotatitg thru proxies and you know if its skipping an ip its banned.
Struggling with Google too.
Followed your instructions, since I use 10 private proxies. Doesnt work though.
Any other idea maybe ?
Thx
Adrien
RE: Unable To Scrape Google Search Results - loopline - 03-11-2016
Most of the time its a matter of blocked ips, if you are using 10 proxies you want to use the detailed harvester and use a delay. Probably a delay of at least 15, but you can test.