+- ScrapeBox Forum (https://www.scrapeboxforum.com)
+-- Forum: ScrapeBox Main Discussion (https://www.scrapeboxforum.com/Forum-scrapebox-main-discussion)
+--- Forum: General ScrapeBox Talk (https://www.scrapeboxforum.com/Forum-general-scrapebox-talk)
+--- Thread: Best proxies and settings to scrape 30k Google keywords? (/Thread-best-proxies-and-settings-to-scrape-30k-google-keywords)
Best proxies and settings to scrape 30k Google keywords? - mortale - 06-29-2015
I've got a decent footprint and I'm trying to harvest as many blogs as possible. It felt like public proxies took forever to harvest and test so I signed up for a service claiming to do it for me while delivering the fresh ones through an API. Nothing bad about the service but in a best case scenario I only seem to get 10-12 keywords done before having to repeat the process of fetching new proxies. Just picking Scrapebox proxies delivers better results.
So, I'm thinking about going over to private proxies. After doing some research, I end up with two questions:
1) Should I go for private or shared when harvesting Google? (seems to be half price)
2) How long would it take to harvest 30k Google results when using 5-10 proxies?
I saw loopline's video about harvesting 1 million results and I really like the idea behind the service (rotating so you always have fresh private proxies) but it seems the site is closed for signups at the moment.
RE: Best proxies and settings to scrape 30k Google keywords? - loopline - 06-29-2015
I use shared, just tell the provider you want to harvest google and they will match you up with people who don't want to harvest google. Thats been my experience anyway, buyproxies does a good job with this.
No idea, I don't time them, the idea behind public proxies is lower connections and slower harvesting so you can yield more results quicker in the long run by making it set it and forget it and reduce hassle.
So with 10 proxies you wouldn't even be able to do 1 connection solid, you would need a delay. So its roughly like
1 connection for ever 20-30+ proxies for basic keyword harvesting
1 conneciton for ever 30-50+ proxies for advanced operators.
But that 1 connection is fast and will yield a lot of results faster then you think.
The next thing to keep in mind is scrapebox has over 20 engines, many of which are google powered, like deeperweb, and so you can get a lot of results without having to worry about the same google ip bans, plus other engines too.
I also had someone say they were getting ok results using proxy rack, which might be an option for you. They will fail the scrapebox proxy test, but thats fine, just load them and use them to scrape and they will work.