+- ScrapeBox Forum (https://www.scrapeboxforum.com)
+-- Forum: ScrapeBox Main Discussion (https://www.scrapeboxforum.com/Forum-scrapebox-main-discussion)
+--- Forum: General ScrapeBox Talk (https://www.scrapeboxforum.com/Forum-general-scrapebox-talk)
+--- Thread: Scrapebox stopped scraping! (/Thread-scrapebox-stopped-scraping)
Scrapebox stopped scraping! - webmaster-andy - 11-02-2011
Hi,
During the last few days I have found a nice load of urls and have some more work to do.
I use 10 private proxies and yesterday Google stopped giving me results, now I get nothing from any search engine.
This is the heaviest use of SB I have made, in all I probably scraped 500,000 urls in 4 days. I have also filtered out a load of rubbish and obtained PR for perhaps 100,000 urls. I also did a run of 50000 urls for 'dofollow' yesterday.
Would this type of use burn the proxies out?
They have always been really good. I use Matt Rankin shared private proxies which I found way better than the squid private proxies which were utterly hopeless.
Any ideas guys?
Should I write to Matt?
Thanks!
RE: Scrapebox stopped scraping! - s4nt0s - 11-07-2011
I'm using Squid shared private proxies and scrape millions of blogs/post, etc without any issues. I'm using 100 shared and not 10 though. It sounds like you proxies are burnt out (banned from G).
RE: Scrapebox stopped scraping! - Thoth - 11-09-2011
Have you tried doing testing on the proxies with SB?
Some proxy places offer you to change proxies. Try changing them and trying again, though the problem with this is that you will need to update all your software where you have proxies entered, still if you have it, try it.
Enjoy!
Thoth
RE: Scrapebox stopped scraping! - pendergast - 11-15-2011
(11-07-2011, 08:57 PM)s4nt0s Wrote: I'm using Squid shared private proxies and scrape millions of blogs/post, etc without any issues. I'm using 100 shared and not 10 though. It sounds like you proxies are burnt out (banned from G).
I wrote my own page rank tool, using an http get function from Michael Shrenk (relies on Curl). It stopped working sometime after 11/1/2011, after G switched over to https (and made a number of other changes to their html and css classes). In the process of debugging I determined that it does return about 10795 characters from the Google page before it stops scraping. It returns none of the SERP entries.
So it is not simply blocked and there is no sign of CURL just failing because of the https. Also, the http get function continues to work seamlessly on other sites.
I'm looking for any clues I can find... anyone out there find any ?
RE: Scrapebox stopped scraping! - s4nt0s - 11-16-2011
Haha all that programming talk made it seem like you were speaking a different language. Unfortunately, I don't have a clue what the problem might be.