ScrapeBox Forum
Why does scrapebox stop harvesting & freezes - Printable Version

+- ScrapeBox Forum (https://www.scrapeboxforum.com)
+-- Forum: ScrapeBox Main Discussion (https://www.scrapeboxforum.com/Forum-scrapebox-main-discussion)
+--- Forum: General ScrapeBox Talk (https://www.scrapeboxforum.com/Forum-general-scrapebox-talk)
+--- Thread: Why does scrapebox stop harvesting & freezes (/Thread-why-does-scrapebox-stop-harvesting-freezes)



Why does scrapebox stop harvesting & freezes - NewHarvester - 06-30-2014

Hi
I have been searching for this answer and not been able to find it. Please help.

I have 35 private proxies
I load up - for example 100 keywords
merge them with custom footprints (I used most common footprints available)
Use only Google as the search engine
Set the limit of results to 200 results.
Under settings . maximum connections set to 10. RND Delay set between 20-60 secs.

I start scraping and it will start fine upto 200-300 results and then it will freeze.
Sometimes it will freeze after 450 results.

What am i doing wrong? Thank you in advance for any help


RE: Why does scrapebox stop harvesting & freezes - loopline - 06-30-2014

There are a few things going on here, firstly you should note that the delay only works with the single threaded harvester. So if you are using the multi harvester or the custom harvester then the delay is not going to do you any good.

Next is that using 10 connections with 35 proxies is going to get your proxies based ultra fast. Just set it to 1-3 connections, depending on the footprint and then it will go slow enough that you won't have to worry about the delay or getting banned. Id start with 2 connections and if that works well then go to 3 and try it, if it doesn't work well on 2, try 1.

As for the freezing, are you using bing? Try only using bing with the custom harvester.

Does it say not responding, or does it ghost out or what? Or does the window look normal, and you can move it around, but the count does not increase. Can you be more specific about what you mean by freezing?


RE: Why does scrapebox stop harvesting & freezes - NewHarvester - 07-02-2014

(06-30-2014, 07:14 PM)loopline Wrote: There are a few things going on here, firstly you should note that the delay only works with the single threaded harvester. So if you are using the multi harvester or the custom harvester then the delay is not going to do you any good.

Next is that using 10 connections with 35 proxies is going to get your proxies based ultra fast. Just set it to 1-3 connections, depending on the footprint and then it will go slow enough that you won't have to worry about the delay or getting banned. Id start with 2 connections and if that works well then go to 3 and try it, if it doesn't work well on 2, try 1.

As for the freezing, are you using bing? Try only using bing with the custom harvester.

Does it say not responding, or does it ghost out or what? Or does the window look normal, and you can move it around, but the count does not increase. Can you be more specific about what you mean by freezing?

Hi
I unchecked the multi harvester, and reduce the connections to 2.
Using only Google, i started harvesting and then some time (10-15 mins) the count stopped increasing. The program does not freeze, just the count.

I saved the incomplete keywords and exited the program and reopened it and imported the remaining Keywords and started harvesting.
The count started going up again and then after 10-15 mins stopped increasing

Thanks


RE: Why does scrapebox stop harvesting & freezes - loopline - 07-02-2014

Ok, now I understand, thanks for clarifying. Its a security issue, 99% chance.

Your reaching a threashold and thats triggering a security program to lock the threads and not allow scrapebox to communicate with the internet.

So you need to make sure you have scrapebox set as trusted, allowed in all anti-virus, malware checkers, firewalls etc...

Also you could try shutting down unneeded programs as they can interfere and cause this, but given the consistency I would place my first bets on security software.

If you are using a router then you can try white listing scrapebox and disabling the firewall as a test.

Note that on anti-virus, malware checkers and firewalls you can't just disable them. Disabling them only stops new rules from being created, exsiting rules still fire, so you need to delete existing rules and set scrapebox as trusted.


RE: Why does scrapebox stop harvesting & freezes - NewHarvester - 07-03-2014

(07-02-2014, 08:04 PM)loopline Wrote: Ok, now I understand, thanks for clarifying. Its a security issue, 99% chance.

Your reaching a threashold and thats triggering a security program to lock the threads and not allow scrapebox to communicate with the internet.

So you need to make sure you have scrapebox set as trusted, allowed in all anti-virus, malware checkers, firewalls etc...

Also you could try shutting down unneeded programs as they can interfere and cause this, but given the consistency I would place my first bets on security software.

If you are using a router then you can try white listing scrapebox and disabling the firewall as a test.

Note that on anti-virus, malware checkers and firewalls you can't just disable them. Disabling them only stops new rules from being created, exsiting rules still fire, so you need to delete existing rules and set scrapebox as trusted.

Hi
I'm running SB on a Virtual server with no antivirus or malware software installed. Also it is a pretty good server with quad i5 & 4GB memory on Windows server 2008.

I ran SB using only Bing and it completed all the keywords. I then ran SB using only yahoo and that also completed successfully.

Only with Google. I tested the proxies against google in SB and GSA and all the proxies are working.

I am baffled! Can a SB developer or admin help me please?
Thanks


RE: Why does scrapebox stop harvesting & freezes - loopline - 07-04-2014

Then its probably the proxies are reaching the point where they are getting banned by google. That is assuming that your proxy provider does not limit the number of hits you can do to google.

Also testing the proxies, it just checks against basic google search, if you are doing anything else then they can be banned but still pass the proxy test. I have a video explaining it here.

https://www.youtube.com/watch?v=P9CbGhfc1aY

Also you can try less connections, like try 1 connection or try going to settings and unchecking both use custom ahrvester and use multi threaded harvester. Then harvest, when it fails eventually it will show you errors in the status column.