ScrapeBox Forum
Never finishing... - Printable Version

+- ScrapeBox Forum (https://www.scrapeboxforum.com)
+-- Forum: ScrapeBox Main Discussion (https://www.scrapeboxforum.com/Forum-scrapebox-main-discussion)
+--- Forum: General ScrapeBox Talk (https://www.scrapeboxforum.com/Forum-general-scrapebox-talk)
+--- Thread: Never finishing... (/Thread-never-finishing)



Never finishing... - DigitalMu - 06-27-2019

I've noticed this on several addons, but most recently (just moments ago) on Link Extractor.

I was just doing a quick search to learn the addon...first time using it.  As is the case many times, it'll almost finish.....but won't.  It will just sit there forever until I go to the task manager and stop the process.

Any ideas? 

[Image: DvQu4Rg.jpg]


RE: Never finishing... - loopline - 06-28-2019

ITs locked threads. Does it always do it on the same 2 urls?

Here is a copy and paste from support:

That means that something has locked 1 or more of the threads. This can be security software such as anti-virus, malware checkers and firewalls. So you should whitelist scrapebox in all security software and then you can whitelist the entire scrapebox folder as well.

Further any program that accesses the internet can lock threads, things like skype, utorrent etc… So you can try closing down any unneeded programs. Then if its working you can turn programs back on 1 by 1 to find the culprit.

Further computer optimization software can lock threads so you can shut any such software down.

Take note that disabling security software (such as anti-virus, malware checkers and firewalls) often only stops new rules form forming, but allows existing rules to still fire. So you have to fully whitelist in the security software or uninstall the security software(as a test).

Further some security softwar requires you to whitelist in more then one place before it takes effect.

Also note that disabling a router firewall, does actually fully disable it.


Basically you have to sort out what is locking the threads, because scrapebox is forced to wait until all threads are released. On occasion it can be your operating system that does it, so you can try restarting your machine and/or lowering total connections.

One other thing to note is that this can happen with proxies that keep returning small amounts of data, it won't trigger the timeout because teh connections is still active. So try a test using no proxies or make sure you are using some quality private proxies.

Lastly if your running mac, you can try lowering the connections. Mac has terrible error handling when it comes to lots of errors stacking up quickly. So if there are too many errors stacking up too quick mac can choke, so lowering the threads fixes this. This is a non issue on windows.


RE: Never finishing... - DigitalMu - 07-05-2019

Thanks for the reply and sorry for my slow response. On my scraping computer, I'm not running anything really, but it is behind a router. The URLs where it happens seems to be random, but it also seems to be mostly with Link Extractor. I'm still sorting it all out as I'm learning Smile You gave me some things to consider - thanks!!


RE: Never finishing... - DigitalMu - 07-10-2019

(06-28-2019, 04:33 PM)loopline Wrote: ITs locked threads.  Does it always do it on the same 2 urls?

Here is a copy and paste from support:

That means that something has locked 1 or more of the threads.  This can be security software such as  anti-virus, malware checkers and firewalls.   So you should whitelist scrapebox in all security software and then you can whitelist the entire scrapebox folder as well.  

Further any program that accesses the internet can lock threads, things like skype, utorrent etc…  So you can try closing down any unneeded programs.  Then if its working you can turn programs back on 1 by 1 to find the culprit.  

Further computer optimization software can lock threads so you can shut any such software down.  

Take note that disabling security software (such as anti-virus, malware checkers and firewalls) often only stops new rules form forming, but allows existing rules to still fire.  So you have to fully whitelist in the security software or uninstall the security software(as a test).  

Further some security softwar requires you to whitelist in more then one place before it takes effect.  

Also note that disabling a router firewall, does actually fully disable it.


Basically you have to sort out what is locking the threads, because scrapebox is forced to wait until all threads are released.  On occasion it can be your operating system that does it, so you can try restarting your machine and/or lowering total connections.  

One other thing to note is that this can happen with proxies that keep returning small amounts of data, it won't trigger the timeout because teh connections is still active.  So try a test using no proxies or make sure you are using some quality private proxies.  

Lastly if your running mac, you can try lowering the connections.  Mac has terrible error handling when it comes to lots of errors stacking up quickly.  So if there are too many errors stacking up too quick mac can choke, so lowering the threads fixes this.  This is a non issue on windows.



OK, so this is still happening.  I looked at all of your suggestions (thanks) and they don't really seem to apply here.  check out the video





It seems that in most large batches, there's at least one or two urls that get stuck "reading".  Even when I hit stop, it won't stop.  I have to go stop the process.  It's a mystery to me and happening enough  that it's a pain Smile


RE: Never finishing... - loopline - 07-11-2019

To be honest sometimes even windows will cause this. There is an option in the link extractor settings to auto kill it when there is only X connections left. So you can try that setting as well.

When I run things on my servers that run 24/7 with automator, I have it auto kill the entire process every 12 hours and restart it in case something gets locked. Every 12 hours is aggressive, but I built it all to work in ultra small batches so I lose almost nothing and it makes it hands off. I haven't touched my primary scrapebox list server in over a month and its still running. But its a windows based software and even windows can cause locks. But try the auto kill at X connections option.


RE: Never finishing... - DigitalMu - 07-11-2019

Thanks - very good advice. I'm slowly seeing how to get around various personality quirks like this Smile


RE: Never finishing... - loopline - 07-12-2019

Smile Its an excellent program but yes its still subjective to a windows environment, which has certain drawbacks. But for this application windows is better then mac, its the best available that scrapebox works with and perhaps the best available period as its not available to try on linux etc...


RE: Never finishing... - Fred78 - 09-02-2019

Hello,

Having the same problem than DigitalMu

You said "But try the auto kill at X connections option. "

Where is this option ? I can't find it

Thanks

Edit: I found it ! Sorry


RE: Never finishing... - loopline - 09-04-2019

Glad you got it sorted! Smile