+- ScrapeBox Forum (https://www.scrapeboxforum.com)
+-- Forum: ScrapeBox Main Discussion (https://www.scrapeboxforum.com/Forum-scrapebox-main-discussion)
+--- Forum: General ScrapeBox Talk (https://www.scrapeboxforum.com/Forum-general-scrapebox-talk)
+--- Thread: Meta Scraper Add-on at scale ? (/Thread-meta-scraper-add-on-at-scale)
Meta Scraper Add-on at scale ? - Nosh - 09-26-2020
Hi have a problem with the Meta Scraper Add On.
Imgur trying to scrape 1533 Keywords and I get only 500 results.
Afterwards I renewed the proxies, put delay on 30 seconds and switched connections to 1 and I get 900 results, but it doesn't stop (start button greyed out), it simply does not go on.
Afterwards I checked the proxies and all of them passed both tests.
What am I doing wrong ?
RE: Meta Scraper Add-on at scale ? - loopline - 09-27-2020
Its locked threads. Are these public or private proxies?
This is not uncommon with a public proxy for it to lock the threads. Make sure you add an exception for the entire scrapbox folder in all security software.
Also you can try whitelisting scrapbox in your router. Lastly you can try closing down any unnecessary programs, because anything that accesses the Internet such a Skype utorrent etc... can actually lock threads.
RE: Meta Scraper Add-on at scale ? - Nosh - 09-27-2020
These are private proxies
RE: Meta Scraper Add-on at scale ? - Nosh - 09-27-2020
Tried everything. Anymore ideas?
RE: Meta Scraper Add-on at scale ? - loopline - 09-28-2020
Even windows can lock threads. Try restarting your machine and make sure you have all windows updates installed. AFter that, here is a copy and paste from scrapebox support:
"That means that something has locked 1 or more of the threads. This can be security software such as anti-virus, malware checkers and firewalls. So you should whitelist scrapebox in all security software and then you can whitelist the entire scrapebox folder as well.
Further any program that accesses the internet can lock threads, things like skype, utorrent etc… So you can try closing down any unneeded programs. Then if its working you can turn programs back on 1 by 1 to find the culprit.
Further computer optimization software can lock threads so you can shut any such software down.
Take note that disabling security software (such as anti-virus, malware checkers and firewalls) often only stops new rules form forming, but allows existing rules to still fire. So you have to fully whitelist in the security software or uninstall the security software(as a test).
Further some security softwar requires you to whitelist in more then one place before it takes effect.
Also note that disabling a router firewall, does actually fully disable it.
Basically you have to sort out what is locking the threads, because scrapebox is forced to wait until all threads are released. On occasion it can be your operating system that does it, so you can try restarting your machine and/or lowering total connections.
One other thing to note is that this can happen with proxies that keep returning small amounts of data, it won't trigger the timeout because teh connections is still active. So try a test using no proxies or make sure you are using some quality private proxies.
Lastly if your running mac, you can try lowering the connections. Mac has terrible error handling when it comes to lots of errors stacking up quickly. So if there are too many errors stacking up too quick mac can choke, so lowering the threads fixes this. This is a non issue on windows.
"