+- ScrapeBox Forum (https://www.scrapeboxforum.com)
+-- Forum: ScrapeBox Main Discussion (https://www.scrapeboxforum.com/Forum-scrapebox-main-discussion)
+--- Forum: General ScrapeBox Talk (https://www.scrapeboxforum.com/Forum-general-scrapebox-talk)
+--- Thread: Are proxies important for these addons? (/Thread-are-proxies-important-for-these-addons)
Pages:
1
2
RE: Are proxies important for these addons? - olavor - 12-17-2018
Hi Loopline,
I ran the test on two different sites.
Site 1:
The error occurred both using the proxies and without proxy.
The error with proxies was Error 403.
The error without proxies was 503.
Then I ran the custom test you suggested for the site.
All proxies passed the anonymous test.
Filled the configuration tab with the site url and string to be searched and ran the test.
In the "Google Test Result" column the test failed for all proxies (error 403 and 503).
Understand the is the correct column to check the results although it doesn't make much sense because the test wasn't on Google.
Site 2:
The error occured with proxies, but not without proxies.
With local ip it worked just fine.
With proxies the error was "Error socket connection".
Then aso ran the custom test.
Interesting that in the custom test all of the proxies passed both in the anonymous test and in the custom test.
That confused a bit on why proxies didn't work in the broken link checker if they passed the custom test.
thanks
RE: Are proxies important for these addons? - loopline - 12-18-2018
It shouldn't say google test, it should say custom test, make sure the custom test box has a check mark in it in the configuration.
How many connections are you running? It may be that they are just too high. I can't say for sure why they would pass the custom test but fail in the broken links checker.
403 is a forbidden error and 503 is a server error (unless its google, it just means the ip is blocked).
503 could happen if the site is over loaded or your using too many connections. 403, with proxies, is possibly coming from the proxies its self or the end site saying those ips are blocked.
RE: Are proxies important for these addons? - olavor - 12-18-2018
Hi Loopline,
I'm going with 10 proxies and 20 connections on these tests.
Indeed. I think I had missed to check the custom test box.
Now it says "Custom Test Result". Thanks for noting.
Ran that for site 1 and it said failed for all proxies (403s and 503s).
So for site 1 it's congruent. Proxies and local ip are blocked for broken link checker for whatever reason.
Just don't know what can be done to fix that.
Site 2 is the one that bugs me most.
It passed the custom test for 9 out of the 10 proxies I'm using.
So I removed this one that was failing and ran broken link checker with the ones that passed.
Still got "Error socket connection" for most urls.
thanks
RE: Are proxies important for these addons? - loopline - 12-19-2018
If you want to post the site, I can give it a go. Im not sure why it would, unless its just from something else entirely like security software or something.
As for the first site, its possible its the user agent. Ill see if that can be changed via the ini.
RE: Are proxies important for these addons? - olavor - 12-19-2018
Not confortable with sharing the site publicly.
Thanks for checking the ini thing.
RE: Are proxies important for these addons? - loopline - 12-19-2018
Sure. If you want you can pm me the site, or not, up to you
I spoke with the dev team and they said adding the ability to modify the user agent in the ini file should be possible on the next update. I have no eta, but once you see an update you can update the broken links checker.
Then go to your scrapebox folder \ addons \ broken links checker folder - and in there is a sbbrokenlinkschecker.addon.ini
in that file should then be a line item where you can put your own user agent. So just try out some different user agents. Note that when you modify this you will need to do it with the broken links checker addon closed down. So just close it down, modify and save the ini, then restart the addon and try it.
If you need user agents, you can just google for a list, you can try the user agent of your browser as well.
RE: Are proxies important for these addons? - olavor - 12-20-2018
Not sure what is an user agent and how to deal with that.
Anyway will need to wait for the update to bother figuring out.
For now will send you a private msg with the site.
thanks a lot for your help.
RE: Are proxies important for these addons? - loopline - 12-20-2018
Just google "what is a user agent"
or
List of user agents