+- ScrapeBox Forum (https://www.scrapeboxforum.com)
+-- Forum: ScrapeBox Main Discussion (https://www.scrapeboxforum.com/Forum-scrapebox-main-discussion)
+--- Forum: General ScrapeBox Talk (https://www.scrapeboxforum.com/Forum-general-scrapebox-talk)
+--- Thread: expired domain plugin (/Thread-expired-domain-plugin)
expired domain plugin - [email protected] - 02-06-2017
I am very new to scrape box and scrapping in general - I hope my three question below makes sense.
1. Is there a guide of how many private proxies should be used to in domain finder plugin vs how many threads should be used. - default is 30 but for example, if I want to bump that up to 50 how many proxies should I use?
2. I used domain finder without proxy and a URL that I know was available was not showing up - is that because my IP address was banned from checking if domain was available
the domain I was testing with was edna.edu.au (now no longer available because I have since registered it.
3A. Are private proxies or rotating proxies better for the expired domain finder
3B. Are private proxies for harvesting URLs from google
RE: expired domain plugin - loopline - 02-08-2017
I would say 1-3 threads on most sites is safe with no proxies, default of 30 might get your ip banned if you are not using proxies. thus about 15 proxies, using that same ratio, is probably safe for 50 connections.
Its hard to say, but quite possibly because you were using too many connections and your ip was blocked.
don't use rotating proxies for the expired finder, the general success rates on those are lower then private proxies. So use private.
rotating are good for harvesting, but so are private.
This video should be also useful for harvesting
https://www.youtube.com/watch?v=GadX5AXiW34
RE: expired domain plugin - [email protected] - 02-10-2017
thanks loopline, I appreciate you taking the time to answer the question and video link.
With the harvester tool there is an error screen which tells you that IP has got blocked is there the same sort of thing for domain finder.
Also how does it test if address is available - when I tested edna.edu.au it was the only address I entered - when I used proxy it came up as available but with no proxies it didn't show up. So basically which website banned my IP?
RE: expired domain plugin - loopline - 02-11-2017
There is no test, but its rare that it happens, its not like with google. Its possible that with your local ip your ISP hijacked the request. Try changing your DNS to google DNS as a test.
Its also possible though that the site did block your IP. That would typically not last very long though. I have had exactly zero people ever say they had an issue with a site blocking their ip. Though it is possible if you ran too many connections.
RE: expired domain plugin - scrapefreak - 08-31-2017
Let's say we are scraping a very large website while using less poxies than I should or better yet, no proxies at all. Is there a way to tell if the website banned the ips used for scanning? In my experience, I noticed that the expired domain finder plugin just kept on "working" even if it had been banned long time ago.
RE: expired domain plugin - loopline - 09-06-2017
It will just keep working and getting errors. Scrapebox doesn't try and interpret the errors, it will just keep working thru the requests it needs to make. So just turn down the connections to 1 or 2 and try that.