01-14-2014, 04:53 AM
(01-13-2014, 08:09 PM)deepx Wrote: You were so right!
I'm so sorry for not educating myself enough by watching all of your videos before posting this thread.
I did watch the art of harvesting video and it all looked so simple when your were doing it. I bought my scrapebox license long time ago but I was not using it at the time when I was watching your video.
I didn't thought that my IP was banned because I started using a new VPS server that same day and I only scraped for 1-2 thousands to find some free proxies.
Question: Will I solve my IP block issue by using good proxies? If I buy 10 shared proxies that are being shared between max 3 users, will I be able to scrape for like 10k url's each day without a problem. Or do I need to buy private proxies?
After looking at the art of harvesting video where I saw the Google maps in custom harvester settings, I had to give it a try.
Why? Just because of the irrelevancy of highly targeted results in Google search.
If I would let's say do a search for the term "web design company in London, UK", even the top 100 results are not relevant enough. And I would always get sites like Wikipedia, LinkedIn and others which are hard to filter out.
I figured out how to avoid this filter problem by scraping Facebook pages for the same term. None of the email harvesters that I tried were able to pull out the emails efficiently from FB pages.
The email is clearly visible in the info section of each facebook page (link/info) but the harvester is just not grabbing it.
That's why I need Google Maps results. They are highly relevant, even the first few thousands. You told me to use the normal search engine but that won't work.
If I do a search for site:maps.google.com "dentist" "london", I don't get the same results as I would get with maps.google.com search engine. The results are really weird and not that related to each other.
I hope that I'll be able to make the custom harvester work with Google Maps by using shared proxies.
Would you mind sharing your opinion on this? How would you search for such keywords? I'm sure there are some special tricks that you use but many are not aware of.
Thank you for taking the time to answer all these questions. Your goodwill and professional customer support are easily recognizable online.
Glad you got the ip ban figured out, google is ban happy these days.
I would definitely just use the google maps option, its going to be the best bet.
You can solve your IP ban issues with proxies. As for the shared proxies, I use a good deal of shared proxies. If you have a good provider and you tell them what your using them for, scraping google, they will try and match you up with people not using them for that. Buyproxies.org has always been great for this with me, but I have others that do good as well.
Else private proxies solves the issue as you are the only one using them.
You should watch the how to safely harvest google with private proxies. http://www.youtube.com/watch?v=Ri6cci288vk