07-11-2014, 04:26 AM
Hi,
How should I set my custom harvester to
1. run scrapping initially by loading list of proxies
2. automatically load new proxy list from the same file above (the file will be updated every 10 minutes) on the fly?
Another question: To scrape on engines other than google, we should use different footprints right? Or we just simple operator like "powered by" "keyword" something like that? I want to try it because scrapping other engines maybe easier then google.
Thanks alot
How should I set my custom harvester to
1. run scrapping initially by loading list of proxies
2. automatically load new proxy list from the same file above (the file will be updated every 10 minutes) on the fly?
Another question: To scrape on engines other than google, we should use different footprints right? Or we just simple operator like "powered by" "keyword" something like that? I want to try it because scrapping other engines maybe easier then google.
Thanks alot