Looplines Scrapebox List

Post Reply 
 
Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
email scraping
02-16-2017, 08:01 PM
Post: #1
email scraping
I have huge site that has hundreds of thousands of emails that I´d like to scrape.
All the emails are on the surface, so a simple site:domain.com would suffice.
I succeeded scraping a few hundred.

Can someone tell me how many proxies i would need to scrape something like 500.000 pages?

Also how would i best go about in terms of organization?
Angel
Find all posts by this user
Quote this message in a reply
02-17-2017, 07:40 PM
Post: #2
RE: email scraping
You can use the grab mails by crawling site funciton. if you set it at 1 connection you probably don't need any proxies, but I would do 1 proxy for every 1-2 connections.

https://www.youtube.com/watch?v=CykedqJg92w&t=5s
Find all posts by this user
Quote this message in a reply
02-20-2017, 06:37 PM
Post: #3
RE: email scraping
ty loopline,
that works like a charm.

Generally speaking does having more proxies make things faster?
Should i spend more time on harvesting free proxies or invest into
premium proxies?

Is it scraping only inside the URL or also following outside domain links?
Find all posts by this user
Quote this message in a reply
02-20-2017, 07:23 PM
Post: #4
RE: email scraping
IT can, I mean it makes it so the end site doesn't block your ip because your spreading out the requests with proxies. But if the end site server can't handle all the requests you throw at it then its just slowing things down. So "it depends" is about the best answer I can give you.

you don't want to use free proxies for anything except scraping from the search engines. They are built for speed at the cost of accuracy, but with email scraping you want premium proxies or no proxies.

Its only scraping inside the domain for email scraping, its not following outside links.
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump:


User(s) browsing this thread: 1 Guest(s)
Looplines Scrapebox List