The Blueprint Training

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Scraping a directory probem
#1
Hi

I am a newbie so please excuse me if Im missing something simple.
Last week I began successfully scraping a directory for emails (thomsonlocal.com). I first scraped the directory for thousands of internal pages then scraped those pages for external links. Then scraped those links for emails.

This process worked fine and i managed to get quite a few emails. Then suddenly I found that I could no longer get outbound links from the thomson local pages. I received the error 502.

I tried changing my proxies and this time including a 30 sec delay with only 1 connection, but i receive the same error. I have changed my proxies numerous times now informing buyproxies.com of my problem, they have told me that as long as the proxies are google passed, this should be ok.

I would really appreciate any help in this as I am completely stuck.

I look forward to hearing from you.
Reply


Messages In This Thread
Scraping a directory probem - by Deep SEO - 07-26-2015, 09:39 AM
RE: Scraping a directory probem - by loopline - 07-26-2015, 06:42 PM
RE: Scraping a directory probem - by Deep SEO - 07-26-2015, 10:27 PM
RE: Scraping a directory probem - by loopline - 07-31-2015, 07:24 PM



Users browsing this thread: 1 Guest(s)

Looplines Scrapebox List