Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Spoofing a header during scraping?
Hi LoopLine and others. Wondering here if it is possible to spoof a header (referrer) while scraping? I hit a site I really need to scrape but the page requires a referrer. It shows 404 if you trying to access url without referrer.

How to go about that?
Maybe, using what function? Like the custom harvester will let you add whatever header data you want, but not all functions do.
I was scraping expired domains with custom data grabber (right side menu in main window option GRAB)
The custom data scraper lets you change user agents, but not other header data like referrer.
[-] The following 1 user says Thank You to loopline for this post:
  • MichaelA2014

Users browsing this thread: 1 Guest(s)