+- ScrapeBox Forum (https://www.scrapeboxforum.com)
+-- Forum: ScrapeBox Main Discussion (https://www.scrapeboxforum.com/Forum-scrapebox-main-discussion)
+--- Forum: General ScrapeBox Talk (https://www.scrapeboxforum.com/Forum-general-scrapebox-talk)
+--- Thread: scrape for news sites (/Thread-scrape-for-news-sites)
scrape for news sites - blitzme - 03-27-2018
Hello, i just wondering, how can i scrape only news website? I have done this previously, but i think google make changes and i dont know now, how to create url that scrape news sites after keywords ?
RE: scrape for news sites - loopline - 03-28-2018
You want to scrape from google news specifically?
I tested and it works if you just go to settings >> harvester engine configuration >> and then edit the google engine.
Just add
&tbm=nws
to the end of the url string. Then just give it a unique engine name and add it as a new engine.
RE: scrape for news sites - blitzme - 03-30-2018
Yes, thanks thats worked. Just wondering about news sites that has disqus to coments. Could you help with footprint ?
RE: scrape for news sites - loopline - 03-30-2018
here is a video on how to build footprints and it links to another video (in the description) that is also on building footprints
https://www.youtube.com/watch?v=sAf-h9FisGU&t=3s
RE: scrape for news sites - blitzme - 03-31-2018
After added this &tbm=nws , when scraping with footprint it has 0 results, but working with keywords(only). Any idea ?
RE: scrape for news sites - loopline - 04-01-2018
If you go to that same news results pages in google and then put in your footprint, does it still give results? Google news may work differently with footprints. I mean scrapebox is feeding google what you feed it, just like a browser, but I suspect that google literally is returning zero results for your footprint. Because its pass or fail, keywords/footprints etc... they either all work or none work, has been my experience anyway.