+- ScrapeBox Forum (https://www.scrapeboxforum.com)
+-- Forum: ScrapeBox Main Discussion (https://www.scrapeboxforum.com/Forum-scrapebox-main-discussion)
+--- Forum: General ScrapeBox Talk (https://www.scrapeboxforum.com/Forum-general-scrapebox-talk)
+--- Thread: Scraping Google News not returning results in scrapebox but browser works (/Thread-scraping-google-news-not-returning-results-in-scrapebox-but-browser-works)
Scraping Google News not returning results in scrapebox but browser works - boss4all - 11-06-2020
Hey All,
I have been trying to configure Scrapebox to pull results from Google news. I found a previous thread that helped me update my query string but it is still not working within Scrapebox.
Additional Query String
&tbm=nws
Full Query string
https://www.google.com/search?complete=0&hl=en&q={KEYWORD}&num=100&start={PAGENUM}&filter=0&&tbs=qdr:y&tbm=nws
If you put the full query string into Chrome it works as expected. I see Google news stories for the last year. However, when I run harvester with those settings it returns no results. At the same time, I run a general "Google" query and the results return as expected. I am running the latest version of Scrapebox and default engines from the server. I have attached a screenshot of my harvester settings. Any help would be greatly appreciated, thank you!
RE: Scraping Google News not returning results in scrapebox but browser works - boss4all - 11-09-2020
Hot dang I figured it out by learning more about scrapebox's HTML markers. For those who run into a similar problem, this is easily fixed. Simply add the following to the "Just before the URL" field.
<a style="text-decoration:none;display:block" href="
You will also need to add the following to your query string to only search Google news.
https://www.google.com/search?complete=0&hl=en&q={KEYWORD}&num=100&start={PAGENUM}&filter=0&&tbs=qdr:y&tbm=nws
RE: Scraping Google News not returning results in scrapebox but browser works - loopline - 11-12-2020
Sweet, glad you sorted it!
RE: Scraping Google News not returning results in scrapebox but browser works - Nosh - 08-13-2022
Hi everybody,
I just found this thread because I want to do the same, but I does not working my case.
The string:
https://www.google.com/search?complete=0&hl=en&q=%7BKEYWORD%7D&num=100&start=%7BPAGENUM%7D&filter=0&&tbs=qdr:y&tbm=nws
Just before the URL:
<a style="text-decoration:none;display:block" href="
RE: Scraping Google News not returning results in scrapebox but browser works - Nosh - 08-14-2022
Ok, fixed this with
https://www.google.es/search?complete=0&hl=es&q=%7BKEYWORD%7D&num=100&start=%7BPAGENUM%7D&filter=0&&tbs=qdr:y&tbm=nws
Just before URL:
<a jsname="YKoRaf" class="WlydOe" href="
I have now another strange problem. All keywords are in Spanish but all results are in English. At first I thought it would be due to the US Proxies, but if I switch to "normal" Google (not news) I get Spanish results.
What happened ?
RE: Scraping Google News not returning results in scrapebox but browser works - Nosh - 08-15-2022
Just found it by myself. Thanks anyway
RE: Scraping Google News not returning results in scrapebox but browser works - loopline - 08-20-2022
Glad you got it!