+- ScrapeBox Forum (https://www.scrapeboxforum.com)
+-- Forum: ScrapeBox Main Discussion (https://www.scrapeboxforum.com/Forum-scrapebox-main-discussion)
+--- Forum: General ScrapeBox Talk (https://www.scrapeboxforum.com/Forum-general-scrapebox-talk)
+--- Thread: Scrape inconsistent (/Thread-scrape-inconsistent)
Scrape inconsistent - Leatherneck - 01-28-2022
Hello,
I have been setting up the google scholar engine and find when I run the harvester it doesn't provide a consistent number of results.
I can run the same search and it will randomly give me anywhere between 4 and 18 results. My results count is set to 20.
Any suggestions on improving the repeatability?
Thanks, Leatherneck
RE: Scrape inconsistent - loopline - 02-02-2022
how many results are on a page?
It could just be that ips are blocked and pages are being skipped when working on retries.
RE: Scrape inconsistent - Leatherneck - 02-02-2022
(02-02-2022, 11:07 PM)loopline Wrote: how many results are on a page?
It could just be that ips are blocked and pages are being skipped when working on retries.
Google Scholar returns 10 results per page.
So, is it reasonable then to run the same set of footprints several times and then merge and dedupe them to get the most accurate results list?
I'm experimenting to find the place where I am "confident" that I have truly discovered the publically available research reports.
Thanks
RE: Scrape inconsistent - loopline - 02-07-2022
sure, test it and run it various times and see where you get what you need and then go from there.