06-24-2017, 08:02 PM
hi
i scraped over 30 millions urls from various search engines - once the scrape is complete you are shown a summery screen that shows you how many urls were scraped from the varius engines for the given keywords - - After I clicked the exit button then all of the urls are moved to the scrapebox harvester screen - for some reason its only showing 9 million urls and not the 30 million i I scraped
does scrapebox automatically delete duplicate urls when they come into the harvester or is there another reason for this ?
i scraped over 30 millions urls from various search engines - once the scrape is complete you are shown a summery screen that shows you how many urls were scraped from the varius engines for the given keywords - - After I clicked the exit button then all of the urls are moved to the scrapebox harvester screen - for some reason its only showing 9 million urls and not the 30 million i I scraped
does scrapebox automatically delete duplicate urls when they come into the harvester or is there another reason for this ?