01-22-2012, 11:15 AM
(01-21-2012, 08:49 PM)mindmaster Wrote: Are you sure your list has 600.000 urls, because the limit is 999.999 urls that can be added in the 'url's harvester' section.
However I suggest you split your file in 50.000 or 100.000 urls and work with each one.
Here's how:
Go to addons and use the 'dupe remover'.
On that new window, you will see that this addon has 3 features:
*First is to merge multiple files in one.
*Second is to remove duplicate urls or duplicate domains from a given list.
***And third feature is to split a file.
So load your list there, select how many lines (urls) you want each new file to have and hit run.
Thanks Mindmaster, for your helpful solution.