01-05-2020, 09:26 PM
So I see SB has the site crawler where you can crawl one domain at a time (a deathly slow process)....or you can use link extractor (over and over and over and over) in an inefficient way to grab most links at a time) or you can use the "search google" method to see what pops up....but
Isn't there just like a regular ol' webcrawler? You give it a list of urls, it spidered through all the links, pulls out the links on a domain, and ding - turkey's done.
Am I missing something obvious? It seems like such an obvious tool that should be included. Instead, I find myself now poling aroudn the web to find some alternative.
Any ideas?
Thanks!!
Isn't there just like a regular ol' webcrawler? You give it a list of urls, it spidered through all the links, pulls out the links on a domain, and ding - turkey's done.
Am I missing something obvious? It seems like such an obvious tool that should be included. Instead, I find myself now poling aroudn the web to find some alternative.
Any ideas?
Thanks!!