Load WordPress Sites in as fast as 37ms!

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
*Real* site crawling with Scrapebox?
So I see SB has the site crawler where you can crawl one domain at a time (a deathly slow process)....or you can use link extractor (over and over and over and over) in an inefficient way to grab most links at a time) or you can use the "search google" method to see what pops up....but

Isn't there just like a regular ol' webcrawler?  You give it a list of urls, it spidered through all the links, pulls out the links on a domain, and ding - turkey's done.

Am I missing something obvious? It seems like such an obvious tool that should be included.  Instead, I find myself now poling aroudn the web to find some alternative.

Any ideas?
The link extractor can be just as effecient as this, in fact more so. Because what your asking for could use so much memory it would routinely crash and thus is one of the reasons it no in scrapebox. Where as with the link extractor method this is not the case.

Also you can use the link extractor method with the automator to make it automated.

[-] The following 1 user says Thank You to loopline for this post:
  • DigitalMu

Users browsing this thread: 1 Guest(s)