Submits a CrawlRequest to the Engine for crawling. If Crawl(CrawlRequest crawlRequest) returns 'false' the Cache is already aware of the Discovery and the Discovery is awaiting processing or has already been processed and was loaded from the database. If a Crawl is resuming, the Discoveries found in the database table 'Discoveries' will be loaded into RAM, allowing arachnode.net to resume the memory state of the crawl. To re-crawl an AbsoluteUri that was already discovered, delete the row from database table 'Discoveries'. To clear all Discoveries, call 'crawler.ClearDiscoveries'.
Crawler Class | Arachnode.SiteCrawler Namespace