arachnode.net

Cache.AddCrawlRequestToBeCrawled(PriorityQueue, CrawlRequest, Boolean, ArachnodeDAO, Boolean) Method

When used publicly, this method is used to insert CrawlRequests into the main CrawlRequest PriorityQueue. When used internally, this method is used by Crawls to insert Files and Images into their own PriorityQueues. When crawling, Crawls prioritize Files and Images above HyperLinks. This method allows the Cache to manage the Discovery associated with the CrawlRequest.

internal static bool AddCrawlRequestToBeCrawled(
   PriorityQueue<CrawlRequestpriorityQueue,
   CrawlRequest crawlRequest,
   bool isCrawlRequestFromDatabase,
   ArachnodeDAO arachnodeDAO,
   bool lockPriorityQueue
);

Parameters

priorityQueue
The priority queue.
crawlRequest
The crawl request.
isCrawlRequestFromDatabase
arachnodeDAO
The arachnode DAO.
lockPriorityQueue
Crawls won't need to lock access their own PriorityQueues.

Return Value

Was the CrawlRequest added to the PriorityQueue to be crawled?

See Also

Cache Class | Arachnode.SiteCrawler.Core Namespace | Cache.AddCrawlRequestToBeCrawled Overload List