Internal Static Properties
  CredentialCache | Gets or sets the credential cache. |
Public Instance Constructors
Crawler
|
Overloaded. Initializes a new instance of the Crawler class. |
Public Instance Properties
 CrawlActions | |
| CrawlMode | |
| CrawlRules | |
| Engine | The Engine. |
| Guid | |
| QueryProcessor | The QueryProcessor. (reserved for future use) |
| ReportingManager |
Public Instance Methods
| ClearDisallowedAbsoluteUris | Clears all DisallowedAbsoluteUris on disk. |
| ClearDiscoveries | Clears the Discoveries in RAM and on disk. This method is not thread-safe and should not be called while the Engine is running. |
| ClearPolitenesses | Clears the Politenesses in RAM. This method is not thread-safe and should not be called while the Engine is running. |
| ClearUncrawledCrawlRequests | Clears the UncrawledCrawlRequests in RAM and on disk. This method is not thread-safe and should not be called while the Engine is running. |
| Crawl | Submits a CrawlRequest to the Engine for crawling. If Crawl(CrawlRequest crawlRequest) returns 'false' the Cache is already aware of the Discovery and the Discovery is awaiting processing or has already been processed and was loaded from the database. If a Crawl is resuming, the Discoveries found in the database table 'Discoveries' will be loaded into RAM, allowing arachnode.net to resume the memory state of the crawl. To re-crawl an AbsoluteUri that was already discovered, delete the row from database table 'Discoveries'. To clear all Discoveries, call 'crawler.ClearDiscoveries'. |
| Equals (inherited from Object) | |
| GetHashCode (inherited from Object) | |
| GetType (inherited from Object) | |
| ReinitializeConfiguration | Reinitializes the configuration. Refreshes the Crawler configuration without having to create a new Crawler. |
| ToString (inherited from Object) |
Protected Instance Methods
 Finalize (inherited from Object) | |
| MemberwiseClone (inherited from Object) |
Internal Instance Methods
 LoadCrawlActions | |
| LoadCrawlRules | |
| ProcessException | Throws the exception. |
See Also
Crawler Class | Arachnode.SiteCrawler Namespace