ProcessData Property

Gets or sets a value indicating whether [process data]. If ProcessData == false, the CrawlRequest was not allowed to DownloadString from the WebClient as the LastModified HttpRequestHeader was older than LastDiscovered in the database.

public bool ProcessData { public get; internal set; }