hu.juranyi.zsolt.pubsearch.crawl

Class PubListHTMLCrawler

java.lang.Object

java.lang.Thread

hu.juranyi.zsolt.pubsearch.crawl.ACrawler

hu.juranyi.zsolt.pubsearch.crawl.PubListHTMLCrawler

All Implemented Interfaces:

java.lang.Runnable

public class PubListHTMLCrawler
extends ACrawler

Parses a HTML block as a publication list. Extracts publication data or extracts links and starts PubPageCrawlers - depends on the PDatabase object, then stores the grabbed publications.

Nested Class Summary

Nested classes/interfaces inherited from class java.lang.Thread

java.lang.Thread.State, java.lang.Thread.UncaughtExceptionHandler

Field Summary

Fields inherited from class hu.juranyi.zsolt.pubsearch.crawl.ACrawler

crawlers, sched, time

Fields inherited from class java.lang.Thread

MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY

Constructor Summary

Constructors

Constructor and Description
PubListHTMLCrawler(PDatabase pdb, java.lang.String html, int transLev, boolean refPubMode) Sets up the publication list crawling.

Method Summary

Methods

Modifier and Type	Method and Description
protected void	crawl() Crops list block, and parses its content.
java.util.List<Publication>	getPublications()
static void	setThreadLimit(int threadLimit)

Methods inherited from class hu.juranyi.zsolt.pubsearch.crawl.ACrawler

getTime, interrupt, launch, run, scheduleCrawlers, waitForCrawlers

Methods inherited from class java.lang.Thread

activeCount, checkAccess, clone, countStackFrames, currentThread, destroy, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, stop, suspend, toString, yield

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

PubListHTMLCrawler

public PubListHTMLCrawler(PDatabase pdb,
                  java.lang.String html,
                  int transLev,
                  boolean refPubMode)

Sets up the publication list crawling.

Parameters:

pdb - PDatabase object which contains information for database specific crawling.

html - The HTML block containing the publication list.

transLev - 0: only search results, 1: referrer publications also 2: referrer of referrers also will be grabbed.

refPubMode - If true, it handles the list as list of referring publications which may need different patterns to be used.

Method Detail

setThreadLimit

public static void setThreadLimit(int threadLimit)

getPublications

public java.util.List<Publication> getPublications()

crawl

protected void crawl()

Crops list block, and parses its content. Extracts data or link and starts a PubPageCrawler if needed, then stores grabbed publications.

Specified by:

crawl in class ACrawler