it.unipi.di.util
Interface Tokenizer

All Known Implementing Classes:
FixedTokenizer, TermTokenizer, URLTokenizer

public interface Tokenizer

Basic methods to implement for a brand-new Tokenizer.

Author:
Claudio Corsi, Paolo Ferragina

Method Summary
 void close()
           
 String next()
           
 void reset()
           
 

Method Detail

next

String next()
            throws IOException
Throws:
IOException

close

void close()
           throws IOException
Throws:
IOException

reset

void reset()
           throws IOException
Throws:
IOException