Uses of Interface
it.unipi.di.util.Tokenizer

Packages that use Tokenizer
it.unipi.di.textdb   
it.unipi.di.util   
 

Uses of Tokenizer in it.unipi.di.textdb
 

Methods in it.unipi.di.textdb with parameters of type Tokenizer
static TextDB BucketedHuffword.build(Tokenizer tokenizer, String inputfile, int bucketLen, PrintStream log)
          Compresses the input file with Bucketed Huffword using a set of custom parameters.
 

Uses of Tokenizer in it.unipi.di.util
 

Classes in it.unipi.di.util that implement Tokenizer
 class FixedTokenizer
          Splits a text into a list of fixed-size tokens.
 class TermTokenizer
          Splits a text into a list of tokens, '\n' is added.
 class URLTokenizer
          A Tokenizer for a list of URLs.