|
||||||||||
PREV NEXT | FRAMES NO FRAMES |
Packages that use Tokenizer | |
---|---|
it.unipi.di.textdb | |
it.unipi.di.util |
Uses of Tokenizer in it.unipi.di.textdb |
---|
Methods in it.unipi.di.textdb with parameters of type Tokenizer | |
---|---|
static TextDB |
BucketedHuffword.build(Tokenizer tokenizer,
String inputfile,
int bucketLen,
PrintStream log)
Compresses the input file with Bucketed Huffword using a set of custom parameters. |
Uses of Tokenizer in it.unipi.di.util |
---|
Classes in it.unipi.di.util that implement Tokenizer | |
---|---|
class |
FixedTokenizer
Splits a text into a list of fixed-size tokens. |
class |
TermTokenizer
Splits a text into a list of tokens, '\n' is added. |
class |
URLTokenizer
A Tokenizer for a list of URLs. |
|
||||||||||
PREV NEXT | FRAMES NO FRAMES |