|
||||||||||
PREV NEXT | FRAMES NO FRAMES |
Packages that use Tokenizer | |
---|---|
it.unipi.di.textdb | |
it.unipi.di.tokenizer |
Uses of Tokenizer in it.unipi.di.textdb |
---|
Methods in it.unipi.di.textdb with parameters of type Tokenizer | |
---|---|
static TextDB |
BucketedHuffword.build(Tokenizer tokenizer,
String inputfile,
String outfile,
int bucketLen,
PrintStream log)
Compresses the input file with Bucketed Huffword using a set of custom parameters. |
static TextDB |
RSHuffword.build(Tokenizer tokenizer,
String inputfile,
String outfile,
PrintStream log,
boolean withFields,
char separator)
Compress the input file with the bucketed huffword technique using customized parameters. |
Uses of Tokenizer in it.unipi.di.tokenizer |
---|
Classes in it.unipi.di.tokenizer that implement Tokenizer | |
---|---|
class |
FixedTokenizer
Splits a text into a list of fixed-size tokens. |
class |
TermTokenizer
Splits a text into a list of tokens, '\n' is added. |
class |
URLTokenizer
A Tokenizer for a list of URLs. |
|
||||||||||
PREV NEXT | FRAMES NO FRAMES |