Uses of Package
opennlp.tools.tokenize
-
Packages that use opennlp.tools.tokenize Package Description opennlp.tools.cmdline.parser opennlp.tools.cmdline.tokenizer opennlp.tools.formats Experimental package related to converting various corpora to OpenNLP Format.opennlp.tools.formats.ad opennlp.tools.formats.brat opennlp.tools.formats.conllu opennlp.tools.formats.convert opennlp.tools.formats.irishsentencebank opennlp.tools.formats.letsmt opennlp.tools.formats.masc opennlp.tools.formats.muc opennlp.tools.sentdetect Package related to identifying sentece boundries.opennlp.tools.tokenize Contains classes related to finding token or words in a string.opennlp.tools.tokenize.lang opennlp.tools.tokenize.lang.en opennlp.tools.util.featuregen This package contains classes for generating sequence features. -
Classes in opennlp.tools.tokenize used by opennlp.tools.cmdline.parser Class Description Tokenizer The interface for tokenizers, which segment a string into its tokens. -
Classes in opennlp.tools.tokenize used by opennlp.tools.cmdline.tokenizer Class Description TokenizerEvaluationMonitor TokenizerModel TheTokenizerModel
is the model used by a learnableTokenizer
.TokenSample ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats Class Description Detokenizer A Detokenizer merges tokens back to their untokenized representation.TokenSample ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.ad Class Description TokenSample ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.brat Class Description Tokenizer The interface for tokenizers, which segment a string into its tokens.TokenizerModel TheTokenizerModel
is the model used by a learnableTokenizer
. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.conllu Class Description TokenSample ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.convert Class Description Detokenizer A Detokenizer merges tokens back to their untokenized representation.TokenSample ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.irishsentencebank Class Description TokenSample ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.letsmt Class Description Detokenizer A Detokenizer merges tokens back to their untokenized representation. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.masc Class Description TokenSample ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.muc Class Description Tokenizer The interface for tokenizers, which segment a string into its tokens. -
Classes in opennlp.tools.tokenize used by opennlp.tools.sentdetect Class Description Detokenizer A Detokenizer merges tokens back to their untokenized representation. -
Classes in opennlp.tools.tokenize used by opennlp.tools.tokenize Class Description DetokenizationDictionary DetokenizationDictionary.Operation Detokenizer A Detokenizer merges tokens back to their untokenized representation.Detokenizer.DetokenizationOperation This enum contains an operation for every token to merge the tokens together to their detokenized form.SimpleTokenizer Performs tokenization using character classes.TokenContextGenerator Interface forTokenizerME
context generators.Tokenizer The interface for tokenizers, which segment a string into its tokens.TokenizerEvaluationMonitor TokenizerFactory The factory that providesTokenizer
default implementations and resources.TokenizerModel TheTokenizerModel
is the model used by a learnableTokenizer
.TokenSample ATokenSample
is text with token spans.WhitespaceTokenizer This tokenizer uses white spaces to tokenize the input text. -
Classes in opennlp.tools.tokenize used by opennlp.tools.tokenize.lang Class Description TokenContextGenerator Interface forTokenizerME
context generators. -
Classes in opennlp.tools.tokenize used by opennlp.tools.tokenize.lang.en Class Description TokenSample ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.util.featuregen Class Description Tokenizer The interface for tokenizers, which segment a string into its tokens.