Uses of Package
opennlp.tools.tokenize
-
Packages that use opennlp.tools.tokenize Package Description opennlp.tools.cmdline.parser opennlp.tools.cmdline.tokenizer opennlp.tools.formats Experimental package related to converting various corpora to OpenNLP Format.opennlp.tools.formats.ad Experimental package related to theArvores Deitadas corpus
format.opennlp.tools.formats.brat Experimental package related to the corpus format used by the "brat rapid annotation tool" (brat).opennlp.tools.formats.conllu Experimental package related to the CoNNL-U format.opennlp.tools.formats.convert opennlp.tools.formats.irishsentencebank Experimental package related to the Irish Sentence Bank format.opennlp.tools.formats.letsmt Experimental package related to theletsmt
corpus format.opennlp.tools.formats.masc Experimental package related to theMASC
corpus format.opennlp.tools.formats.muc Experimental package related to theMUC
corpus format.opennlp.tools.sentdetect Package related to identifying sentence boundaries.opennlp.tools.tokenize Contains classes related to finding token or words in a string.opennlp.tools.tokenize.lang opennlp.tools.tokenize.lang.en opennlp.tools.util.featuregen This package contains classes for generating sequence features. -
Classes in opennlp.tools.tokenize used by opennlp.tools.cmdline.parser Class Description Tokenizer The interface for tokenizers, which segment a string into its tokens. -
Classes in opennlp.tools.tokenize used by opennlp.tools.cmdline.tokenizer Class Description TokenizerEvaluationMonitor A marker interface for evaluatingtokenizers
.TokenizerModel TheTokenizerModel
is the model used by a learnableTokenizer
.TokenSample ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats Class Description Detokenizer ADetokenizer
merges tokens back to their detokenized representation.TokenSample ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.ad Class Description TokenSample ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.brat Class Description Tokenizer The interface for tokenizers, which segment a string into its tokens.TokenizerModel TheTokenizerModel
is the model used by a learnableTokenizer
. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.conllu Class Description TokenSample ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.convert Class Description Detokenizer ADetokenizer
merges tokens back to their detokenized representation.TokenSample ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.irishsentencebank Class Description TokenSample ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.letsmt Class Description Detokenizer ADetokenizer
merges tokens back to their detokenized representation. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.masc Class Description TokenSample ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.muc Class Description Tokenizer The interface for tokenizers, which segment a string into its tokens. -
Classes in opennlp.tools.tokenize used by opennlp.tools.sentdetect Class Description Detokenizer ADetokenizer
merges tokens back to their detokenized representation. -
Classes in opennlp.tools.tokenize used by opennlp.tools.tokenize Class Description DetokenizationDictionary DetokenizationDictionary.Operation Detokenizer ADetokenizer
merges tokens back to their detokenized representation.Detokenizer.DetokenizationOperation This enum contains an operation for every token to merge the tokens together to their detokenized form.SimpleTokenizer A basicTokenizer
implementation which performs tokenization using character classes.TokenContextGenerator Interface for context generators required forTokenizerME
.Tokenizer The interface for tokenizers, which segment a string into its tokens.TokenizerEvaluationMonitor A marker interface for evaluatingtokenizers
.TokenizerFactory The factory that providesTokenizer
default implementation and resources.TokenizerModel TheTokenizerModel
is the model used by a learnableTokenizer
.TokenSample ATokenSample
is text with token spans.WhitespaceTokenizer A basicTokenizer
implementation which performs tokenization using white spaces. -
Classes in opennlp.tools.tokenize used by opennlp.tools.tokenize.lang Class Description TokenContextGenerator Interface for context generators required forTokenizerME
. -
Classes in opennlp.tools.tokenize used by opennlp.tools.tokenize.lang.en Class Description TokenSample ATokenSample
is text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.util.featuregen Class Description Tokenizer The interface for tokenizers, which segment a string into its tokens.