Class MascSentence
java.lang.Object
opennlp.tools.util.Span
opennlp.tools.formats.masc.MascSentence
- All Implemented Interfaces:
Serializable, Comparable<Span>
- See Also:
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionprivate static final recordA helper class to extract the extract a quark from the corpus file even if it is beyond the bounds of the sentence. -
Field Summary
FieldsModifier and TypeFieldDescriptionprivate static final org.slf4j.Loggerprivate static final longprivate final String -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescription(package private) booleanaddNamedEntities(Map<Integer, String> entityIDtoEntityType, Map<Integer, List<Integer>> entityIDsToTokens) Add the named entity annotation to the tokenized sentencegetTags()Retrieves the boundaries of individual tokens.(package private) booleantokenizePenn(Map<Integer, int[]> tokenToQuarks, Map<Integer, int[]> quarkToTokens, Map<Integer, String> tokenToBase, Map<Integer, String> tokenToTag) Add the Penn tokenization and POS tagging to the sentence.Methods inherited from class Span
compareTo, contains, contains, crosses, equals, getCoveredText, getEnd, getProb, getStart, getType, hashCode, intersects, length, spansToStrings, spansToStrings, startsWith, toString, trim
-
Field Details
-
serialVersionUID
private static final long serialVersionUID- See Also:
-
logger
private static final org.slf4j.Logger logger -
allDocumentWords
-
text
-
words
-
wordsById
-
sentenceTokens
-
tokensById
-
namedEntities
-
-
Constructor Details
-
MascSentence
public MascSentence(int s, int e, String text, List<MascWord> sentenceQuarks, List<MascWord> allQuarks) Initializes aMascSentencecontaining its associated text and quarks- Parameters:
s- Start of the sentence within the corpus filee- End of the sentence within the corpus filetext- The reference to text of the corpus filesentenceQuarks- The quarks found in that sentenceallQuarks- The reference to a list of all quarks in the file
-
-
Method Details
-
tokenizePenn
boolean tokenizePenn(Map<Integer, int[]> tokenToQuarks, Map<Integer, throws IOExceptionint[]> quarkToTokens, Map<Integer, String> tokenToBase, Map<Integer, String> tokenToTag) Add the Penn tokenization and POS tagging to the sentence.- Parameters:
tokenToQuarks- A map from token ID to quarks in that token.quarkToTokens- A map of quark IDs and the token IDs containing that quark.tokenToBase- Token ID to the token base.tokenToTag- Token ID to the POS tag.- Returns:
trueif no issue encountered,falseif tokens cross sentence boundaries.- Throws:
IOException- Thrown if IO errors occurred.
-
addNamedEntities
boolean addNamedEntities(Map<Integer, String> entityIDtoEntityType, Map<Integer, List<Integer>> entityIDsToTokens) throws IOExceptionAdd the named entity annotation to the tokenized sentence- Parameters:
entityIDtoEntityType- Maps the named entity ID to its typeentityIDsToTokens- A list of tokens covered by each named entity- Returns:
trueif all went well,falseif named entities overlap.- Throws:
IOException- Thrown if IO errors occurred.
-
getNamedEntities
- Returns:
- Retrieves the
named entities, e.g.Span(1,3, "org")for tokens [1,3).
-
getSentDetectText
- Returns:
- Retrieves text of the sentence as defined by the sentence segmentation annotation.
-
getTokenText
- Returns:
- Retrieves text of the sentence as defined by the tokens in it.
-
getTokenStrings
-
getTokensSpans
-
getTags
- Returns:
- Get the (individual) tags of tokens in the sentence.
- Throws:
IOException- Thrown if used on an un-tokenized sentence.
-