Class ADChunkSampleStream
java.lang.Object
opennlp.tools.formats.ad.ADChunkSampleStream
- All Implemented Interfaces:
AutoCloseable, ObjectStream<ChunkSample>
Parser for Floresta Sita(c)tica Arvores Deitadas corpus, output to for the
Portuguese Chunker training.
The heuristic to extract chunks where based o paper 'A Machine Learning
Approach to Portuguese Clause Identification', (Eraldo Fernandes, Cicero
Santos and Ruy Milidiú).
Data can be found on this web site.
Information about the format:
Susana Afonso.
"Árvores deitadas: Descrição do formato e das opções de análise na Floresta Sintáctica".
12 de Fevereiro de 2006.
Detailed info about the NER tagset.
Note: Do not use this class, internal use only!
-
Field Summary
FieldsModifier and TypeFieldDescriptionprotected final ObjectStream<ADSentenceStream.Sentence> private intprivate intstatic final Stringprivate int -
Constructor Summary
ConstructorsConstructorDescriptionADChunkSampleStream(InputStreamFactory in, String charsetName) Instantiates aADChunkSampleStreamstream from anInputStreamFactory.ADChunkSampleStream(ObjectStream<String> lineStream) Instantiates aADChunkSampleStreamstream fromObjectStream<String>, that could be aPlainTextByLineStreamobject. -
Method Summary
Modifier and TypeMethodDescriptionvoidclose()Closes theObjectStreamand releases all allocated resources.static StringconvertFuncTag(String t, boolean useCGTags) protected Stringprotected Stringprotected StringgetPhraseTagFromPosTag(String functionalTag) protected booleanprotected booleanprotected voidprocessLeaf(ADSentenceStream.SentenceParser.Leaf leaf, boolean isIntermediate, String phraseTag, List<String> sentence, List<String> tags, List<String> target) Adds a leaf to the sample.private voidprocessNode(ADSentenceStream.SentenceParser.Node node, List<String> sentence, List<String> tags, List<String> target, String inheritedTag) protected voidprocessRoot(ADSentenceStream.SentenceParser.Node root, List<String> sentence, List<String> tags, List<String> target) read()Returns the nextTobject.voidreset()Repositions the stream at the beginning and the previously seen object sequence will be repeated exactly.voidsetEnd(int aEnd) voidsetStart(int aStart)
-
Field Details
-
adSentenceStream
-
start
private int start -
end
private int end -
index
private int index -
OTHER
- See Also:
-
-
Constructor Details
-
ADChunkSampleStream
Instantiates aADChunkSampleStreamstream fromObjectStream<String>, that could be aPlainTextByLineStreamobject.- Parameters:
lineStream- AnObjectStream<String>as input.
-
ADChunkSampleStream
Instantiates aADChunkSampleStreamstream from anInputStreamFactory.- Parameters:
in- TheInputStreamFactoryfor the corpus.charsetName- Thecharsetto use for reading of the corpus.- Throws:
IOException
-
-
Method Details
-
read
Description copied from interface:ObjectStreamReturns the nextTobject. Calling this method repeatedly until it returnsnullwill return each object from the underlying source exactly once.- Specified by:
readin interfaceObjectStream<ChunkSample>- Returns:
- The next object or
nullto signal that the stream is exhausted. - Throws:
IOException- Thrown if there is an error during reading.
-
processRoot
-
processNode
-
processLeaf
protected void processLeaf(ADSentenceStream.SentenceParser.Leaf leaf, boolean isIntermediate, String phraseTag, List<String> sentence, List<String> tags, List<String> target) Adds a leaf to the sample. The tag is the functional tag of the leaf; a leaf without one, such as punctuation, is tagged with its lexeme, asADPOSSampleStreamdoes.- Parameters:
leaf- The leaf.isIntermediate-trueif the leaf continues the chunk of the leaf before it.phraseTag- The chunk tag of the phrase the leaf belongs to, orOTHER.sentence- The tokens read so far.tags- The tags read so far.target- The chunk tags read so far.
-
getPhraseTagFromPosTag
-
convertFuncTag
-
getChunkTag
-
getChunkTag
-
setStart
public void setStart(int aStart) -
setEnd
public void setEnd(int aEnd) -
reset
Description copied from interface:ObjectStreamRepositions the stream at the beginning and the previously seen object sequence will be repeated exactly. This method can be used to re-read the stream if multiple passes over the objects are required.The implementation of this method is optional.
- Specified by:
resetin interfaceObjectStream<ChunkSample>- Throws:
IOException- Thrown if there is an error during resetting the stream.UnsupportedOperationException- Thrown if thereset()is not supported. By default, this is the case.
-
close
Description copied from interface:ObjectStreamCloses theObjectStreamand releases all allocated resources. After close was called, it's not allowed to callObjectStream.read()orObjectStream.reset().- Specified by:
closein interfaceAutoCloseable- Specified by:
closein interfaceObjectStream<ChunkSample>- Throws:
IOException- Thrown if there is an error during closing the stream.
-
isIncludePunctuations
protected boolean isIncludePunctuations() -
isIntermediate
-