Class TokSpanEventStream
- All Implemented Interfaces:
AutoCloseable, ObjectStream<Event>
-
Field Summary
FieldsModifier and TypeFieldDescriptionprivate final Patternprivate final TokenContextGeneratorprivate static final org.slf4j.Loggerprivate final boolean -
Constructor Summary
ConstructorsConstructorDescriptionTokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics) Initializes a new event stream based on the data stream using aTokenContextGeneratorthat relies on aDefaultTokenContextGenerator.TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, Pattern alphaNumeric, TokenContextGenerator cg) Initializes a new event stream based on the data stream using aTokenContextGenerator.TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, TokenContextGenerator cg) Initializes a new event stream based on the data stream using aTokenContextGenerator. -
Method Summary
Modifier and TypeMethodDescriptioncreateEvents(TokenSample tokenSample) Adds training events to the event stream for each of the specifiedsample.Methods inherited from class AbstractEventStream
close, read, reset
-
Field Details
-
logger
private static final org.slf4j.Logger logger -
cg
-
skipAlphaNumerics
private final boolean skipAlphaNumerics -
alphaNumeric
-
-
Constructor Details
-
TokSpanEventStream
public TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, Pattern alphaNumeric, TokenContextGenerator cg) Initializes a new event stream based on the data stream using aTokenContextGenerator.- Parameters:
tokenSamples- Thedata streamfor this event stream.skipAlphaNumerics- Whether alphanumerics are skipped, or not.alphaNumeric- A custom alphanumericPatternornull. Default is:"^[A-Za-z0-9]+$", provided byFactory.DEFAULT_ALPHANUMERIC.cg- ATokenContextGeneratorwhich should be used for the event streamd.
-
TokSpanEventStream
public TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, TokenContextGenerator cg) Initializes a new event stream based on the data stream using aTokenContextGenerator.- Parameters:
tokenSamples- Thedata streamfor this event stream.skipAlphaNumerics- Whether alphanumerics are skipped, or not.cg- ATokenContextGeneratorwhich should be used for the event streamd.
-
TokSpanEventStream
Initializes a new event stream based on the data stream using aTokenContextGeneratorthat relies on aDefaultTokenContextGenerator.- Parameters:
tokenSamples- Thedata streamfor this event stream.skipAlphaNumerics- Whether alphanumerics are skipped, or not.
-
-
Method Details
-
createEvents
Adds training events to the event stream for each of the specifiedsample.- Specified by:
createEventsin classAbstractEventStream<TokenSample>- Parameters:
tokenSample- character offsets into the specified text.- Returns:
- An
Iteratorfor texteventsrepresenting thetokenSample.
-