Class StringUtil
java.lang.Object
opennlp.tools.util.StringUtil
-
Field Summary
Fields -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionstatic voidcomputeShortestEditScript(String wordForm, String lemma, int[][] distance, StringBuffer permutations) Computes the Shortest Edit Script (SES) to convert a word into its lemma.static StringdecodeShortestEditScript(String wordForm, String permutations) Reads the predicted Shortest Edit Script (SES) by a lemmatizer model and applies the permutations to obtain the lemma from thewordForm.static StringgetShortestEditScript(String wordForm, String lemma) static booleanisEmpty(CharSequence theString) static booleanisWhitespace(char charCode) Determines if the specifiedCharacteris a whitespace.static booleanisWhitespace(int charCode) Determines if the specifiedCharacteris a whitespace.static int[][]levenshteinDistance(String wordForm, String lemma) Computes the Levenshtein distance of two strings in a matrix.private static intminimum(int a, int b, int c) Get the minimum of three values.static StringtoLowerCase(CharSequence string) Converts aCharSequenceto lower case, independent of the currentLocaleviaCharacter.toLowerCase(int)which uses mapping information from the UnicodeData file.static CharBuffertoLowerCaseCharBuffer(CharSequence sequence) static StringtoUpperCase(CharSequence string) Converts aCharSequenceto upper case, independent of the currentLocaleviaCharacter.toUpperCase(int)which uses mapping information from the UnicodeData file.
-
Field Details
-
logger
private static final org.slf4j.Logger logger
-
-
Constructor Details
-
StringUtil
public StringUtil()
-
-
Method Details
-
isWhitespace
public static boolean isWhitespace(char charCode) Determines if the specifiedCharacteris a whitespace. A character is considered a whitespace when one of the following conditions is met:- It's a
Character.isWhitespace(int)whitespace. - It's a part of the Unicode Zs category (
Character.SPACE_SEPARATOR).
Character.isWhitespace(int)does not include no-break spaces. In OpenNLP no-break spaces are also considered as white spaces.- Parameters:
charCode- The character to check.- Returns:
trueifcharCoderepresents a white space,falseotherwise.
- It's a
-
isWhitespace
public static boolean isWhitespace(int charCode) Determines if the specifiedCharacteris a whitespace. A character is considered a whitespace when one of the following conditions is met:- Its a
Character.isWhitespace(int)whitespace. - Its a part of the Unicode Zs category (
Character.SPACE_SEPARATOR).
Character.isWhitespace(int)does not include no-break spaces. In OpenNLP no-break spaces are also considered as white spaces.- Parameters:
charCode- An int representation of a character to check.- Returns:
trueifcharCoderepresents a white space,falseotherwise.
- Its a
-
toLowerCase
Converts aCharSequenceto lower case, independent of the currentLocaleviaCharacter.toLowerCase(int)which uses mapping information from the UnicodeData file.- Parameters:
string- TheCharSequenceto transform.- Returns:
- The lower-cased String.
-
toLowerCaseCharBuffer
-
toUpperCase
Converts aCharSequenceto upper case, independent of the currentLocaleviaCharacter.toUpperCase(int)which uses mapping information from the UnicodeData file.- Parameters:
string- TheCharSequenceto transform.- Returns:
- The upper-cased String
-
isEmpty
- Returns:
trueifCharSequence.length()is0ornull, otherwisefalse- Since:
- 1.5.1
-
minimum
private static int minimum(int a, int b, int c) Get the minimum of three values.- Parameters:
a- number ab- number bc- number c- Returns:
- the minimum among the three parameters
a,borc.
-
levenshteinDistance
Computes the Levenshtein distance of two strings in a matrix.Based on this pseudo-code which in turn is based on the paper Wagner, Robert A.; Fischer, Michael J. (1974), "The String-to-String Correction Problem", Journal of the ACM 21 (1): 168-173
- Parameters:
wordForm- The form as input.lemma- The target lemma.- Returns:
- A 2-dimensional Levenshtein distance matrix.
-
computeShortestEditScript
public static void computeShortestEditScript(String wordForm, String lemma, int[][] distance, StringBuffer permutations) Computes the Shortest Edit Script (SES) to convert a word into its lemma. This is based on Chrupala's PhD thesis (2008).- Parameters:
wordForm- The token.lemma- The target lemma.distance- A 2-dimensional Levenshtein distance matrix.permutations- The number of permutations.
-
decodeShortestEditScript
Reads the predicted Shortest Edit Script (SES) by a lemmatizer model and applies the permutations to obtain the lemma from thewordForm.- Parameters:
wordForm- The wordForm as input.permutations- The permutations predicted by the lemmatizer model.- Returns:
- The decoded lemma.
-
getShortestEditScript
-