Class DownloadUtil
java.lang.Object
opennlp.tools.util.DownloadUtil
This class facilitates the downloading of pretrained OpenNLP models.
-
Nested Class Summary
Nested Classes -
Field Summary
Fields -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionprivate static StringbyteArrayToHexString(byte[] bytes) private static StringcalculateSHA512(Path file) private static PathchecksumPathFor(Path model) private static StringdownloadChecksumFile(URL sha512) Retrieves the first line of the published checksum file without modification.static <T extends BaseModel>
TdownloadModel(String language, ModelType modelType, Class<T> type) Triggers a download for the specifiedModelType.static <T extends BaseModel>
TdownloadModel(URL url, Class<T> type) Downloads a model from aURL.(package private) static booleanexistsModel(String language, ModelType modelType) Checks if a model of the specifiedmodelTypehas been downloaded already for a particularlanguage.Lists the models published in the model index.private static Path(package private) static StringindexFileName(String modelPath) Names the local copy of the model index after the last segment of the model path, for exampleud-models-1.3.index.htmlformodels/ud-models-1.3/.private static StringparseChecksum(String checksumFileContent) Extracts the hash from the content of a checksum file, which holds the hash followed by the name of the file it applies to.private static voidstoreChecksumFile(Path model, String checksumFileContent) Stores the checksum file next to the model.private static voidvalidateCachedModel(URL sha512, Path cachedModel) Validates a model that is already present in the download directory.private static voidvalidateModel(URL sha512, Path downloadedModel) Validates a downloaded model against its published SHA-512 checksum and stores the checksum file next to the model, so that later loads of the cached model can be verified without network access.private static voidverifyChecksum(Path model, String expectedChecksum)
-
Field Details
-
logger
private static final org.slf4j.Logger logger -
BASE_URL
-
MODEL_URI_PATH
-
OPENNLP_DOWNLOAD_HOME
- See Also:
-
CHECKSUM_EXTENSION
- See Also:
-
INDEX_EXTENSION
- See Also:
-
TEMP_EXTENSION
- See Also:
-
availableModels
-
-
Constructor Details
-
DownloadUtil
public DownloadUtil()
-
-
Method Details
-
existsModel
Checks if a model of the specifiedmodelTypehas been downloaded already for a particularlanguage.- Parameters:
language- The ISO language code of the requested model.modelType- Thetypeof model.- Returns:
trueif a model exists locally,falseotherwise.- Throws:
IOException- Thrown if IO errors occurred or the computed hash sum of an associated, local model file was incorrect.
-
downloadModel
public static <T extends BaseModel> T downloadModel(String language, ModelType modelType, Class<T> type) throws IOException Triggers a download for the specifiedModelType.- Type Parameters:
T- The generic type which is a subclass ofBaseModel.- Parameters:
language- The ISO language code of the requested model.modelType- Thetypeof model.type- The class of the resulting model.- Returns:
- A model instance of type
T. - Throws:
IOException- Thrown if IO errors occurred or the model is invalid.
-
downloadModel
Downloads a model from aURL.The model is saved to an
.opennlp/directory located in the user's home directory. This directory will be created if it does not already exist. If a model to be downloaded already exists in that directory, the model will not be re-downloaded, but it is verified against its SHA-512 checksum before it is loaded.- Type Parameters:
T- The generic type which is a subclass ofBaseModel.- Parameters:
url- The model'sURL.type- The class of the resulting modelT.- Returns:
- A model instance of type
T. - Throws:
IOException- Thrown if the model cannot be downloaded.
-
getAvailableModels
Lists the models published in the model index. A copy of the last index read is kept in the download directory and used when the index cannot be read. An empty result is not kept, so the next call reads the index again.- Returns:
- The model URLs per language and
ModelType. Empty if neither the index nor its local copy lists models.
-
validateModel
Validates a downloaded model against its published SHA-512 checksum and stores the checksum file next to the model, so that later loads of the cached model can be verified without network access.- Parameters:
sha512- TheURLof the published checksum file.downloadedModel- The model file to check.- Throws:
IOException- Thrown if the checksum could not be retrieved or computed, or if it does not match.
-
validateCachedModel
Validates a model that is already present in the download directory.The checksum file stored by a previous download is used, so no network access is required. If no checksum file is stored, the published checksum is retrieved, verified and stored. If it cannot be retrieved, the model is accepted without verification and a warning is logged.
- Parameters:
sha512- TheURLof the published checksum file.cachedModel- The cached model file to check.- Throws:
IOException- Thrown if the checksum could not be computed or does not match.
-
downloadChecksumFile
Retrieves the first line of the published checksum file without modification.- Parameters:
sha512- TheURLof the published checksum file.- Returns:
- The first line, or
nullif the file is empty. - Throws:
IOException- Thrown if the file could not be read.
-
parseChecksum
Extracts the hash from the content of a checksum file, which holds the hash followed by the name of the file it applies to.- Parameters:
checksumFileContent- The content of the checksum file, may benull.- Returns:
- The hash, or
nullif the content holds none.
-
verifyChecksum
- Throws:
IOException
-
storeChecksumFile
-
checksumPathFor
-
calculateSHA512
- Throws:
IOException
-
byteArrayToHexString
-
indexFileName
Names the local copy of the model index after the last segment of the model path, for exampleud-models-1.3.index.htmlformodels/ud-models-1.3/. The base URL is not part of the name, so hosts that serve the same model path share the copy.- Parameters:
modelPath- The model path, with or without trailing slashes.- Returns:
- The file name.
-
getDownloadHome
-