Class DownloadUtil

java.lang.Object
opennlp.tools.util.DownloadUtil

public class DownloadUtil extends Object
This class facilitates the downloading of pretrained OpenNLP models.
  • Field Details

  • Constructor Details

    • DownloadUtil

      public DownloadUtil()
  • Method Details

    • existsModel

      static boolean existsModel(String language, ModelType modelType) throws IOException
      Checks if a model of the specified modelType has been downloaded already for a particular language.
      Parameters:
      language - The ISO language code of the requested model.
      modelType - The type of model.
      Returns:
      true if a model exists locally, false otherwise.
      Throws:
      IOException - Thrown if IO errors occurred or the computed hash sum of an associated, local model file was incorrect.
    • downloadModel

      public static <T extends BaseModel> T downloadModel(String language, ModelType modelType, Class<T> type) throws IOException
      Triggers a download for the specified ModelType.
      Type Parameters:
      T - The generic type which is a subclass of BaseModel.
      Parameters:
      language - The ISO language code of the requested model.
      modelType - The type of model.
      type - The class of the resulting model.
      Returns:
      A model instance of type T.
      Throws:
      IOException - Thrown if IO errors occurred or the model is invalid.
    • downloadModel

      public static <T extends BaseModel> T downloadModel(URL url, Class<T> type) throws IOException
      Downloads a model from a URL.

      The model is saved to an .opennlp/ directory located in the user's home directory. This directory will be created if it does not already exist. If a model to be downloaded already exists in that directory, the model will not be re-downloaded, but it is verified against its SHA-512 checksum before it is loaded.

      Type Parameters:
      T - The generic type which is a subclass of BaseModel.
      Parameters:
      url - The model's URL.
      type - The class of the resulting model T.
      Returns:
      A model instance of type T.
      Throws:
      IOException - Thrown if the model cannot be downloaded.
    • getAvailableModels

      public static Map<String, Map<ModelType, String>> getAvailableModels()
      Lists the models published in the model index. A copy of the last index read is kept in the download directory and used when the index cannot be read. An empty result is not kept, so the next call reads the index again.
      Returns:
      The model URLs per language and ModelType. Empty if neither the index nor its local copy lists models.
    • validateModel

      private static void validateModel(URL sha512, Path downloadedModel) throws IOException
      Validates a downloaded model against its published SHA-512 checksum and stores the checksum file next to the model, so that later loads of the cached model can be verified without network access.
      Parameters:
      sha512 - The URL of the published checksum file.
      downloadedModel - The model file to check.
      Throws:
      IOException - Thrown if the checksum could not be retrieved or computed, or if it does not match.
    • validateCachedModel

      private static void validateCachedModel(URL sha512, Path cachedModel) throws IOException
      Validates a model that is already present in the download directory.

      The checksum file stored by a previous download is used, so no network access is required. If no checksum file is stored, the published checksum is retrieved, verified and stored. If it cannot be retrieved, the model is accepted without verification and a warning is logged.

      Parameters:
      sha512 - The URL of the published checksum file.
      cachedModel - The cached model file to check.
      Throws:
      IOException - Thrown if the checksum could not be computed or does not match.
    • downloadChecksumFile

      private static String downloadChecksumFile(URL sha512) throws IOException
      Retrieves the first line of the published checksum file without modification.
      Parameters:
      sha512 - The URL of the published checksum file.
      Returns:
      The first line, or null if the file is empty.
      Throws:
      IOException - Thrown if the file could not be read.
    • parseChecksum

      private static String parseChecksum(String checksumFileContent)
      Extracts the hash from the content of a checksum file, which holds the hash followed by the name of the file it applies to.
      Parameters:
      checksumFileContent - The content of the checksum file, may be null.
      Returns:
      The hash, or null if the content holds none.
    • verifyChecksum

      private static void verifyChecksum(Path model, String expectedChecksum) throws IOException
      Throws:
      IOException
    • storeChecksumFile

      private static void storeChecksumFile(Path model, String checksumFileContent)
      Stores the checksum file next to the model. A failure is only logged; the next load then retrieves the published checksum again.
      Parameters:
      model - The model file.
      checksumFileContent - The content of the checksum file.
    • checksumPathFor

      private static Path checksumPathFor(Path model)
    • calculateSHA512

      private static String calculateSHA512(Path file) throws IOException
      Throws:
      IOException
    • byteArrayToHexString

      private static String byteArrayToHexString(byte[] bytes)
    • indexFileName

      static String indexFileName(String modelPath)
      Names the local copy of the model index after the last segment of the model path, for example ud-models-1.3.index.html for models/ud-models-1.3/. The base URL is not part of the name, so hosts that serve the same model path share the copy.
      Parameters:
      modelPath - The model path, with or without trailing slashes.
      Returns:
      The file name.
    • getDownloadHome

      private static Path getDownloadHome()