Create/append to a potentially Seek-Optimized ZIP file (SOZip)

addFilesInZip() will create new or open existing ZIP file, and add one or more compressed files potentially using the seek optimization extension. This function is basically a wrapper for CPLAddFileInZip() in the GDAL Common Portability Library, but optionally creates a new ZIP file first (with CPLCreateZip()). It provides a subset of functionality in the GDAL sozip command-line utility (https://gdal.org/programs/sozip.html). Requires GDAL >= 3.7.

Usage

addFilesInZip(
  zip_file,
  add_files,
  overwrite = FALSE,
  full_paths = TRUE,
  sozip_enabled = NULL,
  sozip_chunk_size = NULL,
  sozip_min_file_size = NULL,
  num_threads = NULL,
  content_type = NULL,
  quiet = FALSE
)

Arguments

zip_file: Filename of the ZIP file. Will be created if it does not exist or if overwrite = TRUE. Otherwise will append to an existing file.
add_files: Character vector of one or more input filenames to add.
overwrite: Logical scalar. Overwrite the target zip file if it already exists.
full_paths: Logical scalar. By default, the full path will be stored (relative to the current directory). FALSE to store just the name of a saved file (drop the path).
sozip_enabled: String. Whether to generate a SOZip index for the file. One of "AUTO" (the default), "YES" or "NO" (see Details).
sozip_chunk_size: The chunk size for a seek-optimized file. Defaults to 32768 bytes. The value is specified in bytes, or K and M suffix can be used respectively to specify a value in kilo-bytes or mega-bytes. Will be coerced to string.
sozip_min_file_size: The minimum file size to decide if a file should be seek-optimized, in sozip_enabled="AUTO" mode. Defaults to 1 MB byte. The value is specified in bytes, or K, M or G suffix can be used respectively to specify a value in kilo-bytes, mega-bytes or giga-bytes. Will be coerced to string.
num_threads: Number of threads used for SOZip generation. Defaults to "ALL_CPUS" or specify an integer value (coerced to string).
content_type: String Content-Type value for the file. This is stored as a key-value pair in the extra field extension 'KV' (0x564b) dedicated to storing key-value pair metadata.
quiet: Logical scalar. TRUE for quiet mode, no progress messages emitted. Defaults to FALSE.

Value

Logical indicating success (invisible TRUE). An error is raised if the operation fails.

Details

A Seek-Optimized ZIP file (SOZip) contains one or more compressed files organized and annotated such that a SOZip-aware reader can perform very fast random access within the .zip file (see https://github.com/sozip/sozip-spec). Large compressed files can be accessed directly from SOZip without prior decompression. The .zip file is otherwise fully backward compatible.

If sozip_enabled="AUTO" (the default), a file is seek-optimized only if its size is above the values of sozip_min_file_size (default 1 MB) and sozip_chunk_size (default 32768). In "YES" mode, all input files will be seek-optimized. In "NO" mode, no input files will be seek-optimized. The default can be changed with the CPL_SOZIP_ENABLED configuration option.

Note

The GDAL_NUM_THREADS configuration option can be set to ALL_CPUS or an integer value to specify the number of threads to use for SOZip-compressed files (see set_config_option()).

SOZip can be validated with:

vsi_get_file_metadata(zip_file, domain="ZIP")

where zip_file uses the /vsizip/ prefix.

Examples

lcp_file <- system.file("extdata/storm_lake.lcp", package="gdalraster")
zip_file <- file.path(tempdir(), "storml_lcp.zip")

# Requires GDAL >= 3.7
if (as.integer(gdal_version()[2]) >= 3070000) {
  addFilesInZip(zip_file, lcp_file, full_paths=FALSE, sozip_enabled="YES",
                num_threads=1)

  print("Files in zip archive:")
  print(unzip(zip_file, list=TRUE))

  # Open with GDAL using Virtual File System handler '/vsizip/'
  # see: https://gdal.org/user/virtual_file_systems.html#vsizip-zip-archives
  lcp_in_zip <- file.path("/vsizip", zip_file, "storm_lake.lcp")
  print("SOZip metadata:")
  print(vsi_get_file_metadata(lcp_in_zip, domain="ZIP"))

  ds <- new(GDALRaster, lcp_in_zip)
  ds$info()
  ds$close()
  DONTSHOW({vsi_unlink(zip_file)})
}