An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

py-kerchunk

Kerchunk is a library that provides a unified way to represent a variety of chunked, compressed data formats (e.g. NetCDF, HDF5, GRIB), allowing efficient access to the data from traditional file systems or cloud object storage. It also provides a flexible way to create virtual datasets from multiple files. It does this by extracting the byte ranges, compression information and other information about the data and storing this metadata in a new, separate object. This means that you can create a virtual aggregate dataset over potentially many source files, for efficient, parallel and cloud-friendly in-situ access without having to copy or translate the originals. It is a gateway to in-the-cloud massive data processing while the data providers still insist on using legacy formats for archival storage.

Ecosystem
spack.io
Latest Release
0.2.9
8 months ago
Versions
1
Links
Registry spack.io
Source Repository
JSON API View JSON
CodeMeta codemeta.json
Package Details
PURL pkg:spack/py-kerchunk
spec
License Other
First Release 8 months ago
Last Synced about 20 hours ago
Repository
Stars 365 on GitHub
Forks 95 on GitHub
Commits 447
Committers 36
Avg per Author 12.417
DDS 0.604