crates.io "chunking" keyword
View the packages on the crates.io package registry that are tagged with the "chunking" keyword.
overlap-chunk 0.0.3
A Rust library for splitting text into chunks of specified size with adjustable overlap percentage.3 versions - Latest release: 4 months ago - 1.52 thousand downloads total - 1 stars on GitHub - 1 maintainer
semchunk-rs 0.1.1
A fast and lightweight Rust library for splitting text into semantically meaningful chunks.2 versions - Latest release: 7 months ago - 1.23 thousand downloads total - 3 stars on GitHub - 1 maintainer
cdc-chunkers 0.1.3
A collection of Content Defined Chunking algorithms4 versions - Latest release: 5 months ago - 2.49 thousand downloads total - 3 stars on GitHub - 1 maintainer
seq_chunking 0.1.0
SeqCDC (content defined chunking) in pure Rust.1 version - Latest release: about 1 month ago - 244 downloads total - 0 stars on GitHub - 1 maintainer
chonky 0.0.0-reserved
General-purpose tooling for segmenting, chunking and embedding files1 version - Latest release: 9 months ago - 785 downloads total - 1 maintainer
read_chunks 0.2.0
An extension to the Read trait allowing easier chunked reading3 versions - Latest release: over 1 year ago - 3.5 thousand downloads total - 1 stars on GitHub - 1 maintainer
cdc-rs 💰
Rabin fingerprint based Content-Defined Chunking1 version - Latest release: 16 days ago - 1.84 thousand downloads total - 24 stars on GitHub - 1 maintainer
file-chunker 0.1.1
Efficiently process a file in (approximately) equally-sized parts2 versions - Latest release: over 3 years ago - 1 dependent package - 2 dependent repositories - 23.4 thousand downloads total - 1 stars on GitHub - 1 maintainer
fastcdc-alt 0.2.2
FastCDC (content defined chunking) implementation in pure Rust with an alternative API to the ori...4 versions - Latest release: almost 2 years ago - 5.86 thousand downloads total - 4 stars on GitHub - 1 maintainer
chunkfs 0.1.3
An in-memory file system that can be used to compare different deduplication algorithms4 versions - Latest release: 5 months ago - 2.19 thousand downloads total - 8 stars on GitHub - 1 maintainer
regex-chunker 0.3.0
Iterate over the data in a `Read` type in a regular-expression-delimited way.4 versions - Latest release: about 2 years ago - 4.23 thousand downloads total - 0 stars on GitHub - 1 maintainer
gearhash 0.1.3
Fast, SIMD-accelerated hash function for content-defined chunking4 versions - Latest release: over 5 years ago - 2 dependent repositories - 54.7 thousand downloads total - 24 stars on GitHub - 1 maintainer
Top 8.5% on crates.io
18 versions - Latest release: 3 months ago - 10 dependent packages - 10 dependent repositories - 224 thousand downloads total - 153 stars on GitHub - 1 maintainer
fastcdc 3.2.1
FastCDC (content defined chunking) in pure Rust.18 versions - Latest release: 3 months ago - 10 dependent packages - 10 dependent repositories - 224 thousand downloads total - 153 stars on GitHub - 1 maintainer
cdchunking 1.0.1
Content-defined chunking7 versions - Latest release: almost 5 years ago - 4 dependent packages - 5 dependent repositories - 29.8 thousand downloads total - 21 stars on GitHub - 1 maintainer
Related Keywords
cdc
8
rust
5
deduplication
3
chunk
3
text
2
parallel
1
text-processing
1
filesystem
1
iterator
1
read
1
regex
1
hash
1
fast
1
gear
1
chunking-algorithm
1
defined
1
chunks
1
content
1
rolling-hash-functions
1
files
1
concurrency
1
rust-library
1
data-stream
1
rabin
1
io
1
segmentation
1
llm
1
extraction
1
embedding
1
token
1
semantic
1
nlp
1
overlap
1