crates.io : text-splitter
Split text into semantic chunks, up to a desired chunk size. Supports calculating length by characters and tokens, and is callable from Rust and Python.
Registry
-
Source
- Documentation
- JSON
purl: pkg:cargo/text-splitter
Keywords:
ai
, nlp
, split
, text
, tokenizer
License: MIT
Latest release: 2 months ago
First release: about 2 years ago
Dependent packages: 5
Dependent repositories: 1
Downloads: 479,705 total
Stars: 455 on GitHub
Forks: 25 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: 4 days ago