crates.io : token
A simple string-tokenizer (and sentence splitter) Note: If you find that you would like to use the name for something more appropriate, please just send me a mail at jaln at itu dot dk
Registry
-
Source
- Documentation
- JSON
purl: pkg:cargo/token
Keywords:
string
, tokenizer
, sentence
, splitter
License: MIT
Latest release: over 10 years ago
First release: over 10 years ago
Dependent repositories: 1
Downloads: 2,982 total
Stars: 5 on GitHub
Forks: 2 on GitHub
Docker dependents: 29
Docker downloads: 23,282,367
See more repository details: repos.ecosyste.ms
Last synced: 20 days ago