pypi.org : blt-tokenizer
High-performance byte-level tokenizer with BPE support
Registry
-
Source
- Documentation
- JSON
purl: pkg:pypi/blt-tokenizer
Keywords:
tokenizer
, bpe
, nlp
, byte-level
, performance
License: Apache-2.0
Latest release: 2 months ago
First release: 2 months ago
Downloads: 54 last month
Stars: 0 on GitHub
Forks: 0 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: 1 day ago
Loading...
Readme
Loading...