pypi.org : flash-tokenizer
Extremely fast bert tokenizer
Registry
-
Source
- Documentation
- JSON
purl: pkg:pypi/flash-tokenizer
Keywords:
bert
, berttokenizer
, cpp
, cpp17
, deep-learning
, flash
, huggingface
, nlp
, pybind11
, python
, tokenizer
, trie
, wordpiece
, wordpiece-tokenization
License: MIT
Latest release: 6 months ago
First release: 7 months ago
Downloads: 147 last month
Stars: 459 on GitHub
Forks: 7 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: 9 days ago
Loading...
Readme
Loading...