pypi.org : real-wordpiece
A score-based implementation of WordPiece tokenization training, compatible with HuggingFace tokenizers.
Registry
-
Source
- Documentation
- JSON
purl: pkg:pypi/real-wordpiece
Keywords:
bert
, gpt
, language-model
, natural-language-processing
, natural-language-understanding
, nlp
, transformers
License: Apache-2.0
Latest release: 8 months ago
First release: 10 months ago
Downloads: 166 last month
Stars: 9,580 on GitHub
Forks: 878 on GitHub
Total Commits: 1729
Committers: 112
Average commits per author: 15.438
Development Distribution Score (DDS): 0.46
More commit stats: commits.ecosyste.ms
See more repository details: repos.ecosyste.ms
Last synced: 1 day ago