Top 0.1% downloads on pypi.org
Top 0.1% dependent packages on pypi.org
Top 0.1% dependent repos on pypi.org
Top 2.3% forks on pypi.org
Top 0.6% docker downloads on pypi.org
pypi.org : tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
Registry
-
Source
- Documentation
- JSON
purl: pkg:pypi/tokenizers
Keywords:
NLP
, tokenizer
, BPE
, transformer
, deep learning
, bert
, gpt
, language-model
, natural-language-processing
, natural-language-understanding
, nlp
, transformers
License: Apache-2.0
Latest release: about 1 month ago
First release: almost 6 years ago
Dependent packages: 380
Dependent repositories: 14,571
Downloads: 85,837,974 last month
Stars: 9,012 on GitHub
Forks: 795 on GitHub
Docker dependents: 2,098
Docker downloads: 42,285,347
Total Commits: 1709
Committers: 106
Average commits per author: 16.123
Development Distribution Score (DDS): 0.453
More commit stats: commits.ecosyste.ms
See more repository details: repos.ecosyste.ms
Last synced: 13 days ago