Ecosyste.ms: Packages
An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.
Top 0.1% downloads on pypi.org
Top 0.1% dependent packages on pypi.org
Top 0.1% dependent repos on pypi.org
Top 2.3% forks on pypi.org
Top 0.6% docker downloads on pypi.org
pypi.org : tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
Registry
-
Source
- Documentation
- JSON
purl: pkg:pypi/tokenizers
Keywords: NLP, tokenizer, BPE, transformer, deep learning, bert, gpt, language-model, natural-language-processing, natural-language-understanding, nlp, transformers
License: Apache-2.0
Latest release: 22 days ago
First release: over 4 years ago
Dependent packages: 224
Dependent repositories: 14,571
Downloads: 25,249,530 last month
Stars: 8,474 on GitHub
Forks: 726 on GitHub
Docker dependents: 2,098
Docker downloads: 42,281,989
Total Commits: 1640
Committers: 97
Average commits per author: 16.907
Development Distribution Score (DDS): 0.43
More commit stats: commits.ecosyste.ms
See more repository details: repos.ecosyste.ms
Last synced: about 20 hours ago