Top 3.1% downloads on pypi.org
Top 3.2% dependent packages on pypi.org
Top 1.5% dependent repos on pypi.org
Top 5.4% forks on pypi.org
Top 4.0% docker downloads on pypi.org
pypi.org : pyonmttok
Fast and customizable text tokenization library with BPE and SentencePiece support
Registry
-
Source
- Homepage
- Documentation
- JSON
purl: pkg:pypi/pyonmttok
Keywords:
tokenization
, opennmt
, unicode
, bpe
, sentencepiece
, subword
, cpp
, icu
, machine-translation
, natural-language-processing
, python
, tokenizer
License: MIT
Latest release: about 2 years ago
First release: about 7 years ago
Dependent packages: 3
Dependent repositories: 103
Downloads: 28,751 last month
Stars: 302 on GitHub
Forks: 74 on GitHub
Docker dependents: 1
Docker downloads: 29
Total Commits: 588
Committers: 15
Average commits per author: 39.2
Development Distribution Score (DDS): 0.495
More commit stats: commits.ecosyste.ms
See more repository details: repos.ecosyste.ms
Last synced: 3 days ago