Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

pypi.org : tokenization-scorer

Package for evaluating text tokenizations.

Registry - Source - Documentation - JSON
purl: pkg:pypi/tokenization-scorer
Keywords: bpe, segmentation, subword, tokenization
License: MIT
Latest release: about 1 year ago
First release: about 1 year ago
Downloads: 211 last month
Stars: 23 on GitHub
Forks: 2 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: 1 day ago

Readme