Ecosyste.ms: Packages
An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.
rubygems.org : youtokentome
High performance unsupervised text tokenization for Ruby
Registry
-
Source
- Documentation
- JSON
purl: pkg:gem/youtokentome
Keywords: bpe, byte-pair-encoding, npl, tokenization, unsupervised-learning, word-segmentation
License: MIT
Latest release: about 3 years ago
First release: over 4 years ago
Downloads: 5,481 total
Stars: 21 on GitHub
Forks: 1 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: 11 days ago