Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

crates.io : gpt_tokenizer

Rust BPE Encoder Decoder (Tokenizer) for GPT-2 / GPT-3

Registry - Source - Documentation - JSON
purl: pkg:cargo/gpt_tokenizer
Keywords: tokenizer, openai, gpt3, chatgpt, bpe
License: MIT
Latest release: about 1 year ago
First release: about 1 year ago
Dependent packages: 1
Downloads: 478 total
Stars: 12 on GitHub
Forks: 1 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: 19 days ago

Readme