Ecosyste.ms: Packages
An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.
Top 4.7% dependent repos on proxy.golang.org
proxy.golang.org : github.com/hupe1980/go-tiktoken
Package tiktoken provides functionality for tokenizing and encoding text using the tiktoken algorithm. The package includes various functions for text processing and encoding using the tiktoken algorithm.
Registry
-
Source
- Documentation
- JSON
purl: pkg:golang/github.com/hupe1980/go-tiktoken
Keywords: bpe, golang, gpt2, openai, tiktoken, tokenizer
License: MIT
Latest release: 11 months ago
First release: 12 months ago
Namespace: github.com/hupe1980
Dependent packages: 4
Dependent repositories: 1
Stars: 1 on GitHub
Forks: 0 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: 21 days ago