Top 4.7% dependent repos on proxy.golang.org
proxy.golang.org : github.com/hupe1980/go-tiktoken
Package tiktoken provides functionality for tokenizing and encoding text using the tiktoken algorithm. The package includes various functions for text processing and encoding using the tiktoken algorithm.
Registry
-
Source
- Documentation
- JSON
- codemeta.json
purl: pkg:golang/github.com/hupe1980/go-tiktoken
Keywords:
bpe
, golang
, gpt2
, openai
, tiktoken
, tokenizer
License: MIT
Latest release: 10 months ago
First release: over 2 years ago
Namespace: github.com/hupe1980
Dependent packages: 4
Dependent repositories: 1
Stars: 1 on GitHub
Forks: 0 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: 21 days ago