Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

npmjs.org : tokengeex

This repository holds the code for the TokenGeeX Rust crate and Python package. TokenGeeX is a tokenizer for [CodeGeeX](https://github.com/THUDM/Codegeex2) aimed at code and Chinese. It is based on [UnigramLM (Taku Kudo 2018)](https://arxiv.org/abs/1804.1

Registry - Source - JSON
purl: pkg:npm/tokengeex
Keywords: NLP, tokenizers
License: ISC
Latest release: 2 months ago
First release: 2 months ago
Downloads: 19 last month
Stars: 3 on GitHub
Forks: 1 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: about 14 hours ago

    Loading...
    Readme
    Loading...