Ecosyste.ms: Packages
An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.
npmjs.org "tokenizers" keyword
tokengeex 0.6.2
This repository holds the code for the TokenGeeX Rust crate and Python package. TokenGeeX is a to...2 versions - Latest release: 2 months ago - 19 downloads last month - 3 stars on GitHub - 1 maintainer
@orama/tokenizers 2.0.18 💰
Additional tokenizers for Orama14 versions - Latest release: 28 days ago - 1.31 thousand downloads last month - 8,156 stars on GitHub - 6 maintainers
kingchop 1.0.6 💰
A text tokenizing library that handles strings, by tokenizing them into arrays, depending on inte...7 versions - Latest release: 6 months ago - 13 downloads last month - 1 stars on GitHub - 1 maintainer
@anush008/tokenizers-win32-x64-msvc 0.2.0
Multi-arch builds of HuggingFace tokenizers2 versions - Latest release: 9 months ago - 2 dependent packages - 2 dependent repositories - 128 downloads last month - 4 stars on GitHub - 1 maintainer
@anush008/tokenizers-linux-x64-gnu 0.2.0
Multi-arch builds of HuggingFace tokenizers2 versions - Latest release: 9 months ago - 2 dependent packages - 2 dependent repositories - 273 downloads last month - 4 stars on GitHub - 1 maintainer
@anush008/tokenizers-darwin-universal 0.2.0
Multi-arch builds of HuggingFace tokenizers2 versions - Latest release: 9 months ago - 2 dependent packages - 2 dependent repositories - 130 downloads last month - 4 stars on GitHub - 1 maintainer
@anush008/tokenizers 0.2.0
Multi-arch builds of HuggingFace tokenizers2 versions - Latest release: 9 months ago - 3 dependent packages - 2 dependent repositories - 400 downloads last month - 4 stars on GitHub - 1 maintainer
Top 5.7% on npmjs.org
29 versions - Latest release: almost 3 years ago - 16 dependent packages - 47 dependent repositories - 910 downloads last month - 8 stars on GitHub - 14 maintainers
string-punctuation-tokenizer 2.2.0
Small library that provides functions to tokenize a string into an array of words with or without...29 versions - Latest release: almost 3 years ago - 16 dependent packages - 47 dependent repositories - 910 downloads last month - 8 stars on GitHub - 14 maintainers
fib-tokenizers 1.2.0
cpp tokenizer module for fibjs.5 versions - Latest release: 5 months ago - 14 downloads last month - 0 stars on GitHub - 1 maintainer
Related Keywords
huggingface
4
javascript
3
text tokenizer
1
node.js tokenizer
1
flexible text tokenizer
1
general text tokenizer
1
paragraph tokenizer
1
sub sentence tokenizer
1
stable tokenizer
1
easy text tokenizer
1
new text tokenizer
1
natural-language-processing
1
nodejs
1
paragraph-tokenizer
1
sentence-tokenizer
1
text-processing
1
text-tokenization
1
word-tokenizer
1
string
1
punctuation
1
tokenizer
1
nlp
1
nlp-library
1
scripture-open-components
1
segmentation
1
fib-tokenizers
1
NLP
1
full-text search
1
search
1
fuzzy search
1
typo-tolerant search
1
full-text
1
vector search
1
stemming
1
mandarin
1
chinese
1
algiorithm
1
data-structures
1
node
1
search-algorithm
1
search-engine
1
typescript
1
typo-tolerance
1
vector
1
vector-database
1
vector-database-embedding
1
vector-search
1
vector-search-engine
1
NLP tokenizer
1
Tokenizer
1
Sentence tokenizer
1