Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

crates.io "tokenizer" keyword

parsit 0.2.0
very simple lib, the parsing combinators, recursive descendent that uses logos as lexer
17 versions - Latest release: 10 months ago - 1 dependent package - 1 dependent repositories - 5.28 thousand downloads total - 7 stars on GitHub - 1 maintainer
luther-derive 0.1.0
The proc macro generator for the Luther lexer generator.
1 version - Latest release: almost 6 years ago - 1.3 thousand downloads total - 5 stars on GitHub - 1 maintainer
cang-jie 0.18.0
A Chinese tokenizer for tantivy
20 versions - Latest release: 7 months ago - 6 dependent packages - 13 dependent repositories - 25 thousand downloads total - 68 stars on GitHub - 1 maintainer
c_lexer 0.1.1
C lexer
2 versions - Latest release: about 5 years ago - 1 dependent package - 1 dependent repositories - 2.22 thousand downloads total - 6 stars on GitHub - 1 maintainer
bytepiece 0.2.0
Rust version of bytepiece tokenizer
2 versions - Latest release: 8 months ago - 527 downloads total - 9 stars on GitHub - 1 maintainer
lindera-tantivy 0.27.1 💰
Lindera Tokenizer for Tantivy.
40 versions - Latest release: 6 months ago - 5 dependent packages - 7 dependent repositories - 21.7 thousand downloads total - 46 stars on GitHub - 4 maintainers
tantivy-stemmers 0.2.0
A collection of Tantivy stemmer tokenizers
2 versions - Latest release: 19 days ago - 185 downloads total - 0 stars on GitHub - 2 maintainers
logos2 💰
Create ridiculously fast Lexers
6 versions - Latest release: 19 days ago - 1.51 thousand downloads total - 2,632 stars on GitHub - 1 maintainer
logos-cli2 💰
Create ridiculously fast Lexers
6 versions - Latest release: 19 days ago - 1.5 thousand downloads total - 2,632 stars on GitHub - 1 maintainer
logos-cli 0.14.0 💰
Create ridiculously fast Lexers
2 versions - Latest release: 4 months ago - 608 downloads total - 2,632 stars on GitHub - 2 maintainers
tantivy-czech-stemmer 0.2.1
Czech stemmer as Tantivy tokenizer
2 versions - Latest release: 21 days ago - 281 downloads total - 0 stars on GitHub - 2 maintainers
pkl_fast 0.1.1
A library aiming to easily and efficiently work with Apple's PKL format.
2 versions - Latest release: 3 months ago - 620 downloads total - 3 stars on GitHub - 1 maintainer
luther 0.1.0
The runtime components of the Luther lexer generator.
1 version - Latest release: almost 6 years ago - 1 dependent package - 1 dependent repositories - 1.84 thousand downloads total - 5 stars on GitHub - 1 maintainer
rust_transformers 0.2.0
High performance tokenizers for Rust
2 versions - Latest release: over 4 years ago - 1 dependent package - 1.01 thousand downloads total - 270 stars on GitHub - 1 maintainer
fileql 0.3.0 💰
A tool to run SQL-like query on local files using GitQL SDK
3 versions - Latest release: 23 days ago - 919 downloads total - 55 stars on GitHub - 1 maintainer
xxcalc 0.2.1
Embeddable or standalone robust floating-point polynomial calculator
4 versions - Latest release: over 7 years ago - 1 dependent repositories - 8.17 thousand downloads total - 13 stars on GitHub - 1 maintainer
html5tokenizer 0.5.2
An HTML5 tokenizer with code span support.
7 versions - Latest release: 8 months ago - 1 dependent repositories - 2.2 thousand downloads total - 1 maintainer
tele_tokenizer 0.2.0
A CSS tokenizer
2 versions - Latest release: about 2 years ago - 3 dependent packages - 1 dependent repositories - 1.69 thousand downloads total - 199 stars on GitHub - 1 maintainer
bytepiece_rs 0.2.2
The Bytepiece Tokenizer Implemented in Rust
7 versions - Latest release: 6 months ago - 1 dependent package - 2.07 thousand downloads total - 14 stars on GitHub - 1 maintainer
svgparser 0.8.1
Featureful, pull-based, zero-allocation SVG parser.
21 versions - Latest release: about 6 years ago - 4 dependent packages - 98 dependent repositories - 101 thousand downloads total - 22 stars on GitHub - 1 maintainer
tusk_lexer 0.4.7
The lexical analysis component of Tusk.
21 versions - Latest release: almost 3 years ago - 1 dependent package - 6.9 thousand downloads total - 1 maintainer
lexical_scanner 0.1.18
A simple lexer which creates over 115+ various tokens based on the rust programming language. Thi...
19 versions - Latest release: about 2 years ago - 7.25 thousand downloads total - 2 stars on GitHub - 1 maintainer
rye-grain 0.0.1
A Python to Rust translator
1 version - Latest release: over 1 year ago - 401 downloads total - 1 stars on GitHub - 1 maintainer
vibrato 0.5.1
Vibrato: viterbi-based accelerated tokenizer
11 versions - Latest release: about 1 year ago - 1 dependent package - 1 dependent repositories - 11.6 thousand downloads total - 292 stars on GitHub - 2 maintainers
azul-simplecss 0.1.1
A very simple CSS 2.1 tokenizer.
2 versions - Latest release: almost 5 years ago - 1 dependent package - 4 dependent repositories - 17.3 thousand downloads total - 29 stars on GitHub - 1 maintainer