npmjs.org "tokenizer" keyword
View the packages on the npmjs.org package registry that are tagged with the "tokenizer" keyword.
parse5-wxmp 3.0.3 π°
HTML parsing/serialization toolset for Node.js. WHATWG HTML Living Standard (aka HTML5)-compliant.1 version - Latest release: about 8 years ago - 2 dependent packages - 1 dependent repositories - 9 downloads last month - 3,826 stars on GitHub - 1 maintainer
Top 7.2% on npmjs.org
2 versions - Latest release: almost 10 years ago - 4 dependent packages - 14 dependent repositories - 1.03 thousand downloads last month - 15 stars on GitHub - 1 maintainer
node-vntokenizer 0.0.2 π°
Tokenizer for Vietnamese in Nodejs and Javascript2 versions - Latest release: almost 10 years ago - 4 dependent packages - 14 dependent repositories - 1.03 thousand downloads last month - 15 stars on GitHub - 1 maintainer
@shimaore/jison-lex 0.3.4
lexical analyzer generator used by jison1 version - Latest release: almost 7 years ago - 3 dependent packages - 1 dependent repositories - 3 downloads last month - 58 stars on GitHub - 1 maintainer
Top 1.3% on npmjs.org
7 versions - Latest release: over 7 years ago - 57 dependent packages - 1,756 dependent repositories - 444 thousand downloads last month - 931 stars on GitHub - 1 maintainer
kuromoji 0.1.2
JavaScript implementation of Japanese morphological analyzer7 versions - Latest release: over 7 years ago - 57 dependent packages - 1,756 dependent repositories - 444 thousand downloads last month - 931 stars on GitHub - 1 maintainer
cjk-tokenizer 0.1.0
A CJK text tokenizer2 versions - Latest release: almost 9 years ago - 1 dependent package - 1 dependent repositories - 249 downloads last month - 4 stars on GitHub - 1 maintainer
react-bootstrap-typeahead-ie11-compatible 1.3.0
React-based typeahead using the Bootstrap theme1 version - Latest release: over 8 years ago - 1 dependent package - 1 dependent repositories - 42 downloads last month - 1,015 stars on GitHub - 1 maintainer
Top 1.0% on npmjs.org
1 version - Latest release: almost 10 years ago - 6 dependent packages - 7,486 dependent repositories - 1.6 million downloads last month - 19 maintainers
glsl-token-whitespace-trim 1.0.0
Trim the whitespace within an array of GLSL tokens1 version - Latest release: almost 10 years ago - 6 dependent packages - 7,486 dependent repositories - 1.6 million downloads last month - 19 maintainers
Top 2.4% on npmjs.org
118 versions - Latest release: 2 months ago - 56 dependent packages - 70,737 dependent repositories - 48.3 million downloads last month - 43 stars on GitHub - 1 maintainer
strtok3 10.3.4 π°
A promise based streaming tokenizer118 versions - Latest release: 2 months ago - 56 dependent packages - 70,737 dependent repositories - 48.3 million downloads last month - 43 stars on GitHub - 1 maintainer
@caoer/strtok3 7.0.3 π°
A promise based streaming tokenizer2 versions - Latest release: almost 3 years ago - 2 dependent packages - 5 downloads last month - 43 stars on GitHub - 1 maintainer
@alexssmusica/strtok3 7.0.1 π°
A promise based streaming tokenizer1 version - Latest release: about 2 years ago - 2 dependent packages - 14 downloads last month - 43 stars on GitHub - 1 maintainer
Top 1.4% on npmjs.org
22 versions - Latest release: 4 months ago - 30 dependent packages - 10,014 dependent repositories - 64.1 million downloads last month - 1,000 stars on GitHub - 3 maintainers
@csstools/css-tokenizer 3.0.4 π°
Tokenize CSS22 versions - Latest release: 4 months ago - 30 dependent packages - 10,014 dependent repositories - 64.1 million downloads last month - 1,000 stars on GitHub - 3 maintainers
Top 3.0% on npmjs.org
45 versions - Latest release: about 2 months ago - 52 dependent packages - 70,723 dependent repositories - 44.9 million downloads last month - 11 stars on GitHub - 1 maintainer
token-types 6.1.1 π°
Common token types for decoding and encoding numeric and string values45 versions - Latest release: about 2 months ago - 52 dependent packages - 70,723 dependent repositories - 44.9 million downloads last month - 11 stars on GitHub - 1 maintainer
@caoer/token-types 5.0.3 π°
Common token types for decoding and encoding numeric and string values2 versions - Latest release: almost 3 years ago - 3 dependent packages - 25 downloads last month - 11 stars on GitHub - 1 maintainer
Top 1.8% on npmjs.org
28 versions - Latest release: over 9 years ago - 15 dependent packages - 167 dependent repositories - 20.5 thousand downloads last month - 677 stars on GitHub - 2 maintainers
react-typeahead 1.1.9
React-based typeahead and typeahead-tokenizer28 versions - Latest release: over 9 years ago - 15 dependent packages - 167 dependent repositories - 20.5 thousand downloads last month - 677 stars on GitHub - 2 maintainers
react-structured-query-search 1.0.0-alpha.5
react-structured-query-search7 versions - Latest release: over 6 years ago - 1 dependent package - 1 dependent repositories - 16 downloads last month - 11 stars on GitHub - 1 maintainer
stemin 0.1.2
Indonesian language stemmer with Levenshtein distance based on ariaghora/mpstemmer and damzaky/sa...4 versions - Latest release: 8 months ago - 7 downloads last month - 0 stars on GitHub - 1 maintainer
postcss-bredon-validate 0.0.2 π°
PostCSS plugin to validate values using bredon-validate2 versions - Latest release: almost 8 years ago - 2 dependent packages - 1 dependent repositories - 6 downloads last month - 40 stars on GitHub - 1 maintainer
polix-rpc 0.0.2
rpc plugin for polix2 versions - Latest release: almost 7 years ago - 1 dependent package - 1 dependent repositories - 7 downloads last month - 10 stars on GitHub - 1 maintainer
Top 0.7% on npmjs.org
51 versions - Latest release: about 8 years ago - 829 dependent packages - 34,659 dependent repositories - 284 thousand downloads last month - 4,328 stars on GitHub - 1 maintainer
jison 0.4.18
A parser generator with Bison's API51 versions - Latest release: about 8 years ago - 829 dependent packages - 34,659 dependent repositories - 284 thousand downloads last month - 4,328 stars on GitHub - 1 maintainer
Top 1.2% on npmjs.org
18 versions - Latest release: about 2 years ago - 13 dependent packages - 7,797 dependent repositories - 7.68 million downloads last month - 2,492 stars on GitHub - 1 maintainer
@chevrotain/types 11.0.3
Type Signatures and API Website for Chevrotain18 versions - Latest release: about 2 years ago - 13 dependent packages - 7,797 dependent repositories - 7.68 million downloads last month - 2,492 stars on GitHub - 1 maintainer
react-bootstrap-typeahead-ali 1.0.2
React-based typeahead using the Bootstrap theme by react-bootstrap-typeahead v1.0.0-beta53 versions - Latest release: over 8 years ago - 1 dependent package - 1 dependent repositories - 2 downloads last month - 1,015 stars on GitHub - 1 maintainer
@pebblecode/react-typeahead 2.0.1
React-based typeahead and typeahead-tokenizer3 versions - Latest release: over 9 years ago - 2 dependent packages - 1 dependent repositories - 11 downloads last month - 677 stars on GitHub - 7 maintainers
react-bootstrap-typeahead-with-restart 1.0.0
React typeahead with Bootstrap styling1 version - Latest release: almost 2 years ago - 68 downloads last month - 1,015 stars on GitHub - 1 maintainer
Top 0.6% on npmjs.org
145 versions - Latest release: 4 months ago - 499 dependent packages - 24,309 dependent repositories - 921 thousand downloads last month - 10,490 stars on GitHub - 3 maintainers
natural 8.1.0
General natural language (tokenizing, stemming (English, Russian, Spanish), part-of-speech taggin...145 versions - Latest release: 4 months ago - 499 dependent packages - 24,309 dependent repositories - 921 thousand downloads last month - 10,490 stars on GitHub - 3 maintainers
@eslint/css-tree 3.6.5 π°
A tool set for CSS: fast detailed parser (CSS β AST), walker (AST traversal), generator (AST β CS...17 versions - Latest release: about 1 month ago - 665 thousand downloads last month - 14 stars on GitHub - 2 maintainers
@aiktb/kuromoji 1.0.2
Forked version of kuromoji with better compatibility for browsers3 versions - Latest release: over 1 year ago - 39 downloads last month - 0 stars on GitHub - 1 maintainer
Top 0.8% on npmjs.org
167 versions - Latest release: about 2 years ago - 282 dependent packages - 17,129 dependent repositories - 11.6 million downloads last month - 2,412 stars on GitHub - 1 maintainer
chevrotain 11.0.3
Chevrotain is a high performance fault tolerant javascript parsing DSL for building recursive dec...167 versions - Latest release: about 2 years ago - 282 dependent packages - 17,129 dependent repositories - 11.6 million downloads last month - 2,412 stars on GitHub - 1 maintainer
Top 1.1% on npmjs.org
15 versions - Latest release: almost 3 years ago - 276 dependent packages - 134,797 dependent repositories - 22.5 million downloads last month - 800 stars on GitHub - 2 maintainers
moo 0.5.2
Optimised tokenizer/lexer generator! π Much performance. Moo!15 versions - Latest release: almost 3 years ago - 276 dependent packages - 134,797 dependent repositories - 22.5 million downloads last month - 800 stars on GitHub - 2 maintainers
kuromoji.js 0.1.6
JavaScript implementation of Japanese morphological analyzer4 versions - Latest release: 9 months ago - 13 downloads last month - 2 stars on GitHub - 1 maintainer
Top 1.8% on npmjs.org
26 versions - Latest release: over 4 years ago - 70 dependent packages - 18,393 dependent repositories - 1.87 million downloads last month - 85 stars on GitHub - 6 maintainers
simple-html-tokenizer 0.5.11
Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind o...26 versions - Latest release: over 4 years ago - 70 dependent packages - 18,393 dependent repositories - 1.87 million downloads last month - 85 stars on GitHub - 6 maintainers
tokenizer-c-like 0.1.1
Tokenizer for a C-like language2 versions - Latest release: 4 months ago - 0 downloads last month - 0 stars on GitHub - 1 maintainer
Top 9.3% on npmjs.org
1 version - Latest release: about 6 years ago - 6 dependent packages - 4 dependent repositories - 221 downloads last month - 2 stars on GitHub - 1 maintainer
jison-es 0.4.18
Fork of jison that emits parsers as ES modules1 version - Latest release: about 6 years ago - 6 dependent packages - 4 dependent repositories - 221 downloads last month - 2 stars on GitHub - 1 maintainer
bredon 4.0.0 π°
A modern CSS value compiler in JavaScript9 versions - Latest release: almost 8 years ago - 15 dependent packages - 1 dependent repositories - 15 downloads last month - 40 stars on GitHub - 1 maintainer
aiot-css-tree 3.0.0 π°
A tool set for CSS: fast detailed parser (CSS β AST), walker (AST traversal), generator (AST β CS...2 versions - Latest release: 12 months ago - 188 downloads last month - 2,040 stars on GitHub - 1 maintainer
@neabyte/markdown-html 1.1.0
Markdown to HTML converter with TypeScript types, streaming support, and universal module compati...3 versions - Latest release: about 20 hours ago - 120 downloads last month - 0 stars on GitHub - 1 maintainer
Top 1.3% on npmjs.org
47 versions - Latest release: 11 months ago - 256 dependent packages - 5,960 dependent repositories - 12 million downloads last month - 958 stars on GitHub - 1 maintainer
stream-json 1.9.1 π°
stream-json is the micro-library of Node.js stream components for creating custom JSON processing...47 versions - Latest release: 11 months ago - 256 dependent packages - 5,960 dependent repositories - 12 million downloads last month - 958 stars on GitHub - 1 maintainer
Top 0.9% on npmjs.org
58 versions - Latest release: 10 months ago - 820 dependent packages - 2,569,508 dependent repositories - 180 million downloads last month - 2,040 stars on GitHub - 2 maintainers
css-tree 3.1.0 π°
A tool set for CSS: fast detailed parser (CSS β AST), walker (AST traversal), generator (AST β CS...58 versions - Latest release: 10 months ago - 820 dependent packages - 2,569,508 dependent repositories - 180 million downloads last month - 2,040 stars on GitHub - 2 maintainers
css-tree-nup 3.0.1 unpublished π°
A tool set for CSS: fast detailed parser (CSS β AST), walker (AST traversal), generator (AST β CS...1 version - Latest release: about 1 year ago - 68 downloads last month - 1,882 stars on GitHub - 1 maintainer
Top 1.0% on npmjs.org
42 versions - Latest release: 7 months ago - 144 dependent packages - 151,327 dependent repositories - 64 million downloads last month - 2,043 stars on GitHub - 1 maintainer
micromark 4.0.2 π°
small commonmark compliant markdown parser with positional info and concrete tokens42 versions - Latest release: 7 months ago - 144 dependent packages - 151,327 dependent repositories - 64 million downloads last month - 2,043 stars on GitHub - 1 maintainer
llama3-tokenizer-js 1.2.0
JS tokenizer for LLaMA 39 versions - Latest release: about 1 year ago - 13.4 thousand downloads last month - 78 stars on GitHub - 1 maintainer
Top 1.6% on npmjs.org
17 versions - Latest release: about 2 years ago - 4 dependent packages - 7,794 dependent repositories - 7.68 million downloads last month - 2,412 stars on GitHub - 1 maintainer
@chevrotain/utils 11.0.3
common utilities17 versions - Latest release: about 2 years ago - 4 dependent packages - 7,794 dependent repositories - 7.68 million downloads last month - 2,412 stars on GitHub - 1 maintainer
shield-lang 0.0.1
An interpreted dynamic class-based programming language1 version - Latest release: over 5 years ago - 1 downloads last month - 1 maintainer
@lenml/tokenizer-aya_expanse 3.7.2
aya_expanse tokenizer for NodeJS/Browser5 versions - Latest release: 27 days ago - 111 downloads last month - 24 stars on GitHub - 1 maintainer
@lenml/tokenizer-gemma2 3.7.2
gemma2 tokenizer for NodeJS/Browser5 versions - Latest release: 27 days ago - 403 downloads last month - 24 stars on GitHub - 1 maintainer
@lenml/tokenizer-minicpm_v4_5 3.7.2
minicpm_v4_5 tokenizer for NodeJS/Browser1 version - Latest release: 27 days ago - 68 downloads last month - 24 stars on GitHub - 1 maintainer
@lenml/tokenizer-mistral_nemo 3.7.2
mistral_nemo tokenizer for NodeJS/Browser5 versions - Latest release: 27 days ago - 103 downloads last month - 24 stars on GitHub - 1 maintainer
@lenml/tokenizer-qwen2_5 3.7.2
qwen2_5 tokenizer for NodeJS/Browser5 versions - Latest release: 27 days ago - 950 downloads last month - 24 stars on GitHub - 1 maintainer
@lenml/tokenizer-qwen3 3.7.2
qwen3 tokenizer for NodeJS/Browser2 versions - Latest release: 27 days ago - 1.23 thousand downloads last month - 24 stars on GitHub - 1 maintainer
@lenml/tokenizer-deepseek_v3 3.7.2
deepseek_v3 tokenizer for NodeJS/Browser4 versions - Latest release: 27 days ago - 1.41 thousand downloads last month - 24 stars on GitHub - 1 maintainer
@lenml/tokenizer-gemini 3.7.2
gemini tokenizer for NodeJS/Browser5 versions - Latest release: 27 days ago - 2.32 thousand downloads last month - 24 stars on GitHub - 1 maintainer
@lenml/tokenizer-llama3_2 3.7.2
llama3_2 tokenizer for NodeJS/Browser4 versions - Latest release: 27 days ago - 92 downloads last month - 24 stars on GitHub - 1 maintainer
@lenml/tokenizer-gemma3 3.7.2
gemma3 tokenizer for NodeJS/Browser3 versions - Latest release: 27 days ago - 154 downloads last month - 24 stars on GitHub - 1 maintainer
@lenml/tokenizer-gptoss 3.7.2
gptoss tokenizer for NodeJS/Browser1 version - Latest release: 27 days ago - 57 downloads last month - 24 stars on GitHub - 1 maintainer
@lenml/tokenizer-internlm2 3.7.2
internlm2 tokenizer for NodeJS/Browser13 versions - Latest release: 27 days ago - 151 downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-claude1 3.7.2
claude1 tokenizer for NodeJS/Browser14 versions - Latest release: 27 days ago - 126 downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-chatglm3 3.7.2
chatglm3 tokenizer for NodeJS/Browser14 versions - Latest release: 27 days ago - 157 downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-claude 3.7.2
claude tokenizer for NodeJS/Browser14 versions - Latest release: 27 days ago - 4.89 thousand downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-gpt35turbo 3.7.2
gpt35turbo tokenizer for NodeJS/Browser13 versions - Latest release: 27 days ago - 158 downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-gpt3 3.7.2
gpt3 tokenizer for NodeJS/Browser13 versions - Latest release: 27 days ago - 105 downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-qwen1_5 3.7.2
qwen1_5 tokenizer for NodeJS/Browser13 versions - Latest release: 27 days ago - 959 downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-gpt4o 3.7.2
gpt4o tokenizer for NodeJS/Browser11 versions - Latest release: 27 days ago - 461 downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-llama2 3.7.2
llama2 tokenizer for NodeJS/Browser13 versions - Latest release: 27 days ago - 969 downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-yi 3.7.2
yi tokenizer for NodeJS/Browser13 versions - Latest release: 27 days ago - 173 downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-text_davinci002 3.7.2
text_davinci002 tokenizer for NodeJS/Browser13 versions - Latest release: 27 days ago - 120 downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-baichuan2 3.7.2
baichuan2 tokenizer for NodeJS/Browser14 versions - Latest release: 27 days ago - 119 downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-gemma 3.7.2
gemma tokenizer for NodeJS/Browser14 versions - Latest release: 27 days ago - 195 downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-gpt2 3.7.2
gpt2 tokenizer for NodeJS/Browser13 versions - Latest release: 27 days ago - 127 downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-llama3 3.7.2
llama3 tokenizer for NodeJS/Browser12 versions - Latest release: 27 days ago - 2.69 thousand downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-gpt4 3.7.2
gpt4 tokenizer for NodeJS/Browser13 versions - Latest release: 27 days ago - 144 downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizers 3.7.2
a lightweight no-dependency fork of transformers.js (only tokenizers)15 versions - Latest release: 27 days ago - 15.1 thousand downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-gpt35turbo16k 3.7.2
gpt35turbo16k tokenizer for NodeJS/Browser13 versions - Latest release: 27 days ago - 158 downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-command_r_plus 3.7.2
command_r_plus tokenizer for NodeJS/Browser13 versions - Latest release: 27 days ago - 193 downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-text_embedding_ada002 3.7.2
text_embedding_ada002 tokenizer for NodeJS/Browser13 versions - Latest release: 27 days ago - 166 downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-llama3_1 3.7.2
llama3_1 tokenizer for NodeJS/Browser6 versions - Latest release: 27 days ago - 124 downloads last month - 6 stars on GitHub - 1 maintainer
@lenml/tokenizer-text_davinci003 3.7.2
text_davinci003 tokenizer for NodeJS/Browser13 versions - Latest release: 27 days ago - 153 downloads last month - 6 stars on GitHub - 1 maintainer
Top 1.8% on npmjs.org
27 versions - Latest release: over 2 years ago - 930 dependent packages - 3,789,143 dependent repositories - 74.3 million downloads last month - 92 stars on GitHub - 1 maintainer
ret 0.5.0 π°
Tokenizes a string that represents a regular expression.27 versions - Latest release: over 2 years ago - 930 dependent packages - 3,789,143 dependent repositories - 74.3 million downloads last month - 92 stars on GitHub - 1 maintainer
Top 1.3% on npmjs.org
80 versions - Latest release: 3 months ago - 101 dependent packages - 3,016 dependent repositories - 982 thousand downloads last month - 513 stars on GitHub - 2 maintainers
php-parser 3.2.5 π°
Parse PHP code from JS and returns its AST80 versions - Latest release: 3 months ago - 101 dependent packages - 3,016 dependent repositories - 982 thousand downloads last month - 513 stars on GitHub - 2 maintainers
Top 1.3% on npmjs.org
24 versions - Latest release: 10 months ago - 2,251 dependent packages - 1,940,799 dependent repositories - 345 million downloads last month - 526 stars on GitHub - 1 maintainer
js-tokens 9.0.1
Tiny JavaScript tokenizer.24 versions - Latest release: 10 months ago - 2,251 dependent packages - 1,940,799 dependent repositories - 345 million downloads last month - 526 stars on GitHub - 1 maintainer
@lenml/llama2-tokenizer-vocab-llama2 1.1.1
llama2 tokenizer for javascript2 versions - Latest release: over 1 year ago - 5.68 thousand downloads last month - 2 stars on GitHub - 1 maintainer
@lenml/llama2-tokenizer-vocab-internlm2 1.1.1
llama2 tokenizer for javascript2 versions - Latest release: over 1 year ago - 22 downloads last month - 2 stars on GitHub - 1 maintainer
@lenml/llama2-tokenizer-vocab-yi 1.1.1
llama2 tokenizer for javascript2 versions - Latest release: over 1 year ago - 18 downloads last month - 2 stars on GitHub - 1 maintainer
@lenml/llama2-tokenizer-vocab-chatglm3 1.1.1
llama2 tokenizer for javascript2 versions - Latest release: over 1 year ago - 22 downloads last month - 2 stars on GitHub - 1 maintainer
@lenml/llama2-tokenizer-vocab-gemma 1.1.1
llama2 tokenizer for javascript2 versions - Latest release: over 1 year ago - 15 downloads last month - 2 stars on GitHub - 1 maintainer
@lenml/llama2-tokenizer-vocab-baichuan2 1.1.1
llama2 tokenizer for javascript2 versions - Latest release: over 1 year ago - 8 downloads last month - 2 stars on GitHub - 1 maintainer
@lenml/llama2-tokenizer 1.1.4
Our library `@lenml/llama2-tokenizer` has been deprecated. We are excited to introduce our new li...6 versions - Latest release: over 1 year ago - 5.68 thousand downloads last month - 2 stars on GitHub - 1 maintainer
@leapward-koex/kuromoji 1.2.0
Forked version of kuromoji with fixes for edge extension review1 version - Latest release: 9 months ago - 11 downloads last month - 0 stars on GitHub - 1 maintainer
Top 7.0% on npmjs.org
5 versions - Latest release: about 11 years ago - 4 dependent packages - 26 dependent repositories - 1.56 thousand downloads last month - 6 stars on GitHub - 2 maintainers
remarked 0.1.4 π°
Markdown parser and lexer. A fork of marked.js maintained for Assemble.5 versions - Latest release: about 11 years ago - 4 dependent packages - 26 dependent repositories - 1.56 thousand downloads last month - 6 stars on GitHub - 2 maintainers
Top 1.8% on npmjs.org
13 versions - Latest release: about 2 years ago - 3 dependent packages - 6,744 dependent repositories - 7.22 million downloads last month - 2,679 stars on GitHub - 1 maintainer
@chevrotain/gast 11.0.3
Grammar AST structure for Chevrotain Parsers13 versions - Latest release: about 2 years ago - 3 dependent packages - 6,744 dependent repositories - 7.22 million downloads last month - 2,679 stars on GitHub - 1 maintainer
Top 5.7% on npmjs.org
206 versions - Latest release: 1 day ago - 2 dependent packages - 3 dependent repositories - 24 thousand downloads last month - 1 maintainer
clarity-pattern-parser 11.3.11
Parsing Library for Typescript and Javascript.206 versions - Latest release: 1 day ago - 2 dependent packages - 3 dependent repositories - 24 thousand downloads last month - 1 maintainer
@nguraijs/core 0.2.0
Lightweight and customizable tokenizer.2 versions - Latest release: 2 months ago - 35 downloads last month - 1 stars on GitHub - 1 maintainer
react-typeahead-tokenizer 1.1.1
React-based typeahead and typeahead-tokenizer5 versions - Latest release: over 8 years ago - 2 dependent packages - 1 dependent repositories - 43 downloads last month - 1 maintainer
@novel-segment/sort-dict-table 1.0.29 π°
Chinese word segmentation η°‘ηΉδΈζεθ―樑ε δ»₯ηΆ²θ·―ε°θͺͺηΊζ¨£ζ¬ εΊδΊ Node.js ηδΈζεθ―樑ε24 versions - Latest release: 11 months ago - 1 dependent package - 30 downloads last month - 11 stars on GitHub - 1 maintainer
Top 0.9% on npmjs.org
182 versions - Latest release: 7 months ago - 226 dependent packages - 2,098 dependent repositories - 761 thousand downloads last month - 993 stars on GitHub - 1 maintainer
react-bootstrap-typeahead 6.4.1
React typeahead with Bootstrap styling182 versions - Latest release: 7 months ago - 226 dependent packages - 2,098 dependent repositories - 761 thousand downloads last month - 993 stars on GitHub - 1 maintainer
text-tokenizer-utils 0.0.3
A class Tokenizer to convert text documents into sequences of tokens3 versions - Latest release: over 2 years ago - 0 downloads last month - 0 stars on GitHub - 1 maintainer
node-red-contrib-bert-tokenizer 1.0.4
A simple NodeRED module to implement bert-tokenizer5 versions - Latest release: over 5 years ago - 1 dependent package - 1 dependent repositories - 41 downloads last month - 0 stars on GitHub - 1 maintainer
tokenloom 1.0.9
A streaming text parser with custom tag and code fence detection for Node.js9 versions - Latest release: 11 days ago - 903 downloads last month - 1 stars on GitHub - 1 maintainer
oscript-ast-walker 0.1.1
A walker for nodes of the abstract syntax tree (AST) for the OScript language.5 versions - Latest release: over 3 years ago - 3 dependent packages - 2 dependent repositories - 19 downloads last month - 3 stars on GitHub - 1 maintainer
corpus_tools 1.0.0
Corpus Tools is a simple NPM package for performing corpus linguistic analysis.1 version - Latest release: over 4 years ago - 2 dependent packages - 1 dependent repositories - 8 downloads last month - 1 maintainer
kr-sentence 1.0.1
Light-weight sentence tokenizer for Korean. Supports both full-width and half-width punctuation m...1 version - Latest release: almost 4 years ago - 2 dependent packages - 1 downloads last month - 1 stars on GitHub - 1 maintainer
Top 4.2% on npmjs.org
2 versions - Latest release: over 9 years ago - 5 dependent packages - 7,586 dependent repositories - 1.61 million downloads last month - 5 stars on GitHub - 1 maintainer
glsl-token-inject-block 1.1.0
safely inject a block of tokens into a shader2 versions - Latest release: over 9 years ago - 5 dependent packages - 7,586 dependent repositories - 1.61 million downloads last month - 5 stars on GitHub - 1 maintainer
Related Keywords
parser
209
lexer
173
token
93
javascript
89
tokenize
67
ast
66
typescript
61
nlp
60
generator
54
compiler
53
stream
49
chatgpt
45
mistral
44
llama
42
llama2
40
react
39
typeahead
35
parse
34
css
34
machine learning
34
tokenization
34
gpt4
34
llama3
34
tokens
33
transfomers
33
baichuan
33
chatglm
33
nodejs
33
parsing
32
scanner
31
html
30
string
30
text
30
autocomplete
30
natural language processing
28
language
27
segmentation
26
NLP
26
async
26
flex
26
lex
25
chinese
25
split
25
word
24
syntax
24
japanese
23
streaming
23
jison
23
analyzer
22
openai
22
llm
22
regexp
21
css-parser
21
ai
20
segment
20
regex
20
pos
20
data
19
huggingface
19
chinese segmentation
18
dict
18
cli
18
jieba
18
δΈζ
18
morphological analyzer
18
react-bootstrap
18
optimizer
18
pos tagger
18
react-bootstrap-tokenizer
18
react-bootstrap-typeahead
18
bootstrap-typeahead
18
interpreter
18
bootstrap-tokenizer
18
bootstrap
18
text mining
18
react-component
17
grammar
17
stopword
17
phrases
17
transformers.js
17
gpt
17
transformers
17
εθ―
17
δΈζεθ―
17
css-in-js
16
traverser
16
json
16
PanGuSegment
16
PoS tagging
16
cssinjs
16
analysis
16
css-value-parser
16
novel
16
novel-segment
16
node-segment
16
opencc
16
pangu
16
segmentalization
16
ML
16
segmenter
16