npmjs.org "tokenize" keyword
Top 1.4% on npmjs.org
14 versions - Latest release: about 1 year ago - 16 dependent packages - 49,392 dependent repositories - 79 million downloads last month - 1,769 stars on GitHub - 1 maintainer
micromark-util-subtokenize 2.1.0 💰
micromark utility to tokenize subtokens14 versions - Latest release: about 1 year ago - 16 dependent packages - 49,392 dependent repositories - 79 million downloads last month - 1,769 stars on GitHub - 1 maintainer
Top 5.5% on npmjs.org
7 versions - Latest release: almost 5 years ago - 5 dependent packages - 44 dependent repositories - 11.4 thousand downloads last month - 5 stars on GitHub - 1 maintainer
complexion 0.2.2
A fast tokenizer/lexer core that is well tested.7 versions - Latest release: almost 5 years ago - 5 dependent packages - 44 dependent repositories - 11.4 thousand downloads last month - 5 stars on GitHub - 1 maintainer
Top 1.4% on npmjs.org
10 versions - Latest release: over 1 year ago - 15 dependent packages - 49,391 dependent repositories - 86.5 million downloads last month - 1,769 stars on GitHub - 1 maintainer
micromark-factory-title 2.0.1 💰
micromark factory to parse markdown titles (found in resources, definitions)10 versions - Latest release: over 1 year ago - 15 dependent packages - 49,391 dependent repositories - 86.5 million downloads last month - 1,769 stars on GitHub - 1 maintainer
cry-search 1.0.3
A fast, memory-efficient search library for large datasets with support for tokenized matching, l...3 versions - Latest release: about 2 months ago - 1 maintainer
Top 1.4% on npmjs.org
9 versions - Latest release: over 1 year ago - 14 dependent packages - 49,392 dependent repositories - 82.9 million downloads last month - 1,576 stars on GitHub - 1 maintainer
micromark-util-html-tag-name 2.0.1 💰
micromark utility with list of html tag names9 versions - Latest release: over 1 year ago - 14 dependent packages - 49,392 dependent repositories - 82.9 million downloads last month - 1,576 stars on GitHub - 1 maintainer
Top 1.3% on npmjs.org
8 versions - Latest release: over 1 year ago - 26 dependent packages - 49,418 dependent repositories - 87.5 million downloads last month - 2,150 stars on GitHub - 1 maintainer
micromark-util-resolve-all 2.0.1 💰
micromark utility to resolve subtokens8 versions - Latest release: over 1 year ago - 26 dependent packages - 49,418 dependent repositories - 87.5 million downloads last month - 2,150 stars on GitHub - 1 maintainer
Top 1.4% on npmjs.org
6 versions - Latest release: over 1 year ago - 17 dependent packages - 48,161 dependent repositories - 81.7 million downloads last month - 2,147 stars on GitHub - 1 maintainer
micromark-util-decode-string 2.0.1 💰
micromark utility to decode markdown strings6 versions - Latest release: over 1 year ago - 17 dependent packages - 48,161 dependent repositories - 81.7 million downloads last month - 2,147 stars on GitHub - 1 maintainer
Top 9.4% on npmjs.org
4 versions - Latest release: over 11 years ago - 2 dependent packages - 10 dependent repositories - 892 downloads last month - 1 stars on GitHub - 1 maintainer
tokenize 0.0.3
Tokenize a string with regex.4 versions - Latest release: over 11 years ago - 2 dependent packages - 10 dependent repositories - 892 downloads last month - 1 stars on GitHub - 1 maintainer
Top 1.7% on npmjs.org
32 versions - Latest release: 18 days ago - 315 dependent packages - 144,051 dependent repositories - 98.8 million downloads last month - 211 stars on GitHub - 2 maintainers
mdast-util-from-markdown 2.0.3 💰
mdast utility to parse markdown32 versions - Latest release: 18 days ago - 315 dependent packages - 144,051 dependent repositories - 98.8 million downloads last month - 211 stars on GitHub - 2 maintainers
Top 1.3% on npmjs.org
8 versions - Latest release: over 1 year ago - 29 dependent packages - 49,411 dependent repositories - 82.9 million downloads last month - 2,150 stars on GitHub - 1 maintainer
micromark-util-combine-extensions 2.0.1 💰
micromark utility to combine syntax or html extensions8 versions - Latest release: over 1 year ago - 29 dependent packages - 49,411 dependent repositories - 82.9 million downloads last month - 2,150 stars on GitHub - 1 maintainer
Top 1.2% on npmjs.org
8 versions - Latest release: over 1 year ago - 28 dependent packages - 49,409 dependent repositories - 90.1 million downloads last month - 2,105 stars on GitHub - 1 maintainer
micromark-util-normalize-identifier 2.0.1 💰
micromark utility normalize identifiers (as found in references, definitions)8 versions - Latest release: over 1 year ago - 28 dependent packages - 49,409 dependent repositories - 90.1 million downloads last month - 2,105 stars on GitHub - 1 maintainer
wat-tokenizer 0.3.0 💰
Tokenize WebAssembly Text-format into list of lists2 versions - Latest release: about 8 years ago - 1 dependent repositories - 8 downloads last month - 8 stars on GitHub - 1 maintainer
Top 1.1% on npmjs.org
9 versions - Latest release: over 1 year ago - 86 dependent packages - 49,751 dependent repositories - 120 million downloads last month - 2,150 stars on GitHub - 1 maintainer
micromark-util-symbol 2.0.1 💰
micromark utility with symbols9 versions - Latest release: over 1 year ago - 86 dependent packages - 49,751 dependent repositories - 120 million downloads last month - 2,150 stars on GitHub - 1 maintainer
Top 1.2% on npmjs.org
16 versions - Latest release: about 1 year ago - 34 dependent packages - 49,395 dependent repositories - 82.3 million downloads last month - 1,576 stars on GitHub - 1 maintainer
micromark-core-commonmark 2.0.3 💰
The CommonMark markdown constructs16 versions - Latest release: about 1 year ago - 34 dependent packages - 49,395 dependent repositories - 82.3 million downloads last month - 1,576 stars on GitHub - 1 maintainer
Top 1.2% on npmjs.org
8 versions - Latest release: over 1 year ago - 33 dependent packages - 49,418 dependent repositories - 90 million downloads last month - 1,769 stars on GitHub - 1 maintainer
micromark-util-chunked 2.0.1 💰
micromark utility to splice and push with giant arrays8 versions - Latest release: over 1 year ago - 33 dependent packages - 49,418 dependent repositories - 90 million downloads last month - 1,769 stars on GitHub - 1 maintainer
Top 1.1% on npmjs.org
11 versions - Latest release: over 1 year ago - 82 dependent packages - 49,741 dependent repositories - 109 million downloads last month - 2,147 stars on GitHub - 1 maintainer
micromark-util-character 2.1.1 💰
micromark utility to handle character codes11 versions - Latest release: over 1 year ago - 82 dependent packages - 49,741 dependent repositories - 109 million downloads last month - 2,147 stars on GitHub - 1 maintainer
Top 1.4% on npmjs.org
9 versions - Latest release: over 1 year ago - 16 dependent packages - 49,516 dependent repositories - 94.8 million downloads last month - 2,150 stars on GitHub - 1 maintainer
micromark-util-encode 2.0.1 💰
micromark utility to encode dangerous html characters9 versions - Latest release: over 1 year ago - 16 dependent packages - 49,516 dependent repositories - 94.8 million downloads last month - 2,150 stars on GitHub - 1 maintainer
Top 1.3% on npmjs.org
8 versions - Latest release: over 1 year ago - 22 dependent packages - 49,415 dependent repositories - 83.8 million downloads last month - 2,150 stars on GitHub - 1 maintainer
micromark-util-classify-character 2.0.1 💰
micromark utility to classify whether a character is whitespace or punctuation8 versions - Latest release: over 1 year ago - 22 dependent packages - 49,415 dependent repositories - 83.8 million downloads last month - 2,150 stars on GitHub - 1 maintainer
Top 1.5% on npmjs.org
33 versions - Latest release: almost 5 years ago - 943 dependent packages - 3,779,990 dependent repositories - 42.3 million downloads last month - 226 stars on GitHub - 3 maintainers
snapdragon 0.12.1
Easy-to-use plugin system for creating powerful, fast and versatile parsers and compilers, with b...33 versions - Latest release: almost 5 years ago - 943 dependent packages - 3,779,990 dependent repositories - 42.3 million downloads last month - 226 stars on GitHub - 3 maintainers
Top 1.4% on npmjs.org
8 versions - Latest release: over 1 year ago - 15 dependent packages - 49,391 dependent repositories - 84.4 million downloads last month - 1,769 stars on GitHub - 1 maintainer
micromark-factory-destination 2.0.1 💰
micromark factory to parse destinations (found in resources, definitions)8 versions - Latest release: over 1 year ago - 15 dependent packages - 49,391 dependent repositories - 84.4 million downloads last month - 1,769 stars on GitHub - 1 maintainer
Top 1.3% on npmjs.org
9 versions - Latest release: over 1 year ago - 26 dependent packages - 49,519 dependent repositories - 91.8 million downloads last month - 1,769 stars on GitHub - 1 maintainer
micromark-util-sanitize-uri 2.0.1 💰
micromark utility to sanitize urls9 versions - Latest release: over 1 year ago - 26 dependent packages - 49,519 dependent repositories - 91.8 million downloads last month - 1,769 stars on GitHub - 1 maintainer
Top 1.4% on npmjs.org
10 versions - Latest release: over 1 year ago - 15 dependent packages - 49,391 dependent repositories - 86.8 million downloads last month - 1,769 stars on GitHub - 1 maintainer
micromark-factory-label 2.0.1 💰
micromark factory to parse labels (found in media, definitions)10 versions - Latest release: over 1 year ago - 15 dependent packages - 49,391 dependent repositories - 86.8 million downloads last month - 1,769 stars on GitHub - 1 maintainer
Top 1.2% on npmjs.org
8 versions - Latest release: over 1 year ago - 57 dependent packages - 49,586 dependent repositories - 94 million downloads last month - 2,150 stars on GitHub - 1 maintainer
micromark-factory-space 2.0.1 💰
micromark factory to parse markdown space (found in lots of places)8 versions - Latest release: over 1 year ago - 57 dependent packages - 49,586 dependent repositories - 94 million downloads last month - 2,150 stars on GitHub - 1 maintainer
Top 1.2% on npmjs.org
8 versions - Latest release: over 1 year ago - 31 dependent packages - 49,517 dependent repositories - 86.8 million downloads last month - 2,150 stars on GitHub - 1 maintainer
micromark-factory-whitespace 2.0.1 💰
micromark factory to parse markdown whitespace (found in lots of places)8 versions - Latest release: over 1 year ago - 31 dependent packages - 49,517 dependent repositories - 86.8 million downloads last month - 2,150 stars on GitHub - 1 maintainer
Top 1.3% on npmjs.org
25 versions - Latest release: 3 months ago - 2,251 dependent packages - 1,940,799 dependent repositories - 455 million downloads last month - 550 stars on GitHub - 1 maintainer
js-tokens 10.0.0
Tiny JavaScript tokenizer.25 versions - Latest release: 3 months ago - 2,251 dependent packages - 1,940,799 dependent repositories - 455 million downloads last month - 550 stars on GitHub - 1 maintainer
Top 1.1% on npmjs.org
11 versions - Latest release: about 1 year ago - 95 dependent packages - 49,745 dependent repositories - 102 million downloads last month - 2,150 stars on GitHub - 1 maintainer
micromark-util-types 2.0.2 💰
micromark utility with a couple of typescript types11 versions - Latest release: about 1 year ago - 95 dependent packages - 49,745 dependent repositories - 102 million downloads last month - 2,150 stars on GitHub - 1 maintainer
Top 1.4% on npmjs.org
6 versions - Latest release: over 1 year ago - 17 dependent packages - 49,462 dependent repositories - 83.3 million downloads last month - 1,576 stars on GitHub - 1 maintainer
micromark-util-decode-numeric-character-reference 2.0.2 💰
micromark utility to decode numeric character references6 versions - Latest release: over 1 year ago - 17 dependent packages - 49,462 dependent repositories - 83.3 million downloads last month - 1,576 stars on GitHub - 1 maintainer
Top 2.1% on npmjs.org
12 versions - Latest release: almost 6 years ago - 32 dependent packages - 8,222 dependent repositories - 3.48 million downloads last month - 67 stars on GitHub - 1 maintainer
html-tokenize 2.0.1
transform stream to tokenize html12 versions - Latest release: almost 6 years ago - 32 dependent packages - 8,222 dependent repositories - 3.48 million downloads last month - 67 stars on GitHub - 1 maintainer
@referralgps/selectra 1.0.10 💰
Selectra — A powerful, extensible <select> UI control for tagging, contact lists, country selecto...7 versions - Latest release: 1 day ago - 538 downloads last month - 2 stars on GitHub - 2 maintainers
Top 3.4% on npmjs.org
2 versions - Latest release: about 8 years ago - 11 dependent packages - 1,043,711 dependent repositories - 2.48 million downloads last month - 14 stars on GitHub - 1 maintainer
babel-extract-comments 1.0.0 💰
Uses babel (babylon) to extract JavaScript code comments from a JavaScript string or file.2 versions - Latest release: about 8 years ago - 11 dependent packages - 1,043,711 dependent repositories - 2.48 million downloads last month - 14 stars on GitHub - 1 maintainer
Top 3.3% on npmjs.org
13 versions - Latest release: over 1 year ago - 23 dependent packages - 30 dependent repositories - 1.48 thousand downloads last month - 2,150 stars on GitHub - 1 maintainer
micromark-build 2.0.3 💰
Build production files from dev files13 versions - Latest release: over 1 year ago - 23 dependent packages - 30 dependent repositories - 1.48 thousand downloads last month - 2,150 stars on GitHub - 1 maintainer
Top 3.7% on npmjs.org
7 versions - Latest release: over 1 year ago - 89 dependent packages - 7,730 dependent repositories - 18.2 million downloads last month - 40 stars on GitHub - 2 maintainers
hast-util-from-html 2.0.3 💰
hast utility to parse from HTML7 versions - Latest release: over 1 year ago - 89 dependent packages - 7,730 dependent repositories - 18.2 million downloads last month - 40 stars on GitHub - 2 maintainers
Top 1.0% on npmjs.org
42 versions - Latest release: about 1 year ago - 144 dependent packages - 151,327 dependent repositories - 96.8 million downloads last month - 2,150 stars on GitHub - 1 maintainer
micromark 4.0.2 💰
small commonmark compliant markdown parser with positional info and concrete tokens42 versions - Latest release: about 1 year ago - 144 dependent packages - 151,327 dependent repositories - 96.8 million downloads last month - 2,150 stars on GitHub - 1 maintainer
intl-segmenter 1.0.0 💰
A high-performance wrapper around `Intl.Segmenter` for efficient text segmentation. This class re...1 version - Latest release: about 1 year ago - 42 downloads last month - 14 stars on GitHub - 1 maintainer
Top 5.2% on npmjs.org
15 versions - Latest release: over 5 years ago - 4 dependent packages - 4 dependent repositories - 4.21 thousand downloads last month - 40 stars on GitHub - 1 maintainer
html-tokenizer 4.0.1
Small, fast, event-driven, fault-tolerant html tokenizer. Works in node or browsers.15 versions - Latest release: over 5 years ago - 4 dependent packages - 4 dependent repositories - 4.21 thousand downloads last month - 40 stars on GitHub - 1 maintainer
Top 2.1% on npmjs.org
43 versions - Latest release: 8 months ago - 17 dependent packages - 48 dependent repositories - 300 thousand downloads last month - 1,290 stars on GitHub - 1 maintainer
wink-nlp 2.4.0
Developer friendly Natural Language Processing ✨43 versions - Latest release: 8 months ago - 17 dependent packages - 48 dependent repositories - 300 thousand downloads last month - 1,290 stars on GitHub - 1 maintainer
lexos 0.0.3
A very simple lexer generator3 versions - Latest release: 6 months ago - 5 downloads last month - 0 stars on GitHub - 1 maintainer
glsl-100-to-300 1.2.0
transpiles GLSL tokens from v100 to v300 es2 versions - Latest release: about 10 years ago - 1 dependent package - 1 dependent repositories - 11 downloads last month - 40 stars on GitHub - 1 maintainer
Top 9.6% on npmjs.org
4 versions - Latest release: over 7 years ago - 3 dependent packages - 36 dependent repositories - 3.52 thousand downloads last month - 0 stars on GitHub - 3 maintainers
@nxmix/tokenize-ansi 3.0.0
tokenize a string that includes ansi code4 versions - Latest release: over 7 years ago - 3 dependent packages - 36 dependent repositories - 3.52 thousand downloads last month - 0 stars on GitHub - 3 maintainers
@fsnjs/truthy 1.0.0
Easily assert that values are NonNullable or Nullable.1 version - Latest release: over 1 year ago - 4 downloads last month - 0 stars on GitHub - 1 maintainer
parse5-wxmp 3.0.3 💰
HTML parsing/serialization toolset for Node.js. WHATWG HTML Living Standard (aka HTML5)-compliant.1 version - Latest release: over 8 years ago - 2 dependent packages - 1 dependent repositories - 5 downloads last month - 3,852 stars on GitHub - 1 maintainer
onaml 0.0.4
Oh No, Another Markup Language. Based on YAML, but stores considerably more object metadata.4 versions - Latest release: over 10 years ago - 1 dependent package - 1 dependent repositories - 2 downloads last month - 1 maintainer
snapdragon-scanner 1.0.0
Easily scan a string with an object of regex patterns to produce an array of tokens. ~100 sloc.1 version - Latest release: over 7 years ago - 1 dependent repositories - 9 downloads last month - 7 stars on GitHub - 1 maintainer
ngramable 0.2.1 💰
String ngram splitter.3 versions - Latest release: over 6 years ago - 1 dependent package - 2 dependent repositories - 16 downloads last month - 2 stars on GitHub - 1 maintainer
trustami-ai 1.0.4
A connector for the Trustami.AI API.5 versions - Latest release: over 2 years ago - 5 downloads last month - 0 stars on GitHub - 1 maintainer
node-icu-tokenizer 1.0.4
String Tokenizer for Node.js using ICU's BreakIterators5 versions - Latest release: about 9 years ago - 1 dependent package - 1 dependent repositories - 23 downloads last month - 1 maintainer
mixed-english-and-chinese-tokenizer 1.1.0
Tokenizes utterances that contain a mix of English and Chinese words.8 versions - Latest release: over 7 years ago - 2 dependent packages - 1 dependent repositories - 43 downloads last month - 5 stars on GitHub - 1 maintainer
opentoken-lib 1.0.1
Used to interact with the OpenToken API.2 versions - Latest release: about 9 years ago - 1 dependent package - 1 dependent repositories - 4 downloads last month - 0 stars on GitHub - 1 maintainer
Top 3.5% on npmjs.org
4 versions - Latest release: almost 4 years ago - 14 dependent packages - 12,219 dependent repositories - 1.23 million downloads last month - 16 stars on GitHub - 1 maintainer
json-lexer 1.2.0
lexing json4 versions - Latest release: almost 4 years ago - 14 dependent packages - 12,219 dependent repositories - 1.23 million downloads last month - 16 stars on GitHub - 1 maintainer
positional-tokenizer 1.0.6
Tokenizes a text using regex rules and returns the tokens with their positions in the text given.6 versions - Latest release: about 3 years ago - 11 downloads last month - 0 stars on GitHub - 1 maintainer
attributes-parser 2.2.3
Parsing and tokenizing attributes string14 versions - Latest release: over 1 year ago - 8 dependent packages - 1 dependent repositories - 94.3 thousand downloads last month - 3 stars on GitHub - 1 maintainer
json-operations 0.0.6
json-operations allows you to work with JSON, parse in a standardized way, get tokenize JSON, etc...6 versions - Latest release: about 3 years ago - 2 dependent packages - 18 downloads last month - 1 maintainer
@sciactive/tokenizer 3.4.0
Tokenize string input into 32-bit integers (for full text search and natural language processing).9 versions - Latest release: 7 days ago - 483 downloads last month - 1 maintainer
nomark 1.0.1
Transform hypertext strings (e.g., HTML, Markdown) into plain text for natural language processin...2 versions - Latest release: over 1 year ago - 12 downloads last month - 2 stars on GitHub - 1 maintainer
methodius 2.0.1
A utility for analyzing text to find bigrams, trigrams, and other n-grams.6 versions - Latest release: almost 2 years ago - 2 dependent packages - 1 dependent repositories - 14 downloads last month - 5 stars on GitHub - 1 maintainer
monaco-editor-node 0.3.0 💰
Wrapper of monaco-editor to use some functionalities in Node.js.12 versions - Latest release: about 4 years ago - 1 dependent package - 32 downloads last month - 1 stars on GitHub - 1 maintainer
@flex-development/fsm-tokenizer 1.0.0-alpha.1 💰
finite state machine tokenizer1 version - Latest release: 12 months ago - 80 downloads last month - 3 stars on GitHub - 1 maintainer
sentence-tokenization 1.0.2
Tokenizer for tokenizing sentences, for BERT or other NLP preprocessing.3 versions - Latest release: almost 7 years ago - 2 dependent repositories - 2 downloads last month - 6 stars on GitHub - 1 maintainer
count-pieces 0.1.0
Gives you an inventory of tokens needed to write a given program1 version - Latest release: about 9 years ago - 2 dependent packages - 1 dependent repositories - 4 downloads last month - 0 stars on GitHub - 1 maintainer
Top 9.2% on npmjs.org
29 versions - Latest release: about 2 years ago - 11 dependent packages - 26 dependent repositories - 91 downloads last month - 5 stars on GitHub - 1 maintainer
happynodetokenizer 7.1.0 💰
A simple, Twitter-aware tokenizer.29 versions - Latest release: about 2 years ago - 11 dependent packages - 26 dependent repositories - 91 downloads last month - 5 stars on GitHub - 1 maintainer
lexis 0.0.4
a very simple lexer4 versions - Latest release: over 11 years ago - 1 dependent package - 1 dependent repositories - 16 downloads last month - 2 stars on GitHub - 1 maintainer
ngramjs 0.1.3
String ngram splitter.4 versions - Latest release: 12 months ago - 129 downloads last month - 0 stars on GitHub - 1 maintainer
text-prep-lite 0.0.3
Lightweight text preprocessing utilities for NLP in TypeScript.3 versions - Latest release: 8 months ago - 5 downloads last month - 0 stars on GitHub - 1 maintainer
Top 5.7% on npmjs.org
11 versions - Latest release: about 1 month ago - 5 dependent packages - 801 dependent repositories - 1.89 million downloads last month - 2 stars on GitHub - 4 maintainers
@stdlib/string-base-format-tokenize 0.2.4 💰
Tokenize a string into an array of string parts and format identifier objects.11 versions - Latest release: about 1 month ago - 5 dependent packages - 801 dependent repositories - 1.89 million downloads last month - 2 stars on GitHub - 4 maintainers
poketh 0.6.11
string/array poking for pet params11 versions - Latest release: about 11 years ago - 1 dependent package - 1 dependent repositories - 33 downloads last month - 1 maintainer
Top 8.9% on npmjs.org
17 versions - Latest release: over 6 years ago - 8 dependent packages - 1 dependent repositories - 60 downloads last month - 14 stars on GitHub - 1 maintainer
webgl-to-opengl 0.0.17 💰
Convert GLSL from WebGL to OpenGL17 versions - Latest release: over 6 years ago - 8 dependent packages - 1 dependent repositories - 60 downloads last month - 14 stars on GitHub - 1 maintainer
bracketter 0.0.4
Parse and manipulate tokenized strings with ease!4 versions - Latest release: over 2 years ago - 23 downloads last month - 0 stars on GitHub - 1 maintainer
@calin.dodon/text-splitter-core 1.0.0
> 🧩 A tiny, tree-shakable text chunking utility with zero dependencies.1 version - Latest release: 9 months ago - 8 downloads last month - 1 maintainer
3h-ast 0.14.0
A simple AST generator.25 versions - Latest release: about 4 years ago - 3 dependent packages - 1 dependent repositories - 8 downloads last month - 0 stars on GitHub - 1 maintainer
tokenize-insa-station-name 1.1.0 💰
Get search tokens from an INSA/NASA station name.2 versions - Latest release: about 6 years ago - 1 dependent package - 8 downloads last month - 0 stars on GitHub - 1 maintainer
@technobuddha/library 2.0.3
A large library of useful functions20 versions - Latest release: 6 months ago - 3 dependent packages - 1 dependent repositories - 56 downloads last month - 0 stars on GitHub - 1 maintainer
bocss-lexer 1.0.3
Lexer to tokenize CSS4 versions - Latest release: almost 4 years ago - 3 dependent packages - 6 downloads last month - 1 maintainer
Top 4.4% on npmjs.org
2 versions - Latest release: about 11 years ago - 8 dependent packages - 9,559 dependent repositories - 2.85 million downloads last month - 6 stars on GitHub - 1 maintainer
css-tokenize 1.0.1
Transform stream that tokenizes CSS2 versions - Latest release: about 11 years ago - 8 dependent packages - 9,559 dependent repositories - 2.85 million downloads last month - 6 stars on GitHub - 1 maintainer
jsgex 1.0.0
Not a replacement for REGEX, but an alternative that is far more readable and a bit more flexible1 version - Latest release: over 7 years ago - 1 dependent package - 1 dependent repositories - 4 downloads last month - 0 stars on GitHub - 1 maintainer
js-akashcry-token 5.0.0
A regex that tokenizes JavaScript.2 versions - Latest release: about 2 years ago - 1 downloads last month - 0 stars on GitHub - 1 maintainer
split-words 1.0.2
Easy way to split words3 versions - Latest release: over 10 years ago - 3 dependent packages - 5 dependent repositories - 7 downloads last month - 0 stars on GitHub - 1 maintainer
imago-mixed-eng-chi-tokenizer 1.0.0
Lemmize and tokenize string which contains Chinese and English words1 version - Latest release: almost 7 years ago - 1 dependent package - 1 dependent repositories - 3 downloads last month - 1 maintainer
simple-token 0.1.0
Create a string representation of an object or primative2 versions - Latest release: over 11 years ago - 1 dependent package - 4 dependent repositories - 2.94 thousand downloads last month - 1 stars on GitHub - 1 maintainer
@axontraining-sistemas/tokenize-card 1.0.2
Package for card's tokenization3 versions - Latest release: over 3 years ago - 10 downloads last month - 1 maintainer
ogden 1.0.6 💰
Basic English tokenizer7 versions - Latest release: over 9 years ago - 1 dependent package - 1 dependent repositories - 9 downloads last month - 1 stars on GitHub - 1 maintainer
snapdragon-stack 2.1.0
Snapdragon utility for creating a stack.4 versions - Latest release: about 8 years ago - 1 dependent package - 10 dependent repositories - 286 downloads last month - 4 stars on GitHub - 1 maintainer
@legt/token-service 0.0.0
Middleware for calling blockchain interactions in a web2 environment1 version - Latest release: almost 2 years ago - 3 downloads last month - 0 stars on GitHub - 1 maintainer
@flowaccount/node-icu-tokenizer 2.0.0
String Tokenizer for Node.js using ICU's BreakIterators13 versions - Latest release: over 2 years ago - 1 dependent package - 1.32 thousand downloads last month - 3 stars on GitHub - 5 maintainers
mashtml 1.2.0
html tokenizer2 versions - Latest release: over 3 years ago - 2 dependent packages - 0 downloads last month - 1 maintainer
qb-json-tok-writer 2.1.1
Quick writer for transforming tokenized JSON back into JSON/UTF-8 output (works with qb-json-tok)4 versions - Latest release: almost 9 years ago - 1 dependent package - 1 dependent repositories - 5 downloads last month - 1 maintainer
tokenizer-of-html 1.0.2
A HTML tokenizer3 versions - Latest release: about 9 years ago - 1 dependent package - 1 dependent repositories - 6 downloads last month - 1 stars on GitHub - 1 maintainer
complexion-time-map 1.1.0
Time a JavaScript tokenizer4 versions - Latest release: over 8 years ago - 2 dependent packages - 4 dependent repositories - 34 downloads last month - 0 stars on GitHub - 1 maintainer
lexer-neo 1.0.1
convert a string to list of words2 versions - Latest release: almost 4 years ago - 1 dependent package - 0 downloads last month - 1 maintainer
jsx-to-hyperscript 2.2.1
A CLI command for converting JSX files into standard JS.17 versions - Latest release: almost 3 years ago - 1 dependent repositories - 140 downloads last month - 1 stars on GitHub - 2 maintainers
rn-braintree 0.2.0
React Native interface to Braintree's Android and iOS SDKs2 versions - Latest release: about 9 years ago - 1 dependent package - 1 dependent repositories - 6 downloads last month - 1 stars on GitHub - 1 maintainer
@parsing-kit/lexer 2.3.0-alpha.8
JavaScript library for tokenizing strings with regular expressions9 versions - Latest release: about 3 years ago - 2 dependent packages - 3 downloads last month - 1 maintainer
prop-replace 0.0.3
Replaces spots in a string with their values in an object. Super simple templating engine.3 versions - Latest release: over 5 years ago - 1 dependent package - 3 dependent repositories - 31 downloads last month - 1 stars on GitHub - 1 maintainer
generic-matcher 1.3.0
General purpose generic matcher5 versions - Latest release: over 4 years ago - 1 dependent package - 23 downloads last month - 0 stars on GitHub - 1 maintainer
hx-tokenizer 1.0.0
Split text into tokens.2 versions - Latest release: about 9 years ago - 1 dependent package - 1 dependent repositories - 8 downloads last month - 2 stars on GitHub - 1 maintainer
lexr 1.0.4
Lexical analyzer built in Javascript26 versions - Latest release: over 7 years ago - 1 dependent package - 2 dependent repositories - 109 downloads last month - 16 stars on GitHub - 1 maintainer
bayes-tokens 0.0.1721
get word counts / frequencies on a per-speaker or per-category basis, or as an aggregate7 versions - Latest release: over 3 years ago - 1 dependent package - 3 downloads last month - 27 stars on GitHub - 1 maintainer
@isdk/llama-node 2.0.3
Low-level Node.js bindings for llama.cpp. Core library for running LLM models locally with native...3 versions - Latest release: 3 months ago - 58 downloads last month - 1 maintainer
mini-tokenizer 0.1.2
tiny tokenizer for simple parsing3 versions - Latest release: almost 12 years ago - 3 dependent packages - 6 dependent repositories - 392 downloads last month - 6 stars on GitHub - 1 maintainer
qb-json-tok 2.2.2
Fast (~300 MB/sec) and light (~1.3 kb) JSON/UTF-8 tokenizer for creating custom parsers17 versions - Latest release: almost 9 years ago - 2 dependent packages - 1 dependent repositories - 4 downloads last month - 0 stars on GitHub - 1 maintainer
@liquicode/lib-tokenize 0.1.4
A NodeJS library to tokenize strings5 versions - Latest release: over 5 years ago - 2 dependent packages - 1 dependent repositories - 10 downloads last month - 0 stars on GitHub - 1 maintainer
Related Keywords
parse
73
tokenizer
69
token
58
lexer
39
parser
38
ast
36
markdown
29
compile
28
render
28
javascript
25
cst
24
gfm
24
html
23
string
23
commonmark
22
unified
22
utility
21
micromark
21
nlp
20
tokens
20
util
20
text
19
lex
19
nodejs
17
node
15
split
13
transform
11
scanner
11
convert
10
js
10
lexical
9
tokenization
9
regex
9
code
9
typescript
8
compiler
8
analysis
8
css
8
parsing
8
stream
7
syntax
7
search
7
scan
6
json
6
normalize
6
snapdragon
6
comment
6
jonschlinkert
6
fast
6
words
6
tokenise
6
JSON
5
serialize
5
htmlparser
5
html5 parser
5
html parser
5
regexp
5
factory
5
html5
5
tokeniser
5
cli
5
replace
5
character
5
tags
5
glob
4
replacer
4
es
4
extract
4
context
4
language
4
unist
4
glsl
4
stackgl
4
natural-language-processing
4
glslify
4
NLP
4
strip
4
jsx
4
tagger
4
word
4
mdast
4
WHATWG
3
specification
3
jsx-transform
3
frequency
3
stem
3
excel
3
flex
3
gl
3
webgl
3
webgl2
3
300
3
scanning
3
stemmer
3
splitter
3
match
3
strings
3
hyperscript
3
formula
3
ngrams
3