Ecosyste.ms: Packages
An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.
packagist.org "tokenizer" keyword
lahaxearnaud/laravel-token v0.5.1
Laravel 4 token management6 versions - Latest release: over 9 years ago - 4 dependent repositories - 392 downloads total - 10 stars on GitHub - 1 maintainer
nocika/gpt3-tokenizer
PHP package for Byte Pair Encoding (BPE) used by GPT-3.Latest release: about 1 month ago - 15 downloads total - 0 stars on GitHub - 1 maintainer
phplrt/lexer-contracts 3.6.5
A set of phplrt lexer abstractions51 versions - Latest release: 3 months ago - 8 dependent packages - 9 dependent repositories - 85.8 thousand downloads total - 1 stars on GitHub - 1 maintainer
smaakvoldelen/tokenizer 1.0.1 removed 💰
A tool that uses regular expressions to split a given string into tokens.2 versions - Latest release: 4 months ago - 3 downloads total - 1 maintainer
Top 3.4% on packagist.org
1 version - Latest release: over 9 years ago - 14 dependent packages - 9 dependent repositories - 124 thousand downloads total - 48 stars on GitHub - 1 maintainer
jeremeamia/functionparser 1.0.0 abandoned
Function parser for PHP functions, methods, and closures1 version - Latest release: over 9 years ago - 14 dependent packages - 9 dependent repositories - 124 thousand downloads total - 48 stars on GitHub - 1 maintainer
sikessem/runtime removed
📦️ Install the SIKessEm Runtime (ske)Latest release: over 1 year ago - 0 downloads total - 0 stars on GitHub - 1 maintainer
aurora-project/tokenizer 1.0.0 removed
Library for navigating PHP parser tokens1 version - Latest release: about 2 years ago
skel/core removed
📦️ Install the SIKessEm Runtime (ske)Latest release: over 1 year ago - 0 downloads total - 0 stars on GitHub - 1 maintainer
apigen/token-reflection 1.4.0 removed
Library emulating the PHP internal reflection using just the tokenized source code.4 versions - Latest release: almost 10 years ago - 2 dependent repositories
Related Keywords
php
37
parser
31
token
18
lexer
13
openai
10
decode
9
encode
9
function
7
closure
7
GPT-3
7
nlp
6
ast
6
gpt-2
6
library
5
serialize
5
query
5
javascript
5
php-library
5
tiktoken
5
tokens
5
tokenization
5
html
5
serializable
5
bpe
4
generator
4
laravel
4
lambda
4
compressor
4
reflection
4
parsing
4
xml
3
language
3
php7
3
translator
3
stream
3
regex
3
compiler
3
php8
3
archiver
3
search
3
math
3
formatter
3
sikessem
3
tokenize
3
engine
2
setup
2
string
2
css
2
minify
2
minifier
2
stanford
2
reformat
2
format
2
sql server
2
mysql
2
sql
2
json
2
ner
2
edismax
2
corenlp-server
2
lemma
2
openie
2
cl100k_base
2
script-compiler
2
runtime
2
template
2
installer
2
object
2
console-application
2
class
2
cli-app
2
binary
2
ske
2
lexical-analysis
2
lex
2
phplrt
2
indonesia
2
bnf
2
abnf
2
pcre
2
ngram
2
solr
2
validator
2
HTML5
2
code-style
2
calculator
2
elasticsearch
2
mathematicator
2
pacc
1
invocation-locators
1
text
1
attribute
1
analysis
1
mutable
1
attributed
1
span
1
tokenized
1
scanner
1
pcre2
1
filters
1