Top 1.2% on pypi.org
Top 0.3% downloads on pypi.org
Top 0.2% dependent packages on pypi.org
Top 0.2% dependent repos on pypi.org
Top 3.8% forks on pypi.org
Top 1.0% docker downloads on pypi.org
Top 0.3% downloads on pypi.org
Top 0.2% dependent packages on pypi.org
Top 0.2% dependent repos on pypi.org
Top 3.8% forks on pypi.org
Top 1.0% docker downloads on pypi.org
pypi.org : evaluate
HuggingFace community-driven open-source library of evaluation
Registry
-
Source
- Documentation
- JSON
purl: pkg:pypi/evaluate
Keywords:
metrics
, machine
, learning
, evaluate
, evaluation
, machine-learning
License: Apache-2.0
Latest release: 7 months ago
First release: over 5 years ago
Dependent packages: 222
Dependent repositories: 2,474
Downloads: 2,830,287 last month
Stars: 1,762 on GitHub
Forks: 215 on GitHub
Docker dependents: 161
Docker downloads: 24,936,750
Total Commits: 904
Committers: 128
Average commits per author: 7.063
Development Distribution Score (DDS): 0.873
More commit stats: commits.ecosyste.ms
See more repository details: repos.ecosyste.ms
Last synced: about 8 hours ago
generate-sequences 0.0.9
A python package to generate from deep learning models10 versions - Latest release: 10 months ago - 485 downloads last month - 0 stars on GitHub - 1 maintainer
tailwiz 0.0.24
24 versions - Latest release: 10 months ago - 427 downloads last month - 0 stars on GitHub - 2 maintainers
Top 1.8% on pypi.org
995 versions - Latest release: 10 months ago - 3 dependent packages - 15 dependent repositories - 162 thousand downloads last month - 6,566 stars on GitHub - 1 maintainer
autogluon.multimodal 1.1.1
Fast and Accurate ML in 3 Lines of Code995 versions - Latest release: 10 months ago - 3 dependent packages - 15 dependent repositories - 162 thousand downloads last month - 6,566 stars on GitHub - 1 maintainer
keycare 0.0.18
KeyCARE is a Python library designed for the unsupervised keyword extraction from biomedical docu...18 versions - Latest release: 10 months ago - 403 downloads last month - 1 maintainer
h2ogpt 0.2.1
Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports...2 versions - Latest release: 11 months ago - 95 downloads last month - 11,767 stars on GitHub - 1 maintainer
aepo 0.1.6
Annotation Efficient Preference Optimization4 versions - Latest release: 11 months ago - 206 downloads last month - 1 stars on GitHub - 1 maintainer
bleuscore 0.1.3
A fast bleu score calculator4 versions - Latest release: 11 months ago - 658 downloads last month - 10 stars on GitHub - 1 maintainer
autodistill-distilbert 0.1.0
DistilBERT model for use with Autodistill1 version - Latest release: 11 months ago - 62 downloads last month - 0 stars on GitHub - 1 maintainer
Top 7.9% on pypi.org
23 versions - Latest release: 11 months ago - 1 dependent package - 2 dependent repositories - 876 downloads last month - 183 stars on GitHub - 1 maintainer
jury 2.3.1
Evaluation toolkit for neural language generation.23 versions - Latest release: 11 months ago - 1 dependent package - 2 dependent repositories - 876 downloads last month - 183 stars on GitHub - 1 maintainer
openseneca 0.1.1
OpenSeneca6 versions - Latest release: 11 months ago - 174 downloads last month - 1 maintainer
Top 6.7% on pypi.org
150 versions - Latest release: 12 months ago - 3 dependent packages - 1 dependent repositories - 1.58 thousand downloads last month - 2,664 stars on GitHub - 1 maintainer
deepsparse-nightly 1.8.0.20240502
An inference runtime offering GPU-class performance on CPUs and APIs to integrate ML into your ap...150 versions - Latest release: 12 months ago - 3 dependent packages - 1 dependent repositories - 1.58 thousand downloads last month - 2,664 stars on GitHub - 1 maintainer
visualizing-training 1.0.0
Modelling Training Dynamics and Interpreting the dynamics1 version - Latest release: 12 months ago - 56 downloads last month - 1 stars on GitHub - 1 maintainer
responsibleai-text 0.2.7
SDK API to assess text Machine Learning models.14 versions - Latest release: 12 months ago - 463 downloads last month - 1,517 stars on GitHub - 1 maintainer
audioaugmentor 0.1.1
Python package for simple application of wide range of audio augmentations.11 versions - Latest release: 12 months ago - 425 downloads last month - 4 stars on GitHub - 1 maintainer
that-nlp-library 0.2.2
Aim to be a convenient NLP library with the help from HuggingFace15 versions - Latest release: 12 months ago - 534 downloads last month - 8 stars on GitHub - 1 maintainer
Top 8.9% on pypi.org
12 versions - Latest release: 12 months ago - 1 dependent package - 1 dependent repositories - 3.6 thousand downloads last month - 15,898 stars on GitHub - 3 maintainers
evals 3.0.1
Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.12 versions - Latest release: 12 months ago - 1 dependent package - 1 dependent repositories - 3.6 thousand downloads last month - 15,898 stars on GitHub - 3 maintainers
aisploit 0.0.25
Tiny package designed to support red teams and penetration testers in exploiting large language m...25 versions - Latest release: 12 months ago - 1.17 thousand downloads last month - 23 stars on GitHub - 1 maintainer
llm-blender 0.0.2
LLM-Blender, an innovative ensembling framework to attain consistently superior performance by le...2 versions - Latest release: 12 months ago - 9.78 thousand downloads last month - 928 stars on GitHub - 1 maintainer
prompt2model 0.1.0
A library for distilling models from prompts.10 versions - Latest release: 12 months ago - 388 downloads last month - 1,991 stars on GitHub - 1 maintainer
nixietune 0.0.7
A semantic search embedding model fine-tuning tool5 versions - Latest release: about 1 year ago - 204 downloads last month - 2 stars on GitHub - 1 maintainer
query-package-documentation 0.0.3
A package to explore documentations3 versions - Latest release: about 1 year ago - 103 downloads last month - 1 stars on GitHub - 1 maintainer
danoliterate 0.1.0
Benchmark of Generative Large Language Models in Danish10 versions - Latest release: about 1 year ago - 348 downloads last month - 2 stars on GitHub - 1 maintainer
consistencybench 0.1.3
Tools and Techniques for Consistency Benchmarking4 versions - Latest release: about 1 year ago - 1 dependent package - 160 downloads last month - 1 maintainer
llm-finetune 0.1.0
LLM Finetuning resource hub + toolkit1 version - Latest release: about 1 year ago - 626 stars on GitHub - 1 maintainer
llm-tune-test 0.1.0
LLM Finetuning resource hub + toolkit1 version - Latest release: about 1 year ago - 626 stars on GitHub - 1 maintainer
future-shot 0.0.1
FutureShot: Few-Shot Learning for high-dimensional classification problems1 version - Latest release: about 1 year ago - 61 downloads last month - 4 stars on GitHub - 1 maintainer
llm-finetuning-hub 0.1.0
LLM Finetuning resource hub + toolkit1 version - Latest release: about 1 year ago - 105 downloads last month - 626 stars on GitHub - 1 maintainer
hyfi-ml 0.2.0
HyFI-ML is a Python package that extends the Hydra Fast Interface (HyFI) framework with machine l...2 versions - Latest release: about 1 year ago - 92 downloads last month - 0 stars on GitHub - 1 maintainer
Top 7.7% on pypi.org
81 versions - Latest release: about 1 year ago - 2 dependent packages - 1 dependent repositories - 27.5 thousand downloads last month - 1 maintainer
azureml-metrics 47.0.0
Contains the ML and non-Azure specific common code associated with AzureML metrics.81 versions - Latest release: about 1 year ago - 2 dependent packages - 1 dependent repositories - 27.5 thousand downloads last month - 1 maintainer
nm-transformers 1.7.0.43401
State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow4 versions - Latest release: about 1 year ago - 3 dependent packages - 1 dependent repositories - 113 downloads last month - 9 stars on GitHub - 1 maintainer
gmrev 0.1.16
Librería para evaluar sistemas de generación mejorada por recuperación43 versions - Latest release: about 1 year ago - 536 downloads last month - 0 stars on GitHub - 1 maintainer
nm-transformers-nightly 1.7.0.20240304
State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow37 versions - Latest release: about 1 year ago - 3 dependent packages - 223 downloads last month - 9 stars on GitHub - 1 maintainer
geniusrise-text 0.1.13
Text bolts for geniusrise14 versions - Latest release: about 1 year ago - 436 downloads last month - 5 stars on GitHub - 1 maintainer
localcat 0.0.10 💰
Fine-tune Large Language Models locally.6 versions - Latest release: about 1 year ago - 99 downloads last month - 0 stars on GitHub - 1 maintainer
llm-serve 0.0.1
An LLM inference solution to quickly deploy productive LLM service1 version - Latest release: about 1 year ago - 45 downloads last month - 1 maintainer
autogluon-tonyhu-test.multimodal 1.0.5b20240302
AutoML for Image, Text, and Tabular Data5 versions - Latest release: about 1 year ago - 1 dependent package - 788 downloads last month - 8,039 stars on GitHub - 1 maintainer
geniusrise-audio 0.1.12
audio bolts for geniusrise13 versions - Latest release: about 1 year ago - 387 downloads last month - 2 stars on GitHub - 1 maintainer
domino-code-assist 1.4.2
17 versions - Latest release: about 1 year ago - 26.5 thousand downloads last month - 3 maintainersgeniusrise-vision 0.1.5
Huggingface bolts for geniusrise8 versions - Latest release: about 1 year ago - 276 downloads last month - 6 stars on GitHub - 1 maintainer
autogoal-transformers 0.8.4
transformers algorithm library wrapper for AutoGOAL6 versions - Latest release: about 1 year ago - 245 downloads last month - 0 stars on GitHub - 1 maintainer
arthur-bench 0.3.1
validate models for production6 versions - Latest release: about 1 year ago - 350 downloads last month - 406 stars on GitHub - 1 maintainer
codebook-features 0.1.2
Sparse and discrete interpretability tool for neural networks3 versions - Latest release: about 1 year ago - 121 downloads last month - 61 stars on GitHub - 1 maintainer
separability 0.9.2
LLM Tools for looking at separability of LLM Capabilities4 versions - Latest release: about 1 year ago - 1 dependent repositories - 192 downloads last month - 9 stars on GitHub - 1 maintainer
waffle-hub 0.3.1
Waffle hub31 versions - Latest release: about 1 year ago - 1 thousand downloads last month - 40 stars on GitHub - 1 maintainer
ragchain 0.2.6
Build advanced RAG workflows with LLM, compatible with Langchain13 versions - Latest release: over 1 year ago - 462 downloads last month - 279 stars on GitHub - 1 maintainer
nlpia2 0.0.42
Natural language processing utilities and examples for the book Natural Language Processing in Ac...24 versions - Latest release: over 1 year ago - 1 dependent repositories - 701 downloads last month - 4 stars on gitlab.com - 3 maintainers
sevals 0.0.3
A framework for evaluating language models4 versions - Latest release: over 1 year ago - 115 downloads last month - 0 stars on GitHub - 1 maintainer
mbr 0.2.0
Minimum Bayes risk decoding for Hugging Face Transformers2 versions - Latest release: over 1 year ago - 1 dependent repositories - 115 downloads last month - 57 stars on GitHub - 1 maintainer
pykoi 0.0.10
pykoi: Active learning in one unified interface11 versions - Latest release: over 1 year ago - 1 dependent repositories - 348 downloads last month - 410 stars on GitHub - 1 maintainer
dbgpt-hub 0.3.1
DB-GPT-Hub: Text-to-SQL parsing with LLMs2 versions - Latest release: over 1 year ago - 242 downloads last month - 1,742 stars on GitHub - 1 maintainer
openrl 0.2.0
unified reinforcement learning framework24 versions - Latest release: over 1 year ago - 1 dependent package - 874 downloads last month - 720 stars on GitHub - 1 maintainer
newsframes 1.3.7 💰
Easy-to-use, high-quality identification of generic framing dimensions in English news articles7 versions - Latest release: over 1 year ago - 212 downloads last month - 4 stars on GitHub - 1 maintainer
qkogpt 0.0.4
Quantized KoGPT3 versions - Latest release: over 1 year ago - 92 downloads last month - 1 maintainer
sequence-classifier 0.1.4
5 versions - Latest release: over 1 year ago - 1 dependent package - 1 dependent repositories - 208 downloads last month - 1 maintainerpyatp 1.8.5
a iflytek ailab library ...57 versions - Latest release: over 1 year ago - 572 downloads last month - 1 maintainer
clarinpl-embeddings 0.0.1rc7078377068125
Embeddings: State-of-the-art Text Representations for Natural Language Processing tasks, an initi...117 versions - Latest release: over 1 year ago - 1 dependent repositories - 3.66 thousand downloads last month - 36 stars on GitHub - 1 maintainer
sig-networks 0.2.0
Neural networks for longitudinal NLP classification tasks.2 versions - Latest release: over 1 year ago - 100 downloads last month - 4 stars on GitHub - 1 maintainer
nlpsig 0.2.2
Path signatures for Natural Language Processing.4 versions - Latest release: over 1 year ago - 1 dependent package - 149 downloads last month - 4 stars on GitHub - 1 maintainer
easy-testing 0.2.2
Framework for testing12 versions - Latest release: over 1 year ago - 544 downloads last month - 2 maintainers
dc-lms 1.0.0
3 versions - Latest release: over 1 year ago - 99 downloads last month - 17 stars on GitHub - 1 maintaineroptimum-deepsparse 0.1.0.dev1
Optimum DeepSparse is an extension of the Hugging Face Transformers library that integrates the D...2 versions - Latest release: over 1 year ago - 3 dependent packages - 156 downloads last month - 3,130 stars on GitHub - 1 maintainer
fastrepl 0.0.25
Fast Run-Eval-Polish Loop for LLM App25 versions - Latest release: over 1 year ago - 427 downloads last month - 0 stars on GitHub - 1 maintainer
elpis 0.2.2
A library to perform automatic speech recognition with huggingface transformers.12 versions - Latest release: over 1 year ago - 1 dependent repositories - 458 downloads last month - 3 stars on GitHub - 1 maintainer
kogitune 0.1
The Kogitune 🦊 LLM Project1 version - Latest release: over 1 year ago - 59 downloads last month - 1 stars on GitHub - 1 maintainer
nutmegredundancysolver 0.0.13
13 versions - Latest release: over 1 year ago - 1 dependent package - 471 downloads last month - 1 maintainerquasarx 0.0.2 💰
quasar - Pytorch1 version - Latest release: over 1 year ago - 31 downloads last month - 8 stars on GitHub - 1 maintainer
indomain 0.0.5
Domain Adapted Language Modeling Toolkit - E2E RAG7 versions - Latest release: over 1 year ago - 292 downloads last month - 263 stars on GitHub - 2 maintainers
geniusrise-huggingface 0.4.9 removed
Huggingface bolts for geniusrise13 versions - Latest release: over 1 year ago - 286 downloads last month - 3 stars on GitHub - 1 maintainer
instruct-qa 0.0.2
Empirical evaluation of retrieval-augmented instruction-following models.10 versions - Latest release: over 1 year ago - 211 downloads last month - 83 stars on GitHub - 2 maintainers
nlg-metricverse 0.9.9
An End-to-End Library for Evaluating Natural Language Generation.8 versions - Latest release: over 1 year ago - 1 dependent repositories - 516 downloads last month - 91 stars on GitHub - 2 maintainers
Top 9.4% on pypi.org
37 versions - Latest release: over 1 year ago - 1 dependent repositories - 1.13 thousand downloads last month - 157 stars on GitHub - 1 maintainer
jac-nlp 1.4.2
The Official Jaseci Code Repository37 versions - Latest release: over 1 year ago - 1 dependent repositories - 1.13 thousand downloads last month - 157 stars on GitHub - 1 maintainer
luna-nlg 1.0.0
Source code for the LUNA project1 version - Latest release: over 1 year ago - 65 downloads last month - 12 stars on GitHub - 1 maintainer
xturing 0.1.8
Fine-tuning, evaluation and data generation for LLMs19 versions - Latest release: over 1 year ago - 569 downloads last month - 1 maintainer
spice_agent 0.1.18
spice agent19 versions - Latest release: over 1 year ago - 669 downloads last month - 1 maintainer
axlearn 0.0.1
AXLearn3 versions - Latest release: over 1 year ago - 267 downloads last month - 142,992 stars on GitHub - 1 maintainer
fincompass 0.1.2 removed
A comprehensive toolkit for large model evaluation2 versions - Latest release: over 1 year ago - 1 maintainer
nerblackbox 1.0.0
a high-level library for named entity recognition in python12 versions - Latest release: over 1 year ago - 1 dependent repositories - 534 downloads last month - 23 stars on GitHub - 1 maintainer
huggingface-tool 0.0.6
Toolkit for managing huggingface models and datasets6 versions - Latest release: over 1 year ago - 227 downloads last month - 5 stars on GitHub - 1 maintainer
zyh 0.0.2
Easily show off amazing scripys by zyh2 versions - Latest release: almost 2 years ago - 99 downloads last month - 0 stars on GitHub - 1 maintainer
llamatune 0.4.0
Haven's Tuning Library for LLM finetuning8 versions - Latest release: almost 2 years ago - 210 downloads last month - 1 maintainer
textclassification 0.2.4 removed
Create Text Classification Fine-Tunning Model3 versions - Latest release: almost 2 years ago - 9 downloads last month - 1 maintainer
stitchnet 0.2.2
3 versions - Latest release: almost 2 years ago - 117 downloads last month - 1 maintainerllm-lens 0.0.0.3
llm-lens is a Python package for CV as NLP, where you can run very descriptive image modules on i...2 versions - Latest release: almost 2 years ago - 1 dependent repositories - 186 downloads last month - 352 stars on GitHub - 2 maintainers
navigate-with-image-language-model 1.2
The package provides a pipeline that utilizes models like ClipSeg and StableDiffusion or ClipSeg ...2 versions - Latest release: almost 2 years ago - 100 downloads last month - 1 maintainer
navigation-with-image-language-model 2.0 removed
The package provides a pipeline that utilizes models like ClipSeg and StableDiffusion or ClipSeg ...2 versions - Latest release: almost 2 years ago - 1 maintainer
Top 9.5% on pypi.org
9 versions - Latest release: almost 2 years ago - 2 dependent repositories - 373 downloads last month - 552 stars on GitHub - 1 maintainer
openicl 0.1.8
An open source framework for in-context learning.9 versions - Latest release: almost 2 years ago - 2 dependent repositories - 373 downloads last month - 552 stars on GitHub - 1 maintainer
nlptest 1.5.0
John Snow Labs provides a library for delivering safe & effective NLP models.41 versions - Latest release: almost 2 years ago - 1 dependent repositories - 637 downloads last month - 498 stars on GitHub - 2 maintainers
postocr-3stages 0.1.0
Post-OCR correction in 3 stages.1 version - Latest release: almost 2 years ago - 65 downloads last month - 0 stars on GitHub - 1 maintainer
ifly-ailab 1.0.0
a iflytek ailab library ...5 versions - Latest release: almost 2 years ago - 226 downloads last month - 1 maintainer
lmflow-finetune 0.0.3
LMFlow: Large Model Flow.3 versions - Latest release: almost 2 years ago - 1 dependent repositories - 131 downloads last month - 8,170 stars on GitHub - 1 maintainer
lmflow-vision 0.0.1
LMFlow: Large Model Flow.1 version - Latest release: almost 2 years ago - 53 downloads last month - 8,400 stars on GitHub - 1 maintainer
lmflow-diffusion 0.0.1
LMFlow: Large Model Flow.1 version - Latest release: almost 2 years ago - 54 downloads last month - 8,400 stars on GitHub - 1 maintainer
lmflow-deploy 0.0.1
LMFlow: Large Model Flow.1 version - Latest release: almost 2 years ago - 54 downloads last month - 8,400 stars on GitHub - 1 maintainer
lmflow-inference 0.0.1
LMFlow: Large Model Flow.1 version - Latest release: almost 2 years ago - 55 downloads last month - 8,400 stars on GitHub - 1 maintainer
lmflow-evaluate 0.0.1
LMFlow: Large Model Flow.1 version - Latest release: almost 2 years ago - 48 downloads last month - 8,170 stars on GitHub - 1 maintainer
lmflow-eval 0.0.1
LMFlow: Large Model Flow.1 version - Latest release: almost 2 years ago - 52 downloads last month - 8,400 stars on GitHub - 1 maintainer
lmflow-inferencer 0.0.1
LMFlow: Large Model Flow.1 version - Latest release: almost 2 years ago - 56 downloads last month - 8,400 stars on GitHub - 1 maintainer
lmflow-benchmark 0.0.1
LMFlow: Large Model Flow.1 version - Latest release: almost 2 years ago - 45 downloads last month - 8,400 stars on GitHub - 1 maintainer
lmflow-pretrainer 0.0.1
LMFlow: Large Model Flow.1 version - Latest release: almost 2 years ago - 51 downloads last month - 8,400 stars on GitHub - 1 maintainer
lmflow-pretrain 0.0.1
LMFlow: Large Model Flow.1 version - Latest release: almost 2 years ago - 54 downloads last month - 8,400 stars on GitHub - 1 maintainer
Past Dependents
Include Past Dependents
Check this option to include packages that no longer depend on this package in their latest version but previously did.
Filter by kind
All kinds
runtime
113
dev
4
extra == 'dev'
3
extra == "qa"
2
extra == 'deepspeed-testing'
2
extra == 'dev-tensorflow'
2
extra == 'testing'
2
extra == 'dev-torch'
2
dev-torch
2
dev-tensorflow
2
deepspeed-testing
2
testing
2
all
2
extra == "evaluate"
1
extra == "eval"
1
extra == "dev"
1
extra == 'example'
1
extra == "code-generation"
1
extra == "chat-completion"
1
extra == "bert-score"
1
extra == 'backend'
1
extra == "all"
1
exhaustive
1
evaluate
1
transformers
1
tfm_ner
1
model
1
ft
1
extra == "translation"
1
extra == "transformers"
1
extra == "text-generation"
1
extra == "text"
1
extra == 'tests'
1
extra == 'test'
1
extra == "test"
1
extra == "summarization"
1
extra == "rlhf"
1
extra == 'nlp_test'
1
extra == 'nlp'
1
extra == "multimodal"
1
extra == "llm"
1
extra == "generative-ai"
1
extra == "fine-tuning"
1
extra == "fill-mask"
1
compat_tests
1