An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

pypi.org "benchmark" keyword

mandelbrot-benchmark 1.1.4 💰
Benchmark Numba, Taichi, Warp, CuPy Kernel, Triton using Mandelbrot set
7 versions - Latest release: 12 months ago - 19 downloads last month - 3 stars on GitHub - 1 maintainer
causaldemand 1.0.0
CausalDemand: synthetic retail sales datasets for testing whether a method predicts how sales res...
1 version - Latest release: 1 day ago - 1 maintainer
warpt 0.4.1
Performance monitoring and system utilities
9 versions - Latest release: 5 months ago - 51 downloads last month - 2 stars on GitHub - 1 maintainer
Top 3.1% on pypi.org
google-benchmark 1.9.5
A library to benchmark code snippets.
12 versions - Latest release: 8 months ago - 6 dependent repositories - 15.3 thousand downloads last month - 10,454 stars on GitHub - 1 maintainer
fitbenchmarking 1.5.0
FitBenchmarking: A tool for comparing fitting software
23 versions - Latest release: 3 months ago - 1 dependent repositories - 1.02 thousand downloads last month - 17 stars on GitHub - 5 maintainers
pytest-park 0.3.2 💰
Organise and analyse your pytest benchmarks
10 versions - Latest release: 7 months ago - 184 downloads last month - 0 stars on GitHub - 1 maintainer
vllm-semantic-router-bench 1.0.0 💰
Comprehensive benchmark suite for semantic router vs direct vLLM evaluation across multiple reaso...
1 version - Latest release: about 1 year ago - 21 downloads last month - 6,035 stars on GitHub - 1 maintainer
chainbreak 0.1.1
An empirical benchmark for authorization behavior in delegated and agentic cloud systems
1 version - Latest release: 25 days ago - 143 downloads last month - 0 stars on GitHub - 1 maintainer
flexbench 0.2.0
FlexBench CLI - Docker orchestration for FlexAI Universal Benchmarking
2 versions - Latest release: about 1 year ago - 15 downloads last month - 12 stars on GitHub - 1 maintainer
gpu-benchmark-tool 0.6.5
Multi-vendor GPU health monitoring supporting old GPUs for e-waste reduction
40 versions - Latest release: about 1 year ago - 270 downloads last month - 1 maintainer
opencompass 0.5.4
A comprehensive toolkit for large model evaluation
31 versions - Latest release: about 1 month ago - 3.01 thousand downloads last month - 7,492 stars on GitHub - 1 maintainer
gym-electric-motor 3.0.3
A Farama Gymnasium environment for electric motor control.
16 versions - Latest release: 10 months ago - 1 dependent repositories - 658 downloads last month - 430 stars on GitHub - 2 maintainers
loadpup 1.0.0
专业的API性能压测工具,专门设计用于对大语言模型(LLM)API进行全面的性能测试和分析
1 version - Latest release: over 1 year ago - 12 downloads last month - 1 stars on GitHub - 1 maintainer
Top 1.8% on pypi.org
asv 0.6.6
Airspeed Velocity: A simple Python history benchmarking tool
24 versions - Latest release: 3 months ago - 42 dependent packages - 164 dependent repositories - 301 thousand downloads last month - 1,021 stars on GitHub - 6 maintainers
Top 1.3% on pypi.org
pytest-benchmark 5.3.0 💰
A ``pytest`` fixture for benchmarking code. It will group the tests into rounds that are calibrat...
51 versions - Latest release: about 1 month ago - 385 dependent packages - 3,470 dependent repositories - 9.94 million downloads last month - 1,455 stars on GitHub - 1 maintainer
llm-benchmark 0.5.4 💰
LLM Benchmark
36 versions - Latest release: 17 days ago - 998 downloads last month - 391 stars on GitHub - 1 maintainer
allama 0.1.18
A Python package for interacting with AI models
12 versions - Latest release: over 1 year ago - 91 downloads last month - 0 stars on GitHub - 1 maintainer
benchllama 0.2.7
Benchmark your local LLMs.
9 versions - Latest release: over 2 years ago - 46 downloads last month - 55 stars on GitHub - 1 maintainer
llm-bench 0.4.32
LLM Benchmarking tool for OLLAMA
31 versions - Latest release: about 2 years ago - 146 downloads last month - 0 stars on GitHub - 1 maintainer
lakebench 1.0.1
A multi-modal Python library for benchmarking Azure lakehouse engines and ELT scenarios, supporti...
31 versions - Latest release: 3 months ago - 287 downloads last month - 55 stars on GitHub - 1 maintainer
tacho 0.8.8
CLI tool for measuring and comparing LLM inference speeds
18 versions - Latest release: 10 months ago - 68 downloads last month - 2 stars on GitHub - 1 maintainer
mlx-train-perf 0.9.0 💰
Train longer sequences on your Mac: O(N)-memory attention, logit-free cross-entropy, and sequence...
11 versions - Latest release: 2 days ago - 407 downloads last month - 5 stars on GitHub - 1 maintainer
hardware-benchmark 0.1.4
A deep learning model hardware benchmarking tool
5 versions - Latest release: about 1 year ago - 475 downloads last month - 0 stars on GitHub - 1 maintainer
perfharness 1.0.3 💰
Tool for tracking python performance regressions
4 versions - Latest release: over 1 year ago - 14 downloads last month - 0 stars on GitHub - 1 maintainer
les-audits-affaires-eval-harness 1.3.2
Evaluation harness for Les Audits-Affaires LLM benchmark
7 versions - Latest release: over 1 year ago - 124 downloads last month - 1 maintainer
hpc-inspect 0.1
Intra Node Stencil Performance Evaluation Collection
1 version - Latest release: over 6 years ago - 1 dependent repositories - 18 downloads last month - 8 stars on GitHub - 2 maintainers
docile-benchmark 0.3.5
Tools to work with the DocILE dataset and benchmark
6 versions - Latest release: about 2 months ago - 673 downloads last month - 157 stars on GitHub - 1 maintainer
qpbenchmark 2.7.1
Benchmark for quadratic programming solvers available in Python.
14 versions - Latest release: 2 months ago - 1 dependent repositories - 75 downloads last month - 186 stars on GitHub - 1 maintainer
pymergen 0.5.0
PyMergen is a configuration-driven test automation tool for Linux.
1 version - Latest release: over 1 year ago - 18 downloads last month - 0 stars on GitHub - 1 maintainer
asrbench-cli 0.1.0
A command-line tool for the ASRBench framework, simplifying audio transcription system benchmarki...
1 version - Latest release: over 1 year ago - 107 downloads last month - 0 stars on GitHub
asrbench 0.1.0
Extensible, easy to use Python framework for evaluation of audio transcription systems.
1 version - Latest release: over 1 year ago - 195 downloads last month - 0 stars on GitHub
steadytext 2025.11.25
Deterministic text generation and embedding with zero configuration
37 versions - Latest release: 10 months ago - 181 downloads last month - 46 stars on GitHub - 1 maintainer
llama-optimus 0.1.9
llama-optimus is a lightweight Python tool to automatically optimize llama.cpp performance flags ...
8 versions - Latest release: over 1 year ago - 883 downloads last month - 29 stars on GitHub - 1 maintainer
Top 8.1% on pypi.org
matbench 0.6
a machine learning benchmark for materials science
5 versions - Latest release: about 4 years ago - 3 dependent packages - 2 dependent repositories - 375 downloads last month - 220 stars on GitHub - 2 maintainers
diffcheck 2.0.0
DiffCheck is a package to check the differences between two timber structures
28 versions - Latest release: 24 days ago - 278 downloads last month - 18 stars on GitHub - 1 maintainer
modern-benchmark 0.1.0
A Python package for benchmarking functions on modern versions of Python.
1 version - Latest release: over 1 year ago - 107 downloads last month - 0 stars on GitHub - 1 maintainer
llm-shield-proxy 1.6.10
Streaming privacy gateway for configurable PII transformation and OpenAI-compatible proxy traffic.
47 versions - Latest release: about 20 hours ago - 1.28 thousand downloads last month - 8 stars on GitHub - 1 maintainer
chunk-invariance 0.1.0
Assert that a streaming filter gives the same output however its input is split into chunks.
1 version - Latest release: 8 days ago - 143 downloads last month - 8 stars on GitHub - 1 maintainer
a-evolve 0.1.0
Evolve any agent across any domain using any evolution algorithm — with zero human intervention
1 version - Latest release: 6 months ago - 124 downloads last month - 805 stars on GitHub - 1 maintainer
Top 7.4% on pypi.org
pytest-codspeed 5.0.3
Pytest plugin to create CodSpeed benchmarks
42 versions - Latest release: 5 months ago - 14 dependent packages - 38 dependent repositories - 2.69 million downloads last month - 133 stars on GitHub - 1 maintainer
merlinquantum 0.4.0
Quantum neural network models using photonic circuits - Preview
10 versions - Latest release: 4 months ago - 443 downloads last month - 36 stars on GitHub - 1 maintainer
target_benchmark 0.1.3
Table Retrieval for Generative Tasks Benchmark
4 versions - Latest release: over 1 year ago - 75 downloads last month - 30 stars on GitHub - 1 maintainer
agentperf-local-rustcore 0.3.3
Optional Rust streaming client for agentperf-local.
6 versions - Latest release: 4 days ago - 779 downloads last month - 80 stars on GitHub - 1 maintainer
agentperf-local 0.3.3
Replay recorded agent tasks against an OpenAI-compatible endpoint.
6 versions - Latest release: 4 days ago - 1.39 thousand downloads last month - 80 stars on GitHub - 1 maintainer
researchharness 0.0.52
A lightweight, general harness for tool-using LLM agents.
17 versions - Latest release: 3 months ago - 92 downloads last month - 50 stars on GitHub - 1 maintainer
Top 8.0% on pypi.org
torchscan 0.2.0 💰
Useful information about your Pytorch module
4 versions - Latest release: 2 days ago - 6 dependent repositories - 697 downloads last month - 223 stars on GitHub - 1 maintainer
litraj 0.1
LiTraj: Li-ion migration dataset
1 version - Latest release: over 1 year ago - 19 downloads last month - 38 stars on GitHub - 1 maintainer
chimeraforge 0.50.0
LLM deployment optimizer (performance, cost, and safety) — backed by ~204,000 real measurements o...
73 versions - Latest release: 1 day ago - 4 thousand downloads last month - 2 stars on GitHub - 1 maintainer
plybench 2.1.1
PlyBench is a benchmark suite for evaluating the performance of LLMs and LLM Agents in simple gam...
8 versions - Latest release: 3 days ago - 551 downloads last month - 0 stars on GitHub - 1 maintainer
shardloom 0.3.3
Pre-release Python client for ShardLoom, a Vortex-first no-fallback evidence-certified local comp...
16 versions - Latest release: 6 days ago - 1.06 thousand downloads last month - 0 stars on GitHub - 1 maintainer
wasure 0.9
A WebAssembly benchmarking suite for precise performance measurement
5 versions - Latest release: over 1 year ago - 15 downloads last month - 12 stars on GitHub - 1 maintainer
Top 5.9% on pypi.org
redisbench-admin 0.12.39
Redis benchmark run helper. A wrapper around Redis and Redis Modules benchmark tools ( ftsb_redis...
402 versions - Latest release: 1 day ago - 1 dependent package - 3 dependent repositories - 9.69 thousand downloads last month - 10 stars on GitHub - 2 maintainers
Top 5.7% on pypi.org
optproblems 1.3
Infrastructure to define optimization problems and some test problems for black-box optimization
13 versions - Latest release: over 6 years ago - 3 dependent packages - 15 dependent repositories - 852 downloads last month - 1 maintainer
openchainbench 0.1.0
Python client for OpenChainBench live crypto infrastructure benchmarks
1 version - Latest release: 3 months ago - 8 downloads last month - 8 stars on GitHub - 1 maintainer
Top 7.5% on pypi.org
adbench 0.1.11
Python package of ADBench
14 versions - Latest release: about 3 years ago - 2 dependent repositories - 205 downloads last month - 1,024 stars on GitHub - 1 maintainer
mlx-chronos 0.5.1 💰
Benchmark suite and community leaderboard for local LLM inference on Apple Silicon
12 versions - Latest release: 1 day ago - 507 downloads last month - 25 stars on GitHub - 1 maintainer
pybenchmark 1.0.0
A benchmark utility used in performance tests.
7 versions - Latest release: about 10 years ago - 2 dependent repositories - 38 downloads last month - 22 stars on GitHub - 1 maintainer
sit4tfjs 1.0.0 💰
Simple Inference Test for TensorFlow.js. A benchmark inference speed testing tool for TensorFlow....
1 version - Latest release: over 1 year ago - 25 downloads last month - 4 stars on GitHub - 1 maintainer
agent-sprint-testkit 0.3.1
AgentSprint TestKit - Professional AI agent evaluation with OpenAI Evals integration
6 versions - Latest release: over 1 year ago - 185 downloads last month - 0 stars on GitHub - 1 maintainer
dtaudit 0.0.1
Diagnostic audits for drug-target affinity (DTA) benchmarks (name reserved; first release forthco...
1 version - Latest release: 3 days ago - 116 downloads last month - 1 maintainer
cloudanalyzer 0.5.0 💰
CLI-first QA toolkit for point cloud, trajectory, and 3D perception outputs
5 versions - Latest release: 9 days ago - 294 downloads last month - 17 stars on GitHub - 1 maintainer
posebench 1.1.0
Comprehensive benchmarking of protein-ligand structure prediction methods
7 versions - Latest release: 7 months ago - 32 downloads last month - 249 stars on GitHub - 1 maintainer
astral-dev-toolchain-cargo-codspeed 5.0.2
Cargo extension to build & run your codspeed benchmarks
2 versions - Latest release: 7 days ago - 38.6 thousand downloads last month - 75 stars on GitHub
tolokaforge 0.28.3
Tolokaforge - LLM Tool-Use Benchmarking Harness
76 versions - Latest release: 1 day ago - 7.25 thousand downloads last month - 19 stars on GitHub - 2 maintainers
robust-gymnasium 0.1.4
A standard API for robust reinforcement learning and a diverse set of reference environments (for...
5 versions - Latest release: about 1 year ago - 17 downloads last month - 103 stars on GitHub - 1 maintainer
lakebench-k8s 1.6.0
Deploy and benchmark lakehouse stacks on Kubernetes
19 versions - Latest release: 5 days ago - 294 downloads last month - 0 stars on GitHub - 1 maintainer
Top 4.4% on pypi.org
medmnist 3.0.2
[MedMNIST+] 18x Standardized Datasets for 2D and 3D Biomedical Image Classification with Multiple...
11 versions - Latest release: about 2 years ago - 3 dependent packages - 18 dependent repositories - 24.5 thousand downloads last month - 1,408 stars on GitHub - 1 maintainer
edma-core 0.1.0
EDMA — Evidence-Driven Model Architecture: a frozen, evidence-first cognitive runtime for AI agen...
1 version - Latest release: 4 days ago - 138 downloads last month - 0 stars on GitHub - 1 maintainer
eval-suite 2.1.4
User-friendly evaluation framework: Eval Suite & Benchmarks: UHGEval, HaluEval, HalluQA, etc.
3 versions - Latest release: almost 2 years ago - 18 downloads last month - 183 stars on GitHub - 1 maintainer
inferlab-integration-sglang 0.10.0
Inferlab adapter integration for SGLang
13 versions - Latest release: 27 days ago - 219 downloads last month - 3 stars on GitHub - 1 maintainer
inferlab-adapter-sdk 0.11.0
Generated protocol models and runtime helpers for Inferlab adapters
13 versions - Latest release: 27 days ago - 370 downloads last month - 3 stars on GitHub - 1 maintainer
inferlab-integration-specialized-engine 0.11.0
Inferlab adapter integration for token-only Specialized Engines
12 versions - Latest release: 1 day ago - 272 downloads last month - 3 stars on GitHub - 1 maintainer
inferlab-integration-vllm 0.10.0
Inferlab adapter integration for vLLM
13 versions - Latest release: 27 days ago - 395 downloads last month - 3 stars on GitHub - 1 maintainer
inferlab-integration-tensorrt-llm 0.10.0
Inferlab adapter integration for TensorRT-LLM
12 versions - Latest release: 27 days ago - 403 downloads last month - 3 stars on GitHub - 1 maintainer
inferlab-integration-tokenspeed 0.11.0
Inferlab adapter integration for TokenSpeed
13 versions - Latest release: 1 day ago - 373 downloads last month - 3 stars on GitHub - 1 maintainer
evalforge-lite 1.0.1
Compare text LLMs across OpenRouter, Amazon Bedrock, Google Vertex AI and Microsoft Foundry with ...
2 versions - Latest release: 4 days ago - 0 downloads last month - 0 stars on GitHub - 1 maintainer
evalne 0.4.0
Open Source Network Embedding Evaluation toolkit
4 versions - Latest release: over 4 years ago - 1 dependent repositories - 35 downloads last month - 107 stars on GitHub - 1 maintainer
apibeat 0.1.8
Benchmarking tool to compare REST and GraphQL API performance
9 versions - Latest release: over 1 year ago - 72 downloads last month - 0 stars on GitHub - 1 maintainer
hlido 0.2.0
Hlido — independent, evidence-backed trust scores for AI agents. One package: `pip install hlido`...
2 versions - Latest release: 3 months ago - 15 downloads last month - 0 stars on GitHub - 1 maintainer
hlido-trust 0.2.0
Vet AI agents with Hlido's independent, evidence-backed trust scores — drop-in tools for CrewAI, ...
2 versions - Latest release: 3 months ago - 19 downloads last month - 0 stars on GitHub - 1 maintainer
raman-bench 2.0.1
A large-scale benchmark for machine learning on Raman spectroscopy data
5 versions - Latest release: 17 days ago - 343 downloads last month - 22 stars on GitHub - 1 maintainer
pyinfer 0.0.3
Pyinfer is a model agnostic Python utility tool for ML developers and researchers to benchmark mo...
3 versions - Latest release: over 5 years ago - 1 dependent package - 2 dependent repositories - 1.11 thousand downloads last month - 25 stars on GitHub - 1 maintainer
riskbench-suite 0.1.1
A comprehensive toolkit for end-to-end risk-aware benchmarking of autonomous agents
2 versions - Latest release: over 1 year ago - 57 downloads last month - 0 stars on GitHub - 2 maintainers
lebse 0.2.0
LeBSE — legal-domain sentence embeddings adapted from LaBSE
1 version - Latest release: 3 months ago - 13 downloads last month - 3 stars on GitHub - 1 maintainer
llmsql 0.1.16
A Text2SQL benchmark for evaluation of Large Language Models
14 versions - Latest release: 7 months ago - 24 downloads last month - 42 stars on GitHub - 1 maintainer
finagent-redteam 0.2.0
A red-team benchmark for AI agents in financial workflows: measures regulatory-control bypass (sa...
1 version - Latest release: 4 months ago - 23 downloads last month - 0 stars on GitHub - 1 maintainer
tinyqabenchmarkpp 1.2.3 💰
A tiny synthetic QA LLM benchmark dataset generator using LiteLLM.
7 versions - Latest release: over 1 year ago - 47 downloads last month - 18 stars on GitHub - 1 maintainer
mikasa-robo-suite 1.0.0
Memory-intensive robotic manipulation benchmark for Vision-Language-Action research
6 versions - Latest release: 5 months ago - 275 downloads last month - 139 stars on GitHub - 1 maintainer
py_benchmark_functions 0.2.3
Python benchmark functions for Optimization with NumPy, TensorFlow and PyTorch support.
5 versions - Latest release: over 1 year ago - 94 downloads last month - 1 stars on GitHub - 1 maintainer
genoarmory 0.1.0
A DNA sequence Adversial attack and defense benchmark
1 version - Latest release: over 1 year ago - 8 downloads last month - 38 stars on GitHub - 1 maintainer
bigcodebench 0.2.5
"Evaluation package for BigCodeBench"
42 versions - Latest release: over 1 year ago - 1.66 thousand downloads last month - 516 stars on GitHub - 1 maintainer
agentdojo-core 0.0.3
Core code for AgentDojo
3 versions - Latest release: over 1 year ago - 29 downloads last month - 2 stars on GitHub - 1 maintainer
Top 1.8% on pypi.org
motmetrics 1.4.0
Metrics for multiple object tracker benchmarking.
9 versions - Latest release: almost 4 years ago - 13 dependent packages - 398 dependent repositories - 82.1 thousand downloads last month - 1,488 stars on GitHub - 1 maintainer
python-nano-bench 0.1.0
Control all Build Systems
1 version - Latest release: over 1 year ago - 6 downloads last month - 0 stars on GitHub - 1 maintainer
iperf3-lib 0.3.0
Programmable network benchmarking and analysis powered by libiperf.
3 versions - Latest release: 9 days ago - 479 downloads last month - 1 stars on GitHub - 1 maintainer
cuda-bench 0.3.0
CUDA Kernel Benchmarking Package with CUDA 12.x and 13.x runtime support
4 versions - Latest release: 2 months ago - 5.17 thousand downloads last month - 932 stars on GitHub - 4 maintainers
agent-belt 0.2.1
Evaluation harness for real headless CLI agents - reproducible multi-turn scenarios, rule + LLM s...
5 versions - Latest release: 2 months ago - 181 downloads last month - 17 stars on GitHub - 1 maintainer
factly-eval 1.0.1
CLI tool to evaluate ChatGPT factuality on MMLU benchmark.
1 version - Latest release: over 1 year ago - 57 downloads last month - 2 stars on GitHub - 1 maintainer
tsmd 0.1.4
The TSMD project brings together Motif Discovery methods for Time Series, aiming to compare their...
5 versions - Latest release: about 1 year ago - 41 downloads last month - 9 stars on GitHub - 1 maintainer
csoai-gspc 0.2.20260928 💰
Read the live GSPC AI-governance board and verify its Ed25519-signed measurement cards. Three sta...
14 versions - Latest release: 7 days ago - 2.27 thousand downloads last month - 1 stars on GitHub - 1 maintainer