Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

pypi.org "llm-inference" keyword

openvino-nightly 2024.2.0.dev20240515
OpenVINO(TM) Runtime
118 versions - Latest release: about 7 hours ago - 1 dependent repositories - 36.9 thousand downloads last month - 5,906 stars on GitHub - 1 maintainer
lorax-client 0.5.1
LoRAX Python Client
9 versions - Latest release: about 9 hours ago - 2.06 thousand downloads last month - 516 stars on GitHub - 1 maintainer
Top 10.0% on pypi.org
genv 1.4.3
GPU environment and cluster management
13 versions - Latest release: about 13 hours ago - 1 dependent package - 1 dependent repositories - 1.27 thousand downloads last month - 412 stars on GitHub - 1 maintainer
Top 1.2% on pypi.org
bentoml 1.2.16
BentoML: Build Production-Grade AI Applications
147 versions - Latest release: about 15 hours ago - 13 dependent packages - 499 dependent repositories - 90.5 thousand downloads last month - 6,591 stars on GitHub - 3 maintainers
git-llm 0.1.3
The project integrates Git with a llm (OpenAI, LlamaCpp, and GPT-4-All) to extend the capabilitie...
3 versions - Latest release: 12 months ago - 45 downloads last month - 63,563 stars on GitHub - 1 maintainer
arcee-py 1.0.6
The Arcee client for executing domain-adpated language model routines
19 versions - Latest release: 20 days ago - 306 downloads last month - 14 stars on GitHub - 2 maintainers
autogenstudio 0.0.56
AutoGen Studio
59 versions - Latest release: about 2 months ago - 3 dependent packages - 17.5 thousand downloads last month - 18,528 stars on GitHub - 1 maintainer
superduperdb 0.1.1
🔮 Super-power your database with AI 🔮
24 versions - Latest release: 3 months ago - 1 dependent repositories - 448 downloads last month - 4,305 stars on GitHub - 1 maintainer
llm-hygiene 0.0.1
a data preprocessing toolkit that makes it easy to create common LLM-related data structures; fro...
1 version - Latest release: 5 months ago - 13 downloads last month - 1 stars on GitHub - 1 maintainer
edsl 0.1.22
Create and analyze LLM-based surveys
27 versions - Latest release: 2 days ago - 984 downloads last month - 20 stars on GitHub - 3 maintainers
neural-speed 1.0
Repository of Intel® Intel Extension for Transformers
5 versions - Latest release: about 2 months ago - 2.73 thousand downloads last month - 179 stars on GitHub - 1 maintainer
Top 2.4% on pypi.org
pyautogen 0.2.27
Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework
51 versions - Latest release: 16 days ago - 37 dependent packages - 3 dependent repositories - 169 thousand downloads last month - 18,528 stars on GitHub - 2 maintainers
Top 0.7% on pypi.org
openvino-dev 2024.1.0
OpenVINO(TM) Development Tools
32 versions - Latest release: 22 days ago - 38 dependent packages - 498 dependent repositories - 211 thousand downloads last month - 5,906 stars on GitHub - 1 maintainer
Top 0.7% on pypi.org
openvino 2024.1.0
OpenVINO(TM) Runtime
33 versions - Latest release: 22 days ago - 47 dependent packages - 342 dependent repositories - 336 thousand downloads last month - 5,906 stars on GitHub - 1 maintainer
Top 3.9% on pypi.org
openllm-client 0.4.44
OpenLLM Client: Interacting with OpenLLM HTTP/gRPC server, or any BentoML server.
74 versions - Latest release: 3 months ago - 2 dependent packages - 8 dependent repositories - 13.7 thousand downloads last month - 8,657 stars on GitHub - 1 maintainer
Top 2.2% on pypi.org
openllm 0.4.44
OpenLLM: Operating LLMs in production
153 versions - Latest release: 3 months ago - 3 dependent packages - 295 dependent repositories - 16.8 thousand downloads last month - 8,657 stars on GitHub - 2 maintainers
sibila 0.4.3
Structured queries from local or online LLM models
13 versions - Latest release: 3 days ago - 198 downloads last month - 14 stars on GitHub - 1 maintainer
litgpt 0.3.1
Hackable implementation of state-of-the-art open-source LLMs
6 versions - Latest release: 9 days ago - 1 dependent package - 8.9 thousand downloads last month - 6,784 stars on GitHub - 3 maintainers
Top 6.4% on pypi.org
intel-extension-for-transformers 1.4.1
Repository of Intel® Intel Extension for Transformers
15 versions - Latest release: 26 days ago - 1 dependent package - 4 dependent repositories - 4.45 thousand downloads last month - 1,909 stars on GitHub - 1 maintainer
Top 1.4% on pypi.org
gpt4all 2.6.0
Python bindings for GPT4All
50 versions - Latest release: 21 days ago - 46 dependent packages - 168 dependent repositories - 73.3 thousand downloads last month - 63,563 stars on GitHub - 1 maintainer
friendli-client 1.3.4
Client of Friendli Suite.
13 versions - Latest release: about 1 month ago - 4 dependent packages - 3.66 thousand downloads last month - 33 stars on GitHub - 1 maintainer
exxa 0.6.4 💰
Exa - Pytorch
58 versions - Latest release: about 1 month ago - 2 dependent packages - 1 dependent repositories - 617 downloads last month - 19 stars on GitHub - 1 maintainer
bisheng-pyautogen 0.3.0
Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework
10 versions - Latest release: about 1 month ago - 2 dependent packages - 282 downloads last month - 18,528 stars on GitHub - 1 maintainer
llm-vm 0.1.55
An Open-Source AGI Server for Open-Source LLMs
7 versions - Latest release: 6 months ago - 58 downloads last month - 455 stars on GitHub - 3 maintainers
Top 3.5% on pypi.org
openllm-core 0.4.44
OpenLLM Core: Core components for OpenLLM.
74 versions - Latest release: 3 months ago - 2 dependent packages - 8 dependent repositories - 13.9 thousand downloads last month - 8,657 stars on GitHub - 1 maintainer
autogencap-rajan.jedi 0.0.6
CAP w/ autogen bindings
1 version - Latest release: 16 days ago - 140 downloads last month - 25,516 stars on GitHub - 1 maintainer
sillm-mlx 0.1.3
Running and training LLMs on Apple Silicon via MLX
4 versions - Latest release: 23 days ago - 563 downloads last month - 125 stars on GitHub - 1 maintainer
yunchang 0.1
Distributed Attention for Long Context LLM Model Training and Inference
1 version - Latest release: about 1 month ago - 13 downloads last month - 65 stars on GitHub - 1 maintainer
eagle-llm 1.2.1
Accelerating LLMs by 3x with No Quality Loss
3 versions - Latest release: about 1 month ago - 337 downloads last month - 509 stars on GitHub - 1 maintainer
py-txi 0.8.0
A Python wrapper around TGI and TEI servers
7 versions - Latest release: 7 days ago - 320 downloads last month - 25 stars on GitHub - 1 maintainer
google-jetstream 0.2.1
JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting ...
4 versions - Latest release: 13 days ago - 11.6 thousand downloads last month - 126 stars on GitHub - 1 maintainer
py-tgi 0.2.0
A Python wrapper around TGI and TEI servers
5 versions - Latest release: 3 months ago - 564 downloads last month - 18 stars on GitHub - 1 maintainer
horizon-takeoff 0.0.4.4
Auto-deploy the Takeoff Server on AWS for LLM inference
5 versions - Latest release: 4 months ago - 40 downloads last month - 0 stars on GitHub - 1 maintainer
kait 0.2.1
Kait - Kubernetes AI Tool
3 versions - Latest release: 4 months ago - 21 downloads last month - 25,467 stars on GitHub - 1 maintainer
pyautogenmagic 0.1.16
Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework
13 versions - Latest release: 6 months ago - 39 downloads last month - 18,528 stars on GitHub - 1 maintainer
autogenra 0.0.10a0
Autogen Assistant UI
10 versions - Latest release: 6 months ago - 7.83 thousand downloads last month - 18,528 stars on GitHub - 1 maintainer
anymodality 0.1.0
Make running Multi-Modal LLM easy.
1 version - Latest release: 7 months ago - 17 downloads last month - 2 stars on GitHub - 1 maintainer
llm-stack-kit 0.1.5
Generate Data from LLM easily using LLMKit
6 versions - Latest release: 7 months ago - 22 downloads last month - 3 stars on GitHub - 1 maintainer
talkingheads 0.5.0
A library to communicate with ChatGPT, Claude, Copilot, Gemini, HuggingChat, and Pi
7 versions - Latest release: about 2 months ago - 1 dependent repositories - 212 downloads last month - 207 stars on GitHub - 1 maintainer
torch-nos 0.2.0
Nitrous Oxide for your AI Infrastructure.
13 versions - Latest release: 4 months ago - 200 downloads last month - 115 stars on GitHub - 2 maintainers
medusa-llm 0.1
Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads
1 version - Latest release: 8 months ago - 59 downloads last month - 1,844 stars on GitHub - 1 maintainer
libre-chat 0.0.6
Free and Open Source Large Language Model (LLM) chatbot web UI and API. Self-hosted, offline capa...
2 versions - Latest release: 4 months ago - 68 downloads last month - 99 stars on GitHub - 1 maintainer
llama-trainer 0.2.1
Llama trainer utility
3 versions - Latest release: 8 months ago - 23 downloads last month - 5 stars on GitHub - 1 maintainer
Top 8.9% on pypi.org
llama2-wrapper 0.1.14
Use llama2-wrapper as your local llama2 backend for Generative Agents / Apps
14 versions - Latest release: 8 months ago - 1 dependent repositories - 610 downloads last month - 1,920 stars on GitHub - 1 maintainer
llm-inference 0.0.6 💰
Large Language Models Inference API and Applications
5 versions - Latest release: 10 months ago - 89 downloads last month - 120 stars on GitHub - 1 maintainer
Top 6.0% on pypi.org
lmdeploy 0.4.1
A toolset for compressing, deploying and serving LLM
28 versions - Latest release: 10 days ago - 1 dependent package - 2 dependent repositories - 3.9 thousand downloads last month - 2,405 stars on GitHub - 1 maintainer
llmdeploy 0.0.1
triton inference service of llama
1 version - Latest release: 11 months ago - 23 downloads last month - 2,405 stars on GitHub - 1 maintainer
local-llm-function-calling 0.1.23
A tool for generating function arguments and choosing what function to call with local LLMs
23 versions - Latest release: 2 months ago - 344 downloads last month - 277 stars on GitHub - 1 maintainer
llmflows 0.2.1
LLMFlows - Simple, Explicit and Transparent LLM Apps
15 versions - Latest release: 7 months ago - 1 dependent repositories - 101 downloads last month - 615 stars on GitHub - 1 maintainer
calculemus 0.0.2
Logical verification of probabilistic/language model 'intuitions'.
2 versions - Latest release: about 1 year ago - 31 downloads last month - 0 stars on GitHub - 1 maintainer
autonomi-nos 0.0.10
Nitrous oxide system (NOS) for computer-vision.
21 versions - Latest release: 8 months ago - 1 dependent repositories - 116 downloads last month - 115 stars on GitHub - 2 maintainers
cappr 0.9.0
Completion After Prompt Probability. Make your LLM make a choice
33 versions - Latest release: 4 months ago - 237 downloads last month - 64 stars on GitHub - 1 maintainer
deepsparse-ent 1.7.0
An inference runtime offering GPU-class performance on CPUs and APIs to integrate ML into your ap...
13 versions - Latest release: 2 months ago - 2 dependent packages - 280 downloads last month - 2,664 stars on GitHub - 1 maintainer
Top 7.8% on pypi.org
sagify 0.25.4
Machine Learning Training, Tuning and Deployment on AWS
55 versions - Latest release: 2 months ago - 2 dependent repositories - 164 downloads last month - 430 stars on GitHub - 1 maintainer
Top 3.5% on pypi.org
deepsparse 1.7.1
An inference runtime offering GPU-class performance on CPUs and APIs to integrate ML into your ap...
40 versions - Latest release: about 2 months ago - 3 dependent packages - 6 dependent repositories - 5.1 thousand downloads last month - 2,664 stars on GitHub - 1 maintainer
llmformat 0.0.2
Format LLM language by using LALR(1) grammar. Supports JSON, XML, etc.
5 versions - Latest release: 5 months ago - 31 downloads last month - 2 stars on GitHub - 1 maintainer
api4all 0.4.0
Easy-to-use LLM API from a state-of-the-art provider and comparison
12 versions - Latest release: 6 days ago - 125 downloads last month - 2 stars on GitHub - 1 maintainer
unifyai 0.8.3
A Python package for interacting with the Unify API
16 versions - Latest release: 6 days ago - 1.34 thousand downloads last month - 10 stars on GitHub - 1 maintainer
autogenbench 0.0.3
AutoGen Testbed Tools
20 versions - Latest release: about 2 months ago - 205 downloads last month - 18,528 stars on GitHub - 1 maintainer
periflow-client 0.2.2
Client of PeriFlow, the fastest generative AI serving available.
17 versions - Latest release: about 1 month ago - 151 downloads last month - 33 stars on GitHub - 1 maintainer
Top 6.7% on pypi.org
deepsparse-nightly 1.8.0.20240404
An inference runtime offering GPU-class performance on CPUs and APIs to integrate ML into your ap...
149 versions - Latest release: about 1 month ago - 3 dependent packages - 1 dependent repositories - 2.63 thousand downloads last month - 2,664 stars on GitHub - 1 maintainer
optimum-deepsparse 0.1.0.dev1
Optimum DeepSparse is an extension of the Hugging Face Transformers library that integrates the D...
2 versions - Latest release: 7 months ago - 3 dependent packages - 53 downloads last month - 2,664 stars on GitHub - 1 maintainer
periflow-sdk 0.1.2
PeriFlow SDK
4 versions - Latest release: about 1 year ago - 1 dependent repositories - 10 downloads last month - 18 stars on GitHub - 1 maintainer
openvino-arm 2022.1.0
OpenVINO(TM) Runtime
2 versions - Latest release: about 2 years ago - 1 dependent package - 2 dependent repositories - 44 downloads last month - 5,906 stars on GitHub - 1 maintainer