Ecosyste.ms: Packages
An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.
pypi.org "llm-inference" keyword
unifyai 0.8.3
A Python package for interacting with the Unify API16 versions - Latest release: 23 days ago - 1.28 thousand downloads last month - 10 stars on GitHub - 1 maintainer
Top 1.2% on pypi.org
147 versions - Latest release: 17 days ago - 13 dependent packages - 499 dependent repositories - 99.8 thousand downloads last month - 6,591 stars on GitHub - 3 maintainers
bentoml 1.2.16
BentoML: Build Production-Grade AI Applications147 versions - Latest release: 17 days ago - 13 dependent packages - 499 dependent repositories - 99.8 thousand downloads last month - 6,591 stars on GitHub - 3 maintainers
periflow-sdk 0.1.2
PeriFlow SDK4 versions - Latest release: over 1 year ago - 1 dependent repositories - 10 downloads last month - 18 stars on GitHub - 1 maintainer
autonomi-nos 0.0.10
Nitrous oxide system (NOS) for computer-vision.21 versions - Latest release: 9 months ago - 1 dependent repositories - 55 downloads last month - 118 stars on GitHub - 2 maintainers
torch-nos 0.2.0
Nitrous Oxide for your AI Infrastructure.14 versions - Latest release: 4 months ago - 121 downloads last month - 118 stars on GitHub - 2 maintainers
Top 2.2% on pypi.org
161 versions - Latest release: 4 months ago - 3 dependent packages - 295 dependent repositories - 16.6 thousand downloads last month - 8,657 stars on GitHub - 2 maintainers
openllm 0.4.44
OpenLLM: Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint in...161 versions - Latest release: 4 months ago - 3 dependent packages - 295 dependent repositories - 16.6 thousand downloads last month - 8,657 stars on GitHub - 2 maintainers
Top 3.5% on pypi.org
82 versions - Latest release: 4 months ago - 2 dependent packages - 8 dependent repositories - 13.3 thousand downloads last month - 8,657 stars on GitHub - 1 maintainer
openllm-core 0.4.44
OpenLLM Core: Core components for OpenLLM.82 versions - Latest release: 4 months ago - 2 dependent packages - 8 dependent repositories - 13.3 thousand downloads last month - 8,657 stars on GitHub - 1 maintainer
Top 3.9% on pypi.org
82 versions - Latest release: 4 months ago - 2 dependent packages - 8 dependent repositories - 12.6 thousand downloads last month - 8,657 stars on GitHub - 1 maintainer
openllm-client 0.4.44
OpenLLM Client: Interacting with OpenLLM HTTP/gRPC server, or any BentoML server.82 versions - Latest release: 4 months ago - 2 dependent packages - 8 dependent repositories - 12.6 thousand downloads last month - 8,657 stars on GitHub - 1 maintainer
google-jetstream 0.2.1
JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting ...5 versions - Latest release: 30 days ago - 11.6 thousand downloads last month - 126 stars on GitHub - 1 maintainer
openvino-nightly 2024.2.0.dev20240515
OpenVINO(TM) Runtime128 versions - Latest release: 17 days ago - 1 dependent repositories - 38.8 thousand downloads last month - 6,109 stars on GitHub - 1 maintainer
openvino-arm 2022.1.0
OpenVINO(TM) Runtime2 versions - Latest release: about 2 years ago - 1 dependent package - 2 dependent repositories - 19 downloads last month - 6,109 stars on GitHub - 1 maintainer
Top 0.7% on pypi.org
32 versions - Latest release: about 1 month ago - 38 dependent packages - 498 dependent repositories - 191 thousand downloads last month - 6,109 stars on GitHub - 1 maintainer
openvino-dev 2024.1.0
OpenVINO(TM) Development Tools32 versions - Latest release: about 1 month ago - 38 dependent packages - 498 dependent repositories - 191 thousand downloads last month - 6,109 stars on GitHub - 1 maintainer
Top 0.7% on pypi.org
33 versions - Latest release: about 1 month ago - 47 dependent packages - 342 dependent repositories - 364 thousand downloads last month - 6,109 stars on GitHub - 1 maintainer
openvino 2024.1.0
OpenVINO(TM) Runtime33 versions - Latest release: about 1 month ago - 47 dependent packages - 342 dependent repositories - 364 thousand downloads last month - 6,109 stars on GitHub - 1 maintainer
Top 1.4% on pypi.org
50 versions - Latest release: about 1 month ago - 46 dependent packages - 168 dependent repositories - 74.3 thousand downloads last month - 63,563 stars on GitHub - 1 maintainer
gpt4all 2.6.0
Python bindings for GPT4All50 versions - Latest release: about 1 month ago - 46 dependent packages - 168 dependent repositories - 74.3 thousand downloads last month - 63,563 stars on GitHub - 1 maintainer
Top 2.4% on pypi.org
52 versions - Latest release: about 1 month ago - 37 dependent packages - 3 dependent repositories - 172 thousand downloads last month - 18,528 stars on GitHub - 2 maintainers
pyautogen 0.2.27
Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework52 versions - Latest release: about 1 month ago - 37 dependent packages - 3 dependent repositories - 172 thousand downloads last month - 18,528 stars on GitHub - 2 maintainers
cappr 0.9.0
Completion After Prompt Probability. Make your LLM make a choice33 versions - Latest release: 4 months ago - 162 downloads last month - 64 stars on GitHub - 1 maintainer
llmformat 0.0.2
Format LLM language by using LALR(1) grammar. Supports JSON, XML, etc.5 versions - Latest release: 5 months ago - 26 downloads last month - 3 stars on GitHub - 1 maintainer
talkingheads 0.5.0
A library to communicate with ChatGPT, Claude, Copilot, Gemini, HuggingChat, and Pi7 versions - Latest release: 3 months ago - 1 dependent repositories - 108 downloads last month - 236 stars on GitHub - 1 maintainer
autogenstudio 0.0.56
AutoGen Studio60 versions - Latest release: 2 months ago - 3 dependent packages - 17 thousand downloads last month - 18,528 stars on GitHub - 1 maintainer
anymodality 0.1.0
Make running Multi-Modal LLM easy.1 version - Latest release: 7 months ago - 13 downloads last month - 2 stars on GitHub - 1 maintainer
edsl 0.1.22
Create and analyze LLM-based surveys29 versions - Latest release: 19 days ago - 1.03 thousand downloads last month - 20 stars on GitHub - 3 maintainers
autogencap-rajan.jedi 0.0.6
CAP w/ autogen bindings5 versions - Latest release: about 1 month ago - 277 downloads last month - 25,963 stars on GitHub - 1 maintainer
friendli-client 1.3.4
Client of Friendli Suite.14 versions - Latest release: 2 months ago - 4 dependent packages - 3.83 thousand downloads last month - 33 stars on GitHub - 1 maintainer
Top 6.0% on pypi.org
29 versions - Latest release: 26 days ago - 1 dependent package - 2 dependent repositories - 4.88 thousand downloads last month - 2,405 stars on GitHub - 1 maintainer
lmdeploy 0.4.1
A toolset for compressing, deploying and serving LLM29 versions - Latest release: 26 days ago - 1 dependent package - 2 dependent repositories - 4.88 thousand downloads last month - 2,405 stars on GitHub - 1 maintainer
mistral-inference 0.0.0
Official inference library for Mistral models3 versions - Latest release: 19 days ago - 943 downloads last month - 8,771 stars on GitHub - 1 maintainer
arcee-py 1.0.7
The Arcee client for executing domain-adpated language model routines23 versions - Latest release: 18 days ago - 747 downloads last month - 14 stars on GitHub - 2 maintainers
Top 6.4% on pypi.org
16 versions - Latest release: about 1 month ago - 1 dependent package - 4 dependent repositories - 4.72 thousand downloads last month - 1,909 stars on GitHub - 1 maintainer
intel-extension-for-transformers 1.4.1
Repository of Intel® Intel Extension for Transformers16 versions - Latest release: about 1 month ago - 1 dependent package - 4 dependent repositories - 4.72 thousand downloads last month - 1,909 stars on GitHub - 1 maintainer
lorax-client 0.5.1
LoRAX Python Client10 versions - Latest release: 17 days ago - 8.2 thousand downloads last month - 516 stars on GitHub - 1 maintainer
Top 8.9% on pypi.org
14 versions - Latest release: 8 months ago - 1 dependent repositories - 361 downloads last month - 1,934 stars on GitHub - 1 maintainer
llama2-wrapper 0.1.14
Use llama2-wrapper as your local llama2 backend for Generative Agents / Apps14 versions - Latest release: 8 months ago - 1 dependent repositories - 361 downloads last month - 1,934 stars on GitHub - 1 maintainer
llm-stack-kit 0.1.5
Generate Data from LLM easily using LLMKit6 versions - Latest release: 8 months ago - 52 downloads last month - 3 stars on GitHub - 1 maintainer
scalellm 0.0.9
A high-performance inference system for large language models.3 versions - Latest release: 16 days ago - 536 downloads last month - 289 stars on GitHub - 1 maintainer
llmflows 0.2.1
LLMFlows - Simple, Explicit and Transparent LLM Apps15 versions - Latest release: 8 months ago - 1 dependent repositories - 179 downloads last month - 615 stars on GitHub - 1 maintainer
medusa-llm 0.1
Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads1 version - Latest release: 9 months ago - 52 downloads last month - 1,844 stars on GitHub - 1 maintainer
sillm-mlx 0.1.3
Running and training LLMs on Apple Silicon via MLX4 versions - Latest release: about 1 month ago - 420 downloads last month - 146 stars on GitHub - 1 maintainer
kait 0.2.1
Kait - Kubernetes AI Tool3 versions - Latest release: 5 months ago - 26 downloads last month - 25,963 stars on GitHub - 1 maintainer
pyautogenmagic 0.1.16
Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework13 versions - Latest release: 7 months ago - 39 downloads last month - 18,528 stars on GitHub - 1 maintainer
autogenbench 0.0.3
AutoGen Testbed Tools20 versions - Latest release: 2 months ago - 168 downloads last month - 18,528 stars on GitHub - 1 maintainer
bisheng-pyautogen 0.3.0
Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework10 versions - Latest release: about 2 months ago - 2 dependent packages - 266 downloads last month - 18,528 stars on GitHub - 1 maintainer
autogenra 0.0.10a0
Autogen Assistant UI10 versions - Latest release: 6 months ago - 7.49 thousand downloads last month - 18,528 stars on GitHub - 1 maintainer
libre-chat 0.0.6
Free and Open Source Large Language Model (LLM) chatbot web UI and API. Self-hosted, offline capa...2 versions - Latest release: 5 months ago - 69 downloads last month - 99 stars on GitHub - 1 maintainer
Top 10.0% on pypi.org
13 versions - Latest release: 17 days ago - 1 dependent package - 1 dependent repositories - 1.27 thousand downloads last month - 412 stars on GitHub - 1 maintainer
genv 1.4.3
GPU environment and cluster management13 versions - Latest release: 17 days ago - 1 dependent package - 1 dependent repositories - 1.27 thousand downloads last month - 412 stars on GitHub - 1 maintainer
git-llm 0.1.3
The project integrates Git with a llm (OpenAI, LlamaCpp, and GPT-4-All) to extend the capabilitie...3 versions - Latest release: about 1 year ago - 45 downloads last month - 63,563 stars on GitHub - 1 maintainer
superduperdb 0.1.1
🔮 Super-power your database with AI 🔮24 versions - Latest release: 4 months ago - 1 dependent repositories - 448 downloads last month - 4,305 stars on GitHub - 1 maintainer
llm-hygiene 0.0.1
a data preprocessing toolkit that makes it easy to create common LLM-related data structures; fro...1 version - Latest release: 6 months ago - 13 downloads last month - 1 stars on GitHub - 1 maintainer
neural-speed 1.0
Repository of Intel® Intel Extension for Transformers5 versions - Latest release: 2 months ago - 2.73 thousand downloads last month - 179 stars on GitHub - 1 maintainer
sibila 0.4.3
Structured queries from local or online LLM models13 versions - Latest release: 20 days ago - 198 downloads last month - 14 stars on GitHub - 1 maintainer
litgpt 0.3.1
Hackable implementation of state-of-the-art open-source LLMs6 versions - Latest release: 25 days ago - 1 dependent package - 8.9 thousand downloads last month - 6,784 stars on GitHub - 3 maintainers
exxa 0.6.4 💰
Exa - Pytorch58 versions - Latest release: about 2 months ago - 2 dependent packages - 1 dependent repositories - 617 downloads last month - 19 stars on GitHub - 1 maintainer
llm-vm 0.1.55
An Open-Source AGI Server for Open-Source LLMs7 versions - Latest release: 6 months ago - 58 downloads last month - 455 stars on GitHub - 3 maintainers
yunchang 0.1
Distributed Attention for Long Context LLM Model Training and Inference1 version - Latest release: about 2 months ago - 13 downloads last month - 65 stars on GitHub - 1 maintainer
eagle-llm 1.2.1
Accelerating LLMs by 3x with No Quality Loss3 versions - Latest release: about 2 months ago - 337 downloads last month - 509 stars on GitHub - 1 maintainer
py-txi 0.8.0
A Python wrapper around TGI and TEI servers7 versions - Latest release: 23 days ago - 320 downloads last month - 25 stars on GitHub - 1 maintainer
py-tgi 0.2.0
A Python wrapper around TGI and TEI servers5 versions - Latest release: 3 months ago - 564 downloads last month - 18 stars on GitHub - 1 maintainer
horizon-takeoff 0.0.4.4
Auto-deploy the Takeoff Server on AWS for LLM inference5 versions - Latest release: 5 months ago - 40 downloads last month - 0 stars on GitHub - 1 maintainer
llama-trainer 0.2.1
Llama trainer utility3 versions - Latest release: 8 months ago - 23 downloads last month - 5 stars on GitHub - 1 maintainer
llm-inference 0.0.6 💰
Large Language Models Inference API and Applications5 versions - Latest release: 11 months ago - 89 downloads last month - 120 stars on GitHub - 1 maintainer
llmdeploy 0.0.1
triton inference service of llama1 version - Latest release: 11 months ago - 23 downloads last month - 2,405 stars on GitHub - 1 maintainer
local-llm-function-calling 0.1.23
A tool for generating function arguments and choosing what function to call with local LLMs23 versions - Latest release: 3 months ago - 344 downloads last month - 277 stars on GitHub - 1 maintainer
calculemus 0.0.2
Logical verification of probabilistic/language model 'intuitions'.2 versions - Latest release: about 1 year ago - 31 downloads last month - 0 stars on GitHub - 1 maintainer
deepsparse-ent 1.7.0
An inference runtime offering GPU-class performance on CPUs and APIs to integrate ML into your ap...13 versions - Latest release: 3 months ago - 2 dependent packages - 280 downloads last month - 2,664 stars on GitHub - 1 maintainer
Top 7.8% on pypi.org
55 versions - Latest release: 3 months ago - 2 dependent repositories - 164 downloads last month - 430 stars on GitHub - 1 maintainer
sagify 0.25.4
Machine Learning Training, Tuning and Deployment on AWS55 versions - Latest release: 3 months ago - 2 dependent repositories - 164 downloads last month - 430 stars on GitHub - 1 maintainer
Top 3.5% on pypi.org
40 versions - Latest release: 2 months ago - 3 dependent packages - 6 dependent repositories - 5.1 thousand downloads last month - 2,664 stars on GitHub - 1 maintainer
deepsparse 1.7.1
An inference runtime offering GPU-class performance on CPUs and APIs to integrate ML into your ap...40 versions - Latest release: 2 months ago - 3 dependent packages - 6 dependent repositories - 5.1 thousand downloads last month - 2,664 stars on GitHub - 1 maintainer
api4all 0.4.0
Easy-to-use LLM API from a state-of-the-art provider and comparison12 versions - Latest release: 23 days ago - 125 downloads last month - 2 stars on GitHub - 1 maintainer
periflow-client 0.2.2
Client of PeriFlow, the fastest generative AI serving available.17 versions - Latest release: about 2 months ago - 151 downloads last month - 33 stars on GitHub - 1 maintainer
Top 6.7% on pypi.org
149 versions - Latest release: about 2 months ago - 3 dependent packages - 1 dependent repositories - 2.63 thousand downloads last month - 2,664 stars on GitHub - 1 maintainer
deepsparse-nightly 1.8.0.20240404
An inference runtime offering GPU-class performance on CPUs and APIs to integrate ML into your ap...149 versions - Latest release: about 2 months ago - 3 dependent packages - 1 dependent repositories - 2.63 thousand downloads last month - 2,664 stars on GitHub - 1 maintainer
optimum-deepsparse 0.1.0.dev1
Optimum DeepSparse is an extension of the Hugging Face Transformers library that integrates the D...2 versions - Latest release: 7 months ago - 3 dependent packages - 53 downloads last month - 2,664 stars on GitHub - 1 maintainer
Related Keywords
llm
33
llmops
22
inference
16
ai
15
gpt
15
llama2
14
chatgpt
12
chatbot
12
generative-ai
11
gpt-4
11
computer-vision
10
llama
10
llm-serving
10
deep-learning
10
llm-agent
10
python
10
llms
9
llm-framework
9
chat
8
chat-application
8
gpt-35-turbo
8
agent-oriented-programming
8
agent-based-framework
8
machine-learning
8
transformers
8
mlops
8
nlp
8
large-language-models
7
openai
7
stable-diffusion
6
cpu
6
ml
6
mistral
6
AI
5
huggingface
5
MLOps
5
pretrained-models
5
LLM
5
pytorch
5
gpu
5
llm-ops
5
llm-training
5
quantization
5
performance
5
sparsity
5
yolo
4
onnx
4
pruning
4
Falcon
4
sparsification
4
inference-engine
4
deploy-ai
4
diffusion-models
4
good-first-issue
4
natural-language-processing
4
artificial-intelligence
4
openvino
4
optimize-ai
4
performance-boost
4
recommendation-system
4
speech-recognition
4
Transformers
4
BentoML
4
mpt
4
serving
4
Model Deployment
4
fine-tuning
4
deepsparse
4
Model Serving
4
Large Language Model
4
x86
4
llamacpp
4
open-source-llm
4
cpus
4
machinelearning
4
object-detection
4
prompt-engineering
3
langchain
3
docker
3
machine learning
3
object detection
3
computer vision
3
x86_64
3
runtime
3
avx2
3
avx512
3
inference engine
3
sparse
3
neural network
3
PyTorch
3
Llama 2
3
LLMOps
3
transformer
3
Generative AI
3
model-serving
3
Fine tuning
3
Serverless
3
StableLM
3
Alpaca
3
vicuna
3