Ecosyste.ms: Packages
An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.
pypi.org "llm-serving" keyword
sdap-os 0.1.2
Demo of usage of object storage for SDAP3 versions - Latest release: almost 2 years ago - 1 dependent repositories - 13 downloads last month - 30,478 stars on GitHub - 1 maintainer
Top 2.2% on pypi.org
52 versions - Latest release: 5 days ago - 2 dependent packages - 4 dependent repositories - 44.9 thousand downloads last month - 30,888 stars on GitHub - 22 maintainers
ray-cpp 2.21.0
A subpackage of Ray which provides the Ray C++ API.52 versions - Latest release: 5 days ago - 2 dependent packages - 4 dependent repositories - 44.9 thousand downloads last month - 30,888 stars on GitHub - 22 maintainers
Top 0.2% on pypi.org
102 versions - Latest release: 5 days ago - 234 dependent packages - 3,641 dependent repositories - 4.58 million downloads last month - 30,888 stars on GitHub - 26 maintainers
ray 2.21.0
Ray provides a simple, universal API for building distributed applications.102 versions - Latest release: 5 days ago - 234 dependent packages - 3,641 dependent repositories - 4.58 million downloads last month - 30,888 stars on GitHub - 26 maintainers
myray 0.1.1
my ray desc2 versions - Latest release: over 3 years ago - 1 dependent repositories - 8 downloads last month - 30,478 stars on GitHub - 1 maintainer
tilearn-infer 0.3.3
A high-throughput and memory-efficient inference and serving engine for LLMs3 versions - Latest release: 22 days ago - 173 downloads last month - 17,633 stars on GitHub - 1 maintainer
friendli-client 1.3.4
Client of Friendli Suite.13 versions - Latest release: about 1 month ago - 3.66 thousand downloads last month - 33 stars on GitHub - 1 maintainer
webapp-builder 1.0.0
Webapp Builder -- LLM/LVM Code Generator2 versions - Latest release: 5 months ago - 7 downloads last month - 0 stars on GitHub - 1 maintainer
Top 3.5% on pypi.org
73 versions - Latest release: 3 months ago - 2 dependent packages - 8 dependent repositories - 13 thousand downloads last month - 8,657 stars on GitHub - 1 maintainer
openllm-core 0.4.44
OpenLLM Core: Core components for OpenLLM.73 versions - Latest release: 3 months ago - 2 dependent packages - 8 dependent repositories - 13 thousand downloads last month - 8,657 stars on GitHub - 1 maintainer
Top 3.9% on pypi.org
73 versions - Latest release: 3 months ago - 1 dependent package - 8 dependent repositories - 12.8 thousand downloads last month - 8,657 stars on GitHub - 1 maintainer
openllm-client 0.4.44
OpenLLM Client: Interacting with OpenLLM HTTP/gRPC server, or any BentoML server.73 versions - Latest release: 3 months ago - 1 dependent package - 8 dependent repositories - 12.8 thousand downloads last month - 8,657 stars on GitHub - 1 maintainer
tilearn-test01 0.1
A high-throughput and memory-efficient inference and serving engine for LLMs1 version - Latest release: about 1 month ago - 13 downloads last month - 17,633 stars on GitHub - 1 maintainer
skypilot-nightly 1.0.0.dev20240513
SkyPilot: An intercloud broker for the clouds305 versions - Latest release: about 10 hours ago - 15 thousand downloads last month - 5,536 stars on GitHub - 3 maintainers
Top 7.4% on pypi.org
41 versions - Latest release: 3 months ago - 1 dependent package - 1 dependent repositories - 2.62 thousand downloads last month - 657 stars on GitHub - 2 maintainers
mosec 0.8.4
Model Serving made Efficient in the Cloud41 versions - Latest release: 3 months ago - 1 dependent package - 1 dependent repositories - 2.62 thousand downloads last month - 657 stars on GitHub - 2 maintainers
vllm-xft 0.3.3.1
A high-throughput and memory-efficient inference and serving engine for LLMs2 versions - Latest release: 4 days ago - 104 downloads last month - 17,633 stars on GitHub - 2 maintainers
nextai-vllm 0.0.7
A high-throughput and memory-efficient inference and serving engine for LLMs6 versions - Latest release: 19 days ago - 131 downloads last month - 17,633 stars on GitHub - 2 maintainers
hive-vllm 0.0.1
a1 version - Latest release: 3 months ago - 7 downloads last month - 17,633 stars on GitHub - 1 maintainer
llm_atc 0.1.7
Tools for fine tuning and serving LLMs6 versions - Latest release: 5 months ago - 39 downloads last month - 17,633 stars on GitHub - 2 maintainers
superlaser 0.0.6
An MLOps library for LLM deployment w/ the vLLM engine on RunPod's infra.6 versions - Latest release: 2 months ago - 60 downloads last month - 17,633 stars on GitHub - 2 maintainers
vllm-consul 0.2.1
A high-throughput and memory-efficient inference and serving engine for LLMs5 versions - Latest release: 7 months ago - 39 downloads last month - 12,665 stars on GitHub - 2 maintainers
vllm-acc 0.4.1
A high-throughput and memory-efficient inference and serving engine for LLMs7 versions - Latest release: 10 days ago - 1.17 thousand downloads last month - 17,633 stars on GitHub - 2 maintainers
Top 3.4% on pypi.org
26 versions - Latest release: 9 days ago - 7 dependent packages - 5 dependent repositories - 497 thousand downloads last month - 17,633 stars on GitHub - 4 maintainers
vllm 0.4.2
A high-throughput and memory-efficient inference and serving engine for LLMs26 versions - Latest release: 9 days ago - 7 dependent packages - 5 dependent repositories - 497 thousand downloads last month - 17,633 stars on GitHub - 4 maintainers
llm-swarm 0.1.1
A high-throughput and memory-efficient inference and serving engine for LLMs2 versions - Latest release: 2 months ago - 75 downloads last month - 17,633 stars on GitHub - 2 maintainers
vllm-online 0.4.2
A high-throughput and memory-efficient inference and serving engine for LLMs2 versions - Latest release: 15 days ago - 263 downloads last month - 17,633 stars on GitHub - 2 maintainers
Top 2.2% on pypi.org
152 versions - Latest release: 3 months ago - 2 dependent packages - 295 dependent repositories - 15.9 thousand downloads last month - 8,657 stars on GitHub - 4 maintainers
openllm 0.4.44
OpenLLM: Operating LLMs in production152 versions - Latest release: 3 months ago - 2 dependent packages - 295 dependent repositories - 15.9 thousand downloads last month - 8,657 stars on GitHub - 4 maintainers
Top 4.1% on pypi.org
34 versions - Latest release: 3 months ago - 2 dependent packages - 3 dependent repositories - 65.7 thousand downloads last month - 5,536 stars on GitHub - 3 maintainers
skypilot 0.5.0
SkyPilot: An intercloud broker for the clouds34 versions - Latest release: 3 months ago - 2 dependent packages - 3 dependent repositories - 65.7 thousand downloads last month - 5,536 stars on GitHub - 3 maintainers
happy-vllm 1.1.2
happy_vllm is a REST API for vLLM, production ready5 versions - Latest release: 3 days ago - 251 downloads last month - 18 stars on GitHub - 2 maintainers
nextai-star 0.2.33
An open platform for training, serving, and evaluating large language model based chatbots by nex...9 versions - Latest release: 5 months ago - 57 downloads last month - 5,536 stars on GitHub - 2 maintainers
vllm-cpm 0.2.2
A high-throughput and memory-efficient inference and serving engine for LLMs1 version - Latest release: 3 months ago - 7 downloads last month - 5,536 stars on GitHub - 2 maintainers
nextai-prism 1.0.20
Prism: An intercloud broker for the clouds21 versions - Latest release: 3 months ago - 156 downloads last month - 5,536 stars on GitHub - 1 maintainer
periflow-client 0.2.2
Client of PeriFlow, the fastest generative AI serving available.17 versions - Latest release: about 1 month ago - 151 downloads last month - 33 stars on GitHub - 1 maintainer
horizon-takeoff 0.0.4.4
Auto-deploy the Takeoff Server on AWS for LLM inference5 versions - Latest release: 4 months ago - 40 downloads last month - 0 stars on GitHub - 2 maintainers
lorax-client 0.5.0
LoRAX Python Client8 versions - Latest release: 19 days ago - 2.06 thousand downloads last month - 516 stars on GitHub - 2 maintainers
periflow-sdk 0.1.2
PeriFlow SDK4 versions - Latest release: about 1 year ago - 1 dependent repositories - 10 downloads last month - 18 stars on GitHub - 1 maintainer
Top 6.7% on pypi.org
3 versions - Latest release: over 1 year ago - 2 dependent packages - 1 dependent repositories - 657 downloads last month - 30,478 stars on GitHub - 1 maintainer
secretflow-ray 2.2.0
Ray provides a simple, universal API for building distributed applications.3 versions - Latest release: over 1 year ago - 2 dependent packages - 1 dependent repositories - 657 downloads last month - 30,478 stars on GitHub - 1 maintainer
mosec-tiinfer 0.0.7
Model Serving made Efficient in the Cloud.1 version - Latest release: about 1 year ago - 41 downloads last month - 705 stars on GitHub - 2 maintainers
superduperdb 0.1.1
🔮 Super-power your database with AI 🔮24 versions - Latest release: 3 months ago - 1 dependent repositories - 402 downloads last month - 4,305 stars on GitHub - 2 maintainers
fangyu-pypitest 0.8.0.dev4
A system for parallel and distributed Python that unifies the ML ecosystem.1 version - Latest release: over 4 years ago - 1 dependent repositories - 16 downloads last month - 30,478 stars on GitHub - 1 maintainer
ray-for-mars 1.12.1
Ray provides a simple, universal API for building distributed applications.2 versions - Latest release: almost 2 years ago - 1 dependent repositories - 22 downloads last month - 30,478 stars on GitHub - 1 maintainer
Related Keywords
pytorch
23
llm
22
mlops
21
llmops
20
inference
16
machine-learning
16
llama
16
deep-learning
15
gpt
15
model-serving
15
data-science
13
transformer
12
amd
11
cuda
11
serving
11
inferentia
11
rocm
11
trainium
11
python
10
deployment
9
llm-inference
9
tensorflow
9
ai
7
gpu
7
rllib
7
reinforcement-learning
7
ray
7
parallel
7
optimization
7
model-selection
7
java
7
hyperparameter-search
7
hyperparameter-optimization
7
distributed
7
automl
7
llama2
6
ml
6
finops
5
hyperparameter-tuning
5
mistral
5
distributed-training
5
cost-optimization
5
cost-management
5
cloud-management
5
cloud-computing
5
MLOps
5
job-queue
5
llm-ops
5
job-scheduler
5
llm-training
5
ml-infrastructure
5
ml-platform
5
multicloud
5
spot-instances
5
tpu
5
hyperparameter-tuningreinforcement-learning
4
AI
4
fine-tuning
4
mpt
4
bentoml
3
falcon
3
Vicuna
3
Transformers
3
StableLM
3
llms
3
model-inference
3
open-source-llm
3
openllm
3
stablelm
3
vicuna
3
transformers
3
inference-server
3
inference-engine
3
generative-ai
3
Alpaca
3
BentoML
3
Falcon
3
Fine tuning
3
Generative AI
3
LLMOps
3
Large Language Model
3
Llama 2
3
Model Deployment
3
Model Serving
3
PyTorch
3
Serverless
3
LLM
2
server
2
tts
2
rust
2
nerual-network
2
mxnet
2
cv
2
hacktoberfest
2
jax
2
machine-learning-platform
2
gpt3
2
stable-diffusion
2
NLP
2
vllm
2