Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

pypi.org "llm-serving" keyword

sdap-os 0.1.2
Demo of usage of object storage for SDAP
3 versions - Latest release: almost 2 years ago - 1 dependent repositories - 13 downloads last month - 30,478 stars on GitHub - 1 maintainer
Top 2.2% on pypi.org
ray-cpp 2.21.0
A subpackage of Ray which provides the Ray C++ API.
52 versions - Latest release: 5 days ago - 2 dependent packages - 4 dependent repositories - 44.9 thousand downloads last month - 30,888 stars on GitHub - 22 maintainers
Top 0.2% on pypi.org
ray 2.21.0
Ray provides a simple, universal API for building distributed applications.
102 versions - Latest release: 5 days ago - 234 dependent packages - 3,641 dependent repositories - 4.58 million downloads last month - 30,888 stars on GitHub - 26 maintainers
myray 0.1.1
my ray desc
2 versions - Latest release: over 3 years ago - 1 dependent repositories - 8 downloads last month - 30,478 stars on GitHub - 1 maintainer
tilearn-infer 0.3.3
A high-throughput and memory-efficient inference and serving engine for LLMs
3 versions - Latest release: 22 days ago - 173 downloads last month - 17,633 stars on GitHub - 1 maintainer
friendli-client 1.3.4
Client of Friendli Suite.
13 versions - Latest release: about 1 month ago - 3.66 thousand downloads last month - 33 stars on GitHub - 1 maintainer
webapp-builder 1.0.0
Webapp Builder -- LLM/LVM Code Generator
2 versions - Latest release: 5 months ago - 7 downloads last month - 0 stars on GitHub - 1 maintainer
Top 3.5% on pypi.org
openllm-core 0.4.44
OpenLLM Core: Core components for OpenLLM.
73 versions - Latest release: 3 months ago - 2 dependent packages - 8 dependent repositories - 13 thousand downloads last month - 8,657 stars on GitHub - 1 maintainer
Top 3.9% on pypi.org
openllm-client 0.4.44
OpenLLM Client: Interacting with OpenLLM HTTP/gRPC server, or any BentoML server.
73 versions - Latest release: 3 months ago - 1 dependent package - 8 dependent repositories - 12.8 thousand downloads last month - 8,657 stars on GitHub - 1 maintainer
tilearn-test01 0.1
A high-throughput and memory-efficient inference and serving engine for LLMs
1 version - Latest release: about 1 month ago - 13 downloads last month - 17,633 stars on GitHub - 1 maintainer
skypilot-nightly 1.0.0.dev20240513
SkyPilot: An intercloud broker for the clouds
305 versions - Latest release: about 10 hours ago - 15 thousand downloads last month - 5,536 stars on GitHub - 3 maintainers
Top 7.4% on pypi.org
mosec 0.8.4
Model Serving made Efficient in the Cloud
41 versions - Latest release: 3 months ago - 1 dependent package - 1 dependent repositories - 2.62 thousand downloads last month - 657 stars on GitHub - 2 maintainers
vllm-xft 0.3.3.1
A high-throughput and memory-efficient inference and serving engine for LLMs
2 versions - Latest release: 4 days ago - 104 downloads last month - 17,633 stars on GitHub - 2 maintainers
nextai-vllm 0.0.7
A high-throughput and memory-efficient inference and serving engine for LLMs
6 versions - Latest release: 19 days ago - 131 downloads last month - 17,633 stars on GitHub - 2 maintainers
hive-vllm 0.0.1
a
1 version - Latest release: 3 months ago - 7 downloads last month - 17,633 stars on GitHub - 1 maintainer
llm_atc 0.1.7
Tools for fine tuning and serving LLMs
6 versions - Latest release: 5 months ago - 39 downloads last month - 17,633 stars on GitHub - 2 maintainers
superlaser 0.0.6
An MLOps library for LLM deployment w/ the vLLM engine on RunPod's infra.
6 versions - Latest release: 2 months ago - 60 downloads last month - 17,633 stars on GitHub - 2 maintainers
vllm-consul 0.2.1
A high-throughput and memory-efficient inference and serving engine for LLMs
5 versions - Latest release: 7 months ago - 39 downloads last month - 12,665 stars on GitHub - 2 maintainers
vllm-acc 0.4.1
A high-throughput and memory-efficient inference and serving engine for LLMs
7 versions - Latest release: 10 days ago - 1.17 thousand downloads last month - 17,633 stars on GitHub - 2 maintainers
Top 3.4% on pypi.org
vllm 0.4.2
A high-throughput and memory-efficient inference and serving engine for LLMs
26 versions - Latest release: 9 days ago - 7 dependent packages - 5 dependent repositories - 497 thousand downloads last month - 17,633 stars on GitHub - 4 maintainers
llm-swarm 0.1.1
A high-throughput and memory-efficient inference and serving engine for LLMs
2 versions - Latest release: 2 months ago - 75 downloads last month - 17,633 stars on GitHub - 2 maintainers
vllm-online 0.4.2
A high-throughput and memory-efficient inference and serving engine for LLMs
2 versions - Latest release: 15 days ago - 263 downloads last month - 17,633 stars on GitHub - 2 maintainers
Top 2.2% on pypi.org
openllm 0.4.44
OpenLLM: Operating LLMs in production
152 versions - Latest release: 3 months ago - 2 dependent packages - 295 dependent repositories - 15.9 thousand downloads last month - 8,657 stars on GitHub - 4 maintainers
Top 4.1% on pypi.org
skypilot 0.5.0
SkyPilot: An intercloud broker for the clouds
34 versions - Latest release: 3 months ago - 2 dependent packages - 3 dependent repositories - 65.7 thousand downloads last month - 5,536 stars on GitHub - 3 maintainers
happy-vllm 1.1.2
happy_vllm is a REST API for vLLM, production ready
5 versions - Latest release: 3 days ago - 251 downloads last month - 18 stars on GitHub - 2 maintainers
nextai-star 0.2.33
An open platform for training, serving, and evaluating large language model based chatbots by nex...
9 versions - Latest release: 5 months ago - 57 downloads last month - 5,536 stars on GitHub - 2 maintainers
vllm-cpm 0.2.2
A high-throughput and memory-efficient inference and serving engine for LLMs
1 version - Latest release: 3 months ago - 7 downloads last month - 5,536 stars on GitHub - 2 maintainers
nextai-prism 1.0.20
Prism: An intercloud broker for the clouds
21 versions - Latest release: 3 months ago - 156 downloads last month - 5,536 stars on GitHub - 1 maintainer
periflow-client 0.2.2
Client of PeriFlow, the fastest generative AI serving available.
17 versions - Latest release: about 1 month ago - 151 downloads last month - 33 stars on GitHub - 1 maintainer
horizon-takeoff 0.0.4.4
Auto-deploy the Takeoff Server on AWS for LLM inference
5 versions - Latest release: 4 months ago - 40 downloads last month - 0 stars on GitHub - 2 maintainers
lorax-client 0.5.0
LoRAX Python Client
8 versions - Latest release: 19 days ago - 2.06 thousand downloads last month - 516 stars on GitHub - 2 maintainers
periflow-sdk 0.1.2
PeriFlow SDK
4 versions - Latest release: about 1 year ago - 1 dependent repositories - 10 downloads last month - 18 stars on GitHub - 1 maintainer
Top 6.7% on pypi.org
secretflow-ray 2.2.0
Ray provides a simple, universal API for building distributed applications.
3 versions - Latest release: over 1 year ago - 2 dependent packages - 1 dependent repositories - 657 downloads last month - 30,478 stars on GitHub - 1 maintainer
mosec-tiinfer 0.0.7
Model Serving made Efficient in the Cloud.
1 version - Latest release: about 1 year ago - 41 downloads last month - 705 stars on GitHub - 2 maintainers
superduperdb 0.1.1
🔮 Super-power your database with AI 🔮
24 versions - Latest release: 3 months ago - 1 dependent repositories - 402 downloads last month - 4,305 stars on GitHub - 2 maintainers
fangyu-pypitest 0.8.0.dev4
A system for parallel and distributed Python that unifies the ML ecosystem.
1 version - Latest release: over 4 years ago - 1 dependent repositories - 16 downloads last month - 30,478 stars on GitHub - 1 maintainer
ray-for-mars 1.12.1
Ray provides a simple, universal API for building distributed applications.
2 versions - Latest release: almost 2 years ago - 1 dependent repositories - 22 downloads last month - 30,478 stars on GitHub - 1 maintainer