Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

pypi.org : tilearn-infer

A high-throughput and memory-efficient inference and serving engine for LLMs

Registry - Source - Documentation - JSON
purl: pkg:pypi/tilearn-infer
Keywords: amd, cuda, gpt, inference, inferentia, llama, llm, llm-serving, llmops, mlops, model-serving, pytorch, rocm, trainium, transformer
License: Apache-2.0
Latest release: about 2 months ago
First release: about 2 months ago
Downloads: 26 last month
Stars: 17,633 on GitHub
Forks: 2,286 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: 3 days ago

byzerllm 0.1.89
ByzerLLM: Byzer LLM
94 versions - Latest release: 22 days ago - 1 dependent package - 2 dependent repositories - 2.66 thousand downloads last month - 17,633 stars on GitHub - 1 maintainer
vllm-xft 0.3.3.1
A high-throughput and memory-efficient inference and serving engine for LLMs
3 versions - Latest release: 29 days ago - 120 downloads last month - 17,633 stars on GitHub - 2 maintainers
Top 3.4% on pypi.org
vllm 0.4.2
A high-throughput and memory-efficient inference and serving engine for LLMs
27 versions - Latest release: about 1 month ago - 46 dependent packages - 5 dependent repositories - 416 thousand downloads last month - 17,633 stars on GitHub - 2 maintainers
vllm-acc 0.4.1
A high-throughput and memory-efficient inference and serving engine for LLMs
8 versions - Latest release: about 1 month ago - 838 downloads last month - 17,633 stars on GitHub - 1 maintainer
vllm-online 0.4.2
A high-throughput and memory-efficient inference and serving engine for LLMs
2 versions - Latest release: about 1 month ago - 48 downloads last month - 17,633 stars on GitHub - 1 maintainer
nextai-vllm 0.0.7
A high-throughput and memory-efficient inference and serving engine for LLMs
6 versions - Latest release: about 1 month ago - 33 downloads last month - 17,633 stars on GitHub - 1 maintainer
tilearn-test01 0.1
A high-throughput and memory-efficient inference and serving engine for LLMs
1 version - Latest release: 2 months ago - 10 downloads last month - 17,633 stars on GitHub - 1 maintainer
llm-swarm 0.1.1
A high-throughput and memory-efficient inference and serving engine for LLMs
2 versions - Latest release: 3 months ago - 56 downloads last month - 17,633 stars on GitHub - 1 maintainer
superlaser 0.0.6
An MLOps library for LLM deployment w/ the vLLM engine on RunPod's infra.
6 versions - Latest release: 3 months ago - 72 downloads last month - 17,633 stars on GitHub - 1 maintainer
hive-vllm 0.0.1
a
1 version - Latest release: 3 months ago - 8 downloads last month - 17,633 stars on GitHub - 1 maintainer
llm_atc 0.1.7
Tools for fine tuning and serving LLMs
6 versions - Latest release: 6 months ago - 45 downloads last month - 17,633 stars on GitHub - 1 maintainer
vllm-consul 0.2.1
A high-throughput and memory-efficient inference and serving engine for LLMs
5 versions - Latest release: 8 months ago - 21 downloads last month - 12,665 stars on GitHub - 1 maintainer