Ecosyste.ms: Packages
An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.
pypi.org "inferentia" keyword
Top 3.4% on pypi.org
27 versions - Latest release: 28 days ago - 46 dependent packages - 5 dependent repositories - 408 thousand downloads last month - 17,633 stars on GitHub - 2 maintainers
vllm 0.4.2
A high-throughput and memory-efficient inference and serving engine for LLMs27 versions - Latest release: 28 days ago - 46 dependent packages - 5 dependent repositories - 408 thousand downloads last month - 17,633 stars on GitHub - 2 maintainers
fmbench 1.0.36
Benchmark performance of **any model** deployed on **Amazon SageMaker** or available on **Amazon ...41 versions - Latest release: 16 days ago - 831 downloads last month - 77 stars on GitHub - 1 maintainer
optimum-neuron 0.0.22
Optimum Neuron is the interface between the Hugging Face Transformers and Diffusers libraries and...24 versions - Latest release: 26 days ago - 1 dependent repositories - 35.3 thousand downloads last month - 164 stars on GitHub - 3 maintainers
vllm-acc 0.4.1
A high-throughput and memory-efficient inference and serving engine for LLMs8 versions - Latest release: 29 days ago - 1.4 thousand downloads last month - 17,633 stars on GitHub - 1 maintainer
vllm-xft 0.3.3.1
A high-throughput and memory-efficient inference and serving engine for LLMs2 versions - Latest release: 23 days ago - 113 downloads last month - 17,633 stars on GitHub - 2 maintainers
hive-vllm 0.0.1
a1 version - Latest release: 3 months ago - 10 downloads last month - 17,633 stars on GitHub - 1 maintainer
vllm-online 0.4.2
A high-throughput and memory-efficient inference and serving engine for LLMs2 versions - Latest release: about 1 month ago - 60 downloads last month - 17,633 stars on GitHub - 1 maintainer
tilearn-test01 0.1
A high-throughput and memory-efficient inference and serving engine for LLMs1 version - Latest release: about 2 months ago - 9 downloads last month - 17,633 stars on GitHub - 1 maintainer
llm-swarm 0.1.1
A high-throughput and memory-efficient inference and serving engine for LLMs2 versions - Latest release: 3 months ago - 53 downloads last month - 17,633 stars on GitHub - 1 maintainer
tilearn-infer 0.3.3
A high-throughput and memory-efficient inference and serving engine for LLMs3 versions - Latest release: about 1 month ago - 20 downloads last month - 17,633 stars on GitHub - 1 maintainer
llm_atc 0.1.7
Tools for fine tuning and serving LLMs6 versions - Latest release: 6 months ago - 32 downloads last month - 17,633 stars on GitHub - 1 maintainer
nextai-vllm 0.0.7
A high-throughput and memory-efficient inference and serving engine for LLMs6 versions - Latest release: about 1 month ago - 41 downloads last month - 17,633 stars on GitHub - 1 maintainer
superlaser 0.0.6
An MLOps library for LLM deployment w/ the vLLM engine on RunPod's infra.6 versions - Latest release: 3 months ago - 42 downloads last month - 17,633 stars on GitHub - 1 maintainer
fmbt 1.0.7
Benchmark performance of **any model** on **any supported instance type** on Amazon SageMaker.8 versions - Latest release: 4 months ago - 38 downloads last month - 77 stars on GitHub - 2 maintainers
Related Keywords
inference
12
transformer
11
trainium
11
rocm
11
pytorch
11
model-serving
11
mlops
11
llmops
11
llm-serving
11
llm
11
llama
11
gpt
11
cuda
11
amd
11
p4d
2
llama2
2
benchmark
2
foundation-models
2
generative-ai
2
bedrock
2
sagemaker
2
benchmarking
2
cicd
1
runpod
1
vllm
1
deployment
1
MLOps
1
NLP
1
LLM
1
server
1
aws
1
tranium
1
fine-tuning
1
mixed-precision training
1
diffusers
1
transformers
1
bring your own endpoint
1