proxy.golang.org "inference" keyword
Top 5.7% on proxy.golang.org
128 versions - Latest release: 23 days ago - 43,217 stars on GitHub
github.com/deepspeedai/DeepSpeed v0.19.7
DeepSpeed is a deep learning optimization library that makes distributed training and inference e...128 versions - Latest release: 23 days ago - 43,217 stars on GitHub
Top 5.8% on proxy.golang.org
128 versions - Latest release: 25 days ago - 43,217 stars on GitHub
github.com/deepspeedai/deepspeed v0.19.7
DeepSpeed is a deep learning optimization library that makes distributed training and inference e...128 versions - Latest release: 25 days ago - 43,217 stars on GitHub
Top 4.9% on proxy.golang.org
61 versions - Latest release: over 2 years ago - 155 stars on GitHub
github.com/alteryx/woodwork v0.31.0
Woodwork is a Python library that provides robust methods for managing and communicating data typ...61 versions - Latest release: over 2 years ago - 155 stars on GitHub
Top 6.1% on proxy.golang.org
30 versions - Latest release: 5 days ago - 2,131 stars on GitHub
github.com/envoyproxy/ai-gateway v1.2.0
Manages Unified Access to Generative AI Services built on Envoy Gateway30 versions - Latest release: 5 days ago - 2,131 stars on GitHub
Top 6.7% on proxy.golang.org
133 versions - Latest release: 2 days ago - 9,593 stars on GitHub
github.com/xorbitsai/inference v4.0.0+incompatible
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, spee...133 versions - Latest release: 2 days ago - 9,593 stars on GitHub
Top 4.6% on proxy.golang.org
44 versions - Latest release: 2 days ago - 539 stars on GitHub
github.com/kaito-project/aikit v0.23.2
🏗️ Fine-tune, build, and deploy open-source LLMs easily!44 versions - Latest release: 2 days ago - 539 stars on GitHub
Top 9.6% on proxy.golang.org
87 versions - Latest release: 9 days ago - 92,986 stars on GitHub
github.com/vllm-project/vllm v0.31.0 💰
A high-throughput and memory-efficient inference and serving engine for LLMs87 versions - Latest release: 9 days ago - 92,986 stars on GitHub
Top 9.8% on proxy.golang.org
67 versions - Latest release: 10 months ago - 10,884 stars on GitHub
github.com/huggingface/text-generation-inference v3.3.7+incompatible
Large Language Model Text Generation Inference67 versions - Latest release: 10 months ago - 10,884 stars on GitHub
Top 4.7% on proxy.golang.org
3 versions - Latest release: over 2 years ago - 464 stars on GitHub
github.com/ai-hypercomputer/jetstream v0.2.2
JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting ...3 versions - Latest release: over 2 years ago - 464 stars on GitHub
Top 4.4% on proxy.golang.org
4 versions - Latest release: over 2 years ago - 464 stars on GitHub
github.com/AI-Hypercomputer/JetStream v0.2.2
JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting ...4 versions - Latest release: over 2 years ago - 464 stars on GitHub
Top 7.4% on proxy.golang.org
1 version - Latest release: over 2 years ago - 46 stars on GitHub
github.com/makllama/makllama v0.0.0-20240522073254-1938962ace2d
MaK(Mac+Kubernetes)llama - Running and orchestrating large language models (LLMs) on Kubernetes w...1 version - Latest release: over 2 years ago - 46 stars on GitHub
Top 6.6% on proxy.golang.org
10 versions - Latest release: over 2 years ago - 1 dependent repositories - 192 stars on GitHub
github.com/nikolaydubina/llama2.go v0.7.1 💰
LLaMA-2 in native Go10 versions - Latest release: over 2 years ago - 1 dependent repositories - 192 stars on GitHub
Top 9.0% on proxy.golang.org
6 versions - Latest release: over 2 years ago - 10 stars on GitHub
github.com/leliuga/cohere-configurations v1.5.0
Co:Here Inference configurations6 versions - Latest release: over 2 years ago - 10 stars on GitHub
Top 5.9% on proxy.golang.org
3 versions - Latest release: almost 3 years ago - 34 stars on GitHub
github.com/lwch/llama2.go v1.0.2
Port of Facebook's LLaMA 2 model in pure go and use little memory3 versions - Latest release: almost 3 years ago - 34 stars on GitHub
Top 6.7% on proxy.golang.org
21 versions - Latest release: 12 months ago - 25,806 stars on GitHub
github.com/SYSTRAN/faster-whisper v1.2.1
Faster Whisper transcription with CTranslate221 versions - Latest release: 12 months ago - 25,806 stars on GitHub
Top 6.7% on proxy.golang.org
21 versions - Latest release: 12 months ago - 25,807 stars on GitHub
github.com/systran/faster-whisper v1.2.1
Faster Whisper transcription with CTranslate221 versions - Latest release: 12 months ago - 25,807 stars on GitHub
Top 4.9% on proxy.golang.org
1 version - Latest release: 8 months ago - 3 stars on GitHub
github.com/teranos/QNTX/typegen v0.23.0
Package typegen generates type definitions from Go source code for multiple target languages.1 version - Latest release: 8 months ago - 3 stars on GitHub
Top 5.2% on proxy.golang.org
111 versions - Latest release: about 24 hours ago - 3 stars on GitHub
github.com/teranos/QNTX v0.41.5
QNTX = Experiential ꩜ Learning ⌬ System ≡ Attestation +111 versions - Latest release: about 24 hours ago - 3 stars on GitHub
Top 5.2% on proxy.golang.org
111 versions - Latest release: about 24 hours ago - 3 stars on GitHub
github.com/teranos/qntx v0.41.5
QNTX = Experiential ꩜ Learning ⌬ System ≡ Attestation +111 versions - Latest release: about 24 hours ago - 3 stars on GitHub
Top 4.9% on proxy.golang.org
71 versions - Latest release: 14 days ago - 10 stars on GitHub
github.com/wentbackward/hikyaku v0.5.7
Package hikyaku is the public, importable facade for embedding the hikyaku proxy in another progr...71 versions - Latest release: 14 days ago - 10 stars on GitHub
Top 1.0% on proxy.golang.org
17 versions - Latest release: about 2 months ago - 168 stars on GitHub
github.com/NexusGPU/tensor-fusion/api v0.1.16
Tensor Fusion is a state-of-the-art GPU virtualization and pooling solution designed to optimize ...17 versions - Latest release: about 2 months ago - 168 stars on GitHub
Top 4.1% on proxy.golang.org
54 versions - Latest release: almost 5 years ago - 733 stars on GitHub
github.com/onepanelio/onepanel v1.0.2
The open source, end-to-end computer vision platform. Label, build, train, tune, deploy and autom...54 versions - Latest release: almost 5 years ago - 733 stars on GitHub
Top 6.7% on proxy.golang.org
68 versions - Latest release: 2 months ago - 2,991 stars on GitHub
github.com/pytorch/ao v0.18.0
PyTorch native quantization for training and inference68 versions - Latest release: 2 months ago - 2,991 stars on GitHub
Top 4.9% on proxy.golang.org
98 versions - Latest release: 4 days ago - 4,057 stars on GitHub
github.com/uxlfoundation/onednn v3.13.4+incompatible
Open-source library of optimized deep learning operations (matmul, convolution, attention) for CP...98 versions - Latest release: 4 days ago - 4,057 stars on GitHub
Top 4.9% on proxy.golang.org
98 versions - Latest release: 4 days ago - 4,057 stars on GitHub
github.com/uxlfoundation/oneDNN v3.13.4+incompatible
Open-source library of optimized deep learning operations (matmul, convolution, attention) for CP...98 versions - Latest release: 4 days ago - 4,057 stars on GitHub
Top 4.4% on proxy.golang.org
41 versions - Latest release: 4 days ago - 6,397 stars on GitHub
github.com/argmaxinc/argmax-oss-swift v1.1.1
On-device Speech AI for Apple Silicon41 versions - Latest release: 4 days ago - 6,397 stars on GitHub
Top 4.3% on proxy.golang.org
65 versions - Latest release: 12 months ago - 1 dependent repositories - 15,179 stars on GitHub
github.com/gvergnaud/ts-pattern v5.9.0+incompatible 💰
🎨 The exhaustive Pattern Matching library for TypeScript, with smart type inference.65 versions - Latest release: 12 months ago - 1 dependent repositories - 15,179 stars on GitHub
Top 8.2% on proxy.golang.org
21 versions - Latest release: almost 4 years ago - 775 stars on GitHub
github.com/h2non/filetype.py v1.2.0
Small, dependency-free, fast Python package to infer binary file types checking the magic numbers...21 versions - Latest release: almost 4 years ago - 775 stars on GitHub
Top 5.4% on proxy.golang.org
42 versions - Latest release: almost 3 years ago - 413 stars on GitHub
github.com/aws/sagemaker-inference-toolkit v1.10.1
Serve machine learning models within a 🐳 Docker container using 🧠 Amazon SageMaker.42 versions - Latest release: almost 3 years ago - 413 stars on GitHub
Top 5.8% on proxy.golang.org
44 versions - Latest release: 5 days ago - 54,097 stars on GitHub
github.com/ggml-org/whisper.cpp v1.9.5
Port of OpenAI's Whisper model in C/C++44 versions - Latest release: 5 days ago - 54,097 stars on GitHub
Top 6.7% on proxy.golang.org
75 versions - Latest release: over 1 year ago - 36,814 stars on GitHub
github.com/sgl-project/sglang v1.1.1
SGLang is a high-performance serving framework for large language models and multimodal models.75 versions - Latest release: over 1 year ago - 36,814 stars on GitHub
Top 6.6% on proxy.golang.org
6 versions - Latest release: over 5 years ago - 4,656 stars on GitHub
github.com/Tencent/TNN v0.3.0
TNN: developed by Tencent Youtu Lab and Guangying Lab, a uniform deep learning inference framewor...6 versions - Latest release: over 5 years ago - 4,656 stars on GitHub
Top 6.6% on proxy.golang.org
6 versions - Latest release: over 5 years ago - 4,656 stars on GitHub
github.com/tencent/tnn v0.3.0
TNN: developed by Tencent Youtu Lab and Guangying Lab, a uniform deep learning inference framewor...6 versions - Latest release: over 5 years ago - 4,656 stars on GitHub
Top 4.2% on proxy.golang.org
1 version - Latest release: 9 months ago - 952 stars on GitHub
github.com/OpenIntroStat/ims v0.0.0-20260102082327-7227c6e2f90b removed
📚 Introduction to Modern Statistics - A college-level open-source textbook with a modern approach...1 version - Latest release: 9 months ago - 952 stars on GitHub
Top 5.6% on proxy.golang.org
30 versions - Latest release: 9 months ago - 692 stars on GitHub
github.com/derwenai/kglab v1.0.1 💰
Graph Data Science: an abstraction layer in Python for building knowledge graphs, integrated with...30 versions - Latest release: 9 months ago - 692 stars on GitHub
Top 5.6% on proxy.golang.org
30 versions - Latest release: 9 months ago - 692 stars on GitHub
github.com/DerwenAI/kglab v1.0.1 💰
Graph Data Science: an abstraction layer in Python for building knowledge graphs, integrated with...30 versions - Latest release: 9 months ago - 692 stars on GitHub
Top 6.3% on proxy.golang.org
59 versions - Latest release: 5 days ago - 37,204 stars on GitHub
github.com/google-ai-edge/mediapipe v1.1.0
Cross-platform, customizable ML solutions for live and streaming media.59 versions - Latest release: 5 days ago - 37,204 stars on GitHub
Top 1.9% on proxy.golang.org
358 versions - Latest release: 4 months ago - 21 dependent packages - 18 dependent repositories - 53,918 stars on GitHub
github.com/ggerganov/whisper.cpp/bindings/go v0.0.0-20260623071231-43d78af5be58
github.com/ggml-org/whisper.cpp/bindings/go provides a speech-to-text service bindings for the Go...358 versions - Latest release: 4 months ago - 21 dependent packages - 18 dependent repositories - 53,918 stars on GitHub
Top 4.3% on proxy.golang.org
10 versions - Latest release: 4 months ago - 4,583 stars on GitHub
github.com/hao-ai-lab/fastvideo v0.2.0
A unified inference and post-training framework for accelerated video generation.10 versions - Latest release: 4 months ago - 4,583 stars on GitHub
Top 4.4% on proxy.golang.org
9 versions - Latest release: 4 months ago - 4,583 stars on GitHub
github.com/hao-ai-lab/FastVideo v0.2.0
A unified inference and post-training framework for accelerated video generation.9 versions - Latest release: 4 months ago - 4,583 stars on GitHub
Top 4.7% on proxy.golang.org
1 version - Latest release: about 5 years ago - 192 stars on GitHub
github.com/declare-lab/reccon v1.0.0
This repository contains the dataset and the PyTorch implementations of the models from the paper...1 version - Latest release: about 5 years ago - 192 stars on GitHub
Top 5.2% on proxy.golang.org
374 versions - Latest release: over 2 years ago - 2,275 stars on GitHub
github.com/dstackai/dstack/runner v0.0.0-20240119145857-4578e7d87d19 💰
A unified orchestration layer for heterogeneous AI compute. It standardizes how to manage compute...374 versions - Latest release: over 2 years ago - 2,275 stars on GitHub
Top 4.4% on proxy.golang.org
2 versions - Latest release: almost 2 years ago - 2,383 stars on GitHub
github.com/adithya-s-k/AI-Engineering.academy v0.0.0-20250101184059-fef8b8487059 removed 💰
Mastering Applied AI, One Concept at a Time2 versions - Latest release: almost 2 years ago - 2,383 stars on GitHub
Top 4.4% on proxy.golang.org
2 versions - Latest release: about 1 month ago - 10,987 stars on GitHub
github.com/aws/amazon-sagemaker-examples v1.0.0
Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning mod...2 versions - Latest release: about 1 month ago - 10,987 stars on GitHub
Top 4.9% on proxy.golang.org
107 versions - Latest release: 4 days ago - 231 stars on GitHub
github.com/defilantech/LLMKube v0.10.2 💰
Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, ...107 versions - Latest release: 4 days ago - 231 stars on GitHub
Top 1.0% on proxy.golang.org
107 versions - Latest release: 4 days ago - 231 stars on GitHub
github.com/defilantech/llmkube v0.10.2 💰
Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, ...107 versions - Latest release: 4 days ago - 231 stars on GitHub
Top 4.1% on proxy.golang.org
15 versions - Latest release: about 2 years ago - 90 stars on GitHub
github.com/anskarl/LoMRF v1.0.0
LoMRF is an open-source implementation of Markov Logic Networks15 versions - Latest release: about 2 years ago - 90 stars on GitHub
Top 8.2% on proxy.golang.org
15 versions - Latest release: about 2 years ago - 90 stars on GitHub
github.com/anskarl/lomrf v1.0.0
LoMRF is an open-source implementation of Markov Logic Networks15 versions - Latest release: about 2 years ago - 90 stars on GitHub
Top 5.6% on proxy.golang.org
21 versions - Latest release: over 1 year ago - 2,114 stars on GitHub
github.com/deepspeedai/DeepSpeed-MII v0.3.3
MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.21 versions - Latest release: over 1 year ago - 2,114 stars on GitHub
Top 5.8% on proxy.golang.org
21 versions - Latest release: over 1 year ago - 2,114 stars on GitHub
github.com/deepspeedai/deepspeed-mii v0.3.3
MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.21 versions - Latest release: over 1 year ago - 2,114 stars on GitHub
Top 4.5% on proxy.golang.org
22 versions - Latest release: over 1 year ago - 2,114 stars on GitHub
github.com/deepspeedai/DeepSpeed-mii v0.3.3
MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.22 versions - Latest release: over 1 year ago - 2,114 stars on GitHub
Top 4.4% on proxy.golang.org
4 versions - Latest release: 21 days ago - 3,222 stars on GitHub
github.com/openvinotoolkit/openvino_notebooks v0.0.0-20260920170731-804a34bc960f removed
📚 Jupyter notebook tutorials for OpenVINO™4 versions - Latest release: 21 days ago - 3,222 stars on GitHub
Top 4.4% on proxy.golang.org
41 versions - Latest release: about 1 month ago - 636 stars on GitHub
github.com/open-edge-platform/dlstreamer v2026.2.0+incompatible
Deep Learning Streamer (DL Streamer) Pipeline Framework is an open-source streaming media analyti...41 versions - Latest release: about 1 month ago - 636 stars on GitHub
Top 5.5% on proxy.golang.org
306 versions - Latest release: 27 days ago - 168 stars on GitHub
github.com/nexusgpu/tensor-fusion v1.65.16
Tensor Fusion is a state-of-the-art GPU virtualization and pooling solution designed to optimize ...306 versions - Latest release: 27 days ago - 168 stars on GitHub
Top 5.5% on proxy.golang.org
306 versions - Latest release: 27 days ago - 168 stars on GitHub
github.com/NexusGPU/tensor-fusion v1.65.16
Tensor Fusion is a state-of-the-art GPU virtualization and pooling solution designed to optimize ...306 versions - Latest release: 27 days ago - 168 stars on GitHub
Top 1.0% on proxy.golang.org
17 versions - Latest release: about 2 months ago - 168 stars on GitHub
github.com/nexusgpu/tensor-fusion/api v0.1.16
Tensor Fusion is a state-of-the-art GPU virtualization and pooling solution designed to optimize ...17 versions - Latest release: about 2 months ago - 168 stars on GitHub
Top 4.2% on proxy.golang.org
22 versions - Latest release: over 2 years ago - 259 stars on GitHub
github.com/larq/compute-engine v0.16.0
Highly optimized inference engine for Binarized Neural Networks22 versions - Latest release: over 2 years ago - 259 stars on GitHub
Top 4.4% on proxy.golang.org
78 versions - Latest release: 6 months ago - 1,812 stars on GitHub
github.com/ddalcu/mlx-serve v2026.4.11+incompatible 💰
Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig ...78 versions - Latest release: 6 months ago - 1,812 stars on GitHub
Top 4.1% on proxy.golang.org
24 versions - Latest release: about 1 year ago - 13,391 stars on GitHub
github.com/NVIDIA/TensorRT v10.13.3+incompatible
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This rep...24 versions - Latest release: about 1 year ago - 13,391 stars on GitHub
Top 6.7% on proxy.golang.org
24 versions - Latest release: about 1 year ago - 13,391 stars on GitHub
github.com/nvidia/TensorRT v10.13.3+incompatible
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This rep...24 versions - Latest release: about 1 year ago - 13,391 stars on GitHub
Top 8.2% on proxy.golang.org
24 versions - Latest release: about 1 year ago - 13,391 stars on GitHub
github.com/nvidia/tensorrt v10.13.3+incompatible
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This rep...24 versions - Latest release: about 1 year ago - 13,391 stars on GitHub
Top 5.4% on proxy.golang.org
24 versions - Latest release: about 1 year ago - 13,391 stars on GitHub
github.com/NVIDIA/tensorrt v10.13.3+incompatible
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This rep...24 versions - Latest release: about 1 year ago - 13,391 stars on GitHub
Top 6.7% on proxy.golang.org
42 versions - Latest release: over 1 year ago - 3,154 stars on GitHub
github.com/neuralmagic/deepsparse v1.9.0
Sparsity-aware deep learning inference runtime for CPUs42 versions - Latest release: over 1 year ago - 3,154 stars on GitHub
Top 5.2% on proxy.golang.org
15 versions - Latest release: 6 months ago - 3,718 stars on GitHub
github.com/PaddlePaddle/fastdeploy v2.5.0+incompatible
High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle15 versions - Latest release: 6 months ago - 3,718 stars on GitHub
Top 5.2% on proxy.golang.org
15 versions - Latest release: 6 months ago - 3,718 stars on GitHub
github.com/paddlepaddle/fastdeploy v2.5.0+incompatible
High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle15 versions - Latest release: 6 months ago - 3,718 stars on GitHub
Top 5.0% on proxy.golang.org
15 versions - Latest release: 6 months ago - 3,718 stars on GitHub
github.com/PaddlePaddle/FastDeploy v2.5.0+incompatible
High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle15 versions - Latest release: 6 months ago - 3,718 stars on GitHub
Top 6.7% on proxy.golang.org
1 version - Latest release: over 2 years ago - 1 stars on GitHub
github.com/missingstudio/studio v0.0.1
Missing Studio - building highly capable AI Agents1 version - Latest release: over 2 years ago - 1 stars on GitHub
Top 4.4% on proxy.golang.org
7 versions - Latest release: about 2 months ago - 2,933 stars on GitHub
github.com/vllm-project/vllm-ascend v0.23.0 💰
Community maintained hardware plugin for vLLM on Huawei Ascend7 versions - Latest release: about 2 months ago - 2,933 stars on GitHub
Top 5.6% on proxy.golang.org
1 version - Latest release: about 7 years ago - 6,262 stars on GitHub
github.com/trusted-ai/adversarial-robustness-toolbox v1.0.0
Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Po...1 version - Latest release: about 7 years ago - 6,262 stars on GitHub
Top 5.6% on proxy.golang.org
1 version - Latest release: about 7 years ago - 6,262 stars on GitHub
github.com/Trusted-AI/adversarial-robustness-toolbox v1.0.0
Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Po...1 version - Latest release: about 7 years ago - 6,262 stars on GitHub
Top 4.5% on proxy.golang.org
2 versions - Latest release: about 1 month ago - 1,819 stars on GitHub
github.com/SemiAnalysisAI/InferenceX v0.0.0-20260904020438-c355f549ad06 removed
Open Source Inference Research Platform Standard / 开源推理研究平台2 versions - Latest release: about 1 month ago - 1,819 stars on GitHub
Top 6.6% on proxy.golang.org
207 versions - Latest release: 9 days ago - 2,475 stars on GitHub
github.com/roboflow/inference v1.7.3
Turn any computer or edge device into a command center for your computer vision projects.207 versions - Latest release: 9 days ago - 2,475 stars on GitHub
Top 4.4% on proxy.golang.org
6 versions - Latest release: 19 days ago - 273 stars on GitHub
github.com/NVIDIA/TensorRT-Model-Connect v0.0.0-20260922210930-ef359c2a6cdd removed
From PyTorch model to end-to-end TensorRT inference experience in two commands—AI-native, cross-p...6 versions - Latest release: 19 days ago - 273 stars on GitHub
github.com/nvidia/tensorrt-model-connect v0.0.0-20260918074306-82502513326f removed
From PyTorch model to end-to-end TensorRT inference experience in two commands—AI-native, cross-p...7 versions - Latest release: 24 days ago - 273 stars on GitHub
Top 4.9% on proxy.golang.org
4 versions - Latest release: about 6 years ago - 1,606 stars on GitHub
github.com/delta-ml/delta v0.3.3
DELTA is a deep learning based natural language and speech processing platform. LF AI & DATA Proj...4 versions - Latest release: about 6 years ago - 1,606 stars on GitHub
Top 4.9% on proxy.golang.org
4 versions - Latest release: about 6 years ago - 1,606 stars on GitHub
github.com/Delta-ML/delta v0.3.3
DELTA is a deep learning based natural language and speech processing platform. LF AI & DATA Proj...4 versions - Latest release: about 6 years ago - 1,606 stars on GitHub
Top 5.0% on proxy.golang.org
41 versions - Latest release: 19 days ago - 40 stars on GitHub
github.com/inference-sim/inference-sim v0.9.2
Inference Platform Simulation41 versions - Latest release: 19 days ago - 40 stars on GitHub
Top 1.0% on proxy.golang.org
32 versions - Latest release: 5 months ago - 23 stars on GitHub
github.com/inc4/gonka-nop v0.2.2
Gonka AI Node Onboarding Procedure32 versions - Latest release: 5 months ago - 23 stars on GitHub
Top 4.2% on proxy.golang.org
53 versions - Latest release: 8 months ago - 1,225 stars on GitHub
github.com/run-house/kubetorch v0.5.0
Distribute and run AI workloads on Kubernetes magically in Python, like PyTorch for ML infra.53 versions - Latest release: 8 months ago - 1,225 stars on GitHub
Top 5.1% on proxy.golang.org
14 versions - Latest release: 4 months ago - 6,733 stars on GitHub
github.com/kvcache-ai/Mooncake/mooncake-common/etcd v0.0.0-20260616110007-14ab37e7e386
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.14 versions - Latest release: 4 months ago - 6,733 stars on GitHub
Top 4.9% on proxy.golang.org
Latest release: 3 days ago - 6,733 stars on GitHub
github.com/kvcache-ai/Mooncake/mooncake-store/go
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.Latest release: 3 days ago - 6,733 stars on GitHub
Top 4.0% on proxy.golang.org
10 versions - Latest release: 6 months ago - 6,733 stars on GitHub
github.com/kvcache-ai/Mooncake/mooncake-p2p-store/src/p2pstore v0.0.0-20260402140738-5ff565dcabf4
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.10 versions - Latest release: 6 months ago - 6,733 stars on GitHub
Top 5.5% on proxy.golang.org
10 versions - Latest release: 6 months ago - 6,733 stars on GitHub
github.com/kvcache-ai/Mooncake/mooncake-transfer-engine/example/http-metadata-server v0.0.0-20260402140738-5ff565dcabf4
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.10 versions - Latest release: 6 months ago - 6,733 stars on GitHub
Top 4.2% on proxy.golang.org
1 version - Latest release: 18 days ago - 6,733 stars on GitHub
github.com/kvcache-ai/Mooncake/mooncake-common/k8s-lease v0.0.0-20260924065545-056f67a5947b
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.1 version - Latest release: 18 days ago - 6,733 stars on GitHub
Top 4.0% on proxy.golang.org
32 versions - Latest release: about 2 months ago - 6,733 stars on GitHub
github.com/kvcache-ai/Mooncake v0.3.13
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.32 versions - Latest release: about 2 months ago - 6,733 stars on GitHub
Top 4.0% on proxy.golang.org
32 versions - Latest release: about 2 months ago - 6,733 stars on GitHub
github.com/kvcache-ai/mooncake v0.3.13
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.32 versions - Latest release: about 2 months ago - 6,733 stars on GitHub
Top 5.8% on proxy.golang.org
20 versions - Latest release: 3 months ago - 5,792 stars on GitHub
github.com/gpustack/gpustack v2.2.3+incompatible
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-acce...20 versions - Latest release: 3 months ago - 5,792 stars on GitHub
Top 4.8% on proxy.golang.org
1 version - Latest release: 4 months ago - 2 stars on GitHub
github.com/cognisoc/mullama v0.4.0
Run any LLM locally. Use it from any language. Deploy anywhere.1 version - Latest release: 4 months ago - 2 stars on GitHub
Top 4.3% on proxy.golang.org
71 versions - Latest release: 19 days ago - 10,316 stars on GitHub
github.com/runanywhereai/runanywhere-sdks v0.20.37
Production ready toolkit to run AI locally71 versions - Latest release: 19 days ago - 10,316 stars on GitHub
Top 4.4% on proxy.golang.org
71 versions - Latest release: 19 days ago - 10,318 stars on GitHub
github.com/RunanywhereAI/runanywhere-sdks v0.20.37
Production ready toolkit to run AI locally71 versions - Latest release: 19 days ago - 10,318 stars on GitHub
Top 4.8% on proxy.golang.org
246 versions - Latest release: 15 days ago - 48 stars on GitHub
github.com/OpenCSGs/csglite v0.9.88
CSGLite is a lightweight local LLM inference platform. One command downloads, loads, and chats wi...246 versions - Latest release: 15 days ago - 48 stars on GitHub
Top 4.8% on proxy.golang.org
87 versions - Latest release: 15 days ago - 48 stars on GitHub
github.com/opencsgs/csglite v0.9.88
CSGLite is a lightweight local LLM inference platform. One command downloads, loads, and chats wi...87 versions - Latest release: 15 days ago - 48 stars on GitHub
Top 5.6% on proxy.golang.org
28 versions - Latest release: 24 days ago - 1,224 stars on GitHub
github.com/nvidia/kvpress v0.5.5
LLM KV cache compression made easy28 versions - Latest release: 24 days ago - 1,224 stars on GitHub
Top 5.6% on proxy.golang.org
28 versions - Latest release: 24 days ago - 1,224 stars on GitHub
github.com/NVIDIA/kvpress v0.5.5
LLM KV cache compression made easy28 versions - Latest release: 24 days ago - 1,224 stars on GitHub
Top 4.7% on proxy.golang.org
46 versions - Latest release: over 1 year ago - 41,441 stars on GitHub
github.com/Hpcaitech/Colossalai v0.5.1
Making large AI models cheaper, faster and more accessible46 versions - Latest release: over 1 year ago - 41,441 stars on GitHub
Top 4.7% on proxy.golang.org
46 versions - Latest release: over 1 year ago - 41,441 stars on GitHub
github.com/Hpcaitech/colossalai v0.5.1
Making large AI models cheaper, faster and more accessible46 versions - Latest release: over 1 year ago - 41,441 stars on GitHub
Top 4.7% on proxy.golang.org
46 versions - Latest release: over 1 year ago - 41,441 stars on GitHub
github.com/hpcaitech/Colossalai v0.5.1
Making large AI models cheaper, faster and more accessible46 versions - Latest release: over 1 year ago - 41,441 stars on GitHub
Top 9.6% on proxy.golang.org
46 versions - Latest release: over 1 year ago - 41,444 stars on GitHub
github.com/hpcaitech/colossalai v0.5.1
Making large AI models cheaper, faster and more accessible46 versions - Latest release: over 1 year ago - 41,444 stars on GitHub
Top 9.6% on proxy.golang.org
46 versions - Latest release: over 1 year ago - 41,441 stars on GitHub
github.com/hpcaitech/ColossalAI v0.5.1
Making large AI models cheaper, faster and more accessible46 versions - Latest release: over 1 year ago - 41,441 stars on GitHub
Top 4.8% on proxy.golang.org
1 version - Latest release: 6 months ago - 14 stars on GitHub
github.com/lynxai-team/goinfer v0.0.0-20260426125723-90052f829412
Local LLM proxy, DevOps friendly1 version - Latest release: 6 months ago - 14 stars on GitHub
Related Keywords
llm
111
gpu
87
kubernetes
87
deep-learning
77
pytorch
76
ai
61
machine-learning
57
transformer
48
vllm
43
nvidia
36
python
29
quantization
25
tensorrt
23
transformers
21
onnx
21
cuda
21
large-language-models
21
tensorflow
20
cloud-functions
20
serverless-gpu
20
openai
19
go
18
ai-gateway
18
sglang
18
openvino
18
operator
17
training
17
agentic
17
llm-serving
17
llama
17
leader-worker
16
model-serving
16
llmops
16
grove
16
topology-aware-scheduling
16
multinode
16
golang
16
llm-inference
16
role-based
16
gang-scheduling
16
disaggregated
16
auto-scaling
16
auto-scaling-group
16
llama-cpp
16
llms
15
mlops
14
nlp
13
reinforcement-learning
12
qwen
12
semantic-router
12
inference-engine
12
mixture-of-models
12
llmrouter
12
self-hosted
12
guardrails
12
local-llm
11
computer-vision
11
generative-ai
11
speech-recognition
11
huggingface
11
openapi
10
openai-api
10
cli
10
api
10
grpc
10
pipeline-parallelism
10
local-ai
10
kv-cache
10
whisper
10
onnxruntime
10
reasoning
10
cpp
10
logic
9
docker
9
edge
9
gguf
9
ai-agents
9
speech-to-text
9
serving
9
data-parallelism
9
model-parallelism
9
neural-network
9
fine-tuning
8
gpt
8
translation
8
compression
8
ollama
8
tensorflow-lite
8
mcp
8
macos
8
rdma
8
gpu-acceleration
8
open-source
8
tui
8
autoscaling
8
agentic-ai
8
k8s
7
disaggregation
7
tokenspeed
7
kvcache
7