crates.io "inference" keyword
View the packages on the crates.io package registry that are tagged with the "inference" keyword.
shimmy 0.1.0
5MB Ollama alternative with dynamic port management and zero-friction LoRA serving. 95%+ test cov...1 version - Latest release: 1 day ago - 0 downloads total - 0 stars on GitHub - 1 maintainer
llama_cpp_rs 0.3.0 💰
Rust bindings for LLAMA.CPP inference5 versions - Latest release: almost 2 years ago - 10.8 thousand downloads total - 316 stars on GitHub - 1 maintainer
infers-jsonschema 0.1.0 💰
A crate for inferring JSON Schema from input data1 version - Latest release: over 5 years ago - 1 dependent package - 2 dependent repositories - 3.98 thousand downloads total - 8 stars on GitHub - 1 maintainer
libinfer 0.0.3
Rust interface to TensorRT for high-performance GPU inference3 versions - Latest release: 16 days ago - 972 downloads total - 1 stars on GitHub - 1 maintainer
cervo-runtime 0.9.0
Multi-model multi-agent RL runtime for games.11 versions - Latest release: 1 day ago - 1 dependent package - 1 dependent repositories - 52.3 thousand downloads total - 43 stars on GitHub - 1 maintainer
cervo-asset 0.9.0
A unified asset type for Cervo using ONNX or NNEF models.13 versions - Latest release: 1 day ago - 2 dependent packages - 1 dependent repositories - 57.1 thousand downloads total - 43 stars on GitHub - 1 maintainer
cervo-nnef 0.9.0
Extends cervo with nnef support from tract-nnef.13 versions - Latest release: 1 day ago - 2 dependent packages - 1 dependent repositories - 57.7 thousand downloads total - 43 stars on GitHub - 1 maintainer
cervo-core 0.9.0
High level APIs for RL in games.13 versions - Latest release: 1 day ago - 5 dependent packages - 1 dependent repositories - 58.7 thousand downloads total - 43 stars on GitHub - 1 maintainer
kv-cache 0.1.0
LLM inference in Rust1 version - Latest release: about 1 month ago - 244 downloads total - 0 stars on GitHub - 1 maintainer
polytype 7.0.1
A Hindley-Milner polymorphic typing system.34 versions - Latest release: over 1 year ago - 1 dependent package - 7 dependent repositories - 69 thousand downloads total - 59 stars on GitHub - 2 maintainers
scaled-dot-product-attention 0.1.0
LLM inference in Rust1 version - Latest release: about 1 month ago - 235 downloads total - 0 stars on GitHub - 1 maintainer
inference-gateway-adk 0.2.2
Agent Development Kit (ADK) for building powerful, interoperable AI agents with the Agent-to-Agen...4 versions - Latest release: about 1 month ago - 906 downloads total - 1 stars on GitHub - 1 maintainer
all-close 0.1.0
LLM inference in Rust1 version - Latest release: about 1 month ago - 271 downloads total - 0 stars on GitHub - 1 maintainer
specs-visitor 0.3.0
Utilities for visiting entities in an ECS that is using the specs library.4 versions - Latest release: over 6 years ago - 3 dependent packages - 1 dependent repositories - 8.17 thousand downloads total - 64 stars on GitHub - 1 maintainer
inference_graph 0.1.0
For building graphs of async tasks by having each node only refer to its own inputs.1 version - Latest release: almost 3 years ago - 1.44 thousand downloads total - 0 stars on GitHub - 1 maintainer
runnx 0.2.1
A minimal, verifiable ONNX runtime implementation in Rust4 versions - Latest release: 4 days ago - 750 downloads total - 8 stars on GitHub - 1 maintainer
kn-cuda-sys 0.7.3
A wrapper around the CUDA APIs15 versions - Latest release: about 1 year ago - 2 dependent packages - 18.9 thousand downloads total - 63 stars on GitHub - 1 maintainer
kn-runtime 0.7.3
Dynamic wrapper around CPU and GPU inference6 versions - Latest release: about 1 year ago - 7.68 thousand downloads total - 63 stars on GitHub - 1 maintainer
kn-cuda-eval 0.7.3
A CUDA executor for neural network graphs15 versions - Latest release: about 1 year ago - 1 dependent package - 18.8 thousand downloads total - 63 stars on GitHub - 1 maintainer
kn-graph 0.7.3
A neural network inference graph intermediate representation, with surrounding utilities.17 versions - Latest release: about 1 year ago - 2 dependent packages - 20.6 thousand downloads total - 63 stars on GitHub - 1 maintainer
ort-candle 0.1.0+0.8 💰
ort + candle = 🦀 - An alternative backend for ort, powered by candle.1 version - Latest release: 3 months ago - 417 downloads total - 1,526 stars on GitHub - 1 maintainer
ort-tract 0.1.0+0.21 💰
ort + tract = 🦀 - An alternative backend for ort, powered by tract.1 version - Latest release: 3 months ago - 697 downloads total - 1,526 stars on GitHub - 1 maintainer
Top 6.4% on crates.io
40 versions - Latest release: almost 2 years ago - 26 dependent packages - 32 dependent repositories - 3.29 million downloads total - 529 stars on GitHub - 1 maintainer
ort 1.16.3 💰
A safe Rust wrapper for ONNX Runtime 1.22 - Optimize and accelerate machine learning inference & ...40 versions - Latest release: almost 2 years ago - 26 dependent packages - 32 dependent repositories - 3.29 million downloads total - 529 stars on GitHub - 1 maintainer
ort-sys 2.0.0-rc.10 💰
Unsafe Rust bindings for ONNX Runtime 1.22 - Optimize and Accelerate Machine Learning Inferencing16 versions - Latest release: 3 months ago - 1 dependent package - 3.01 million downloads total - 529 stars on GitHub - 1 maintainer
pipeless-ai 1.11.0
An open-source computer vision framework to build and deploy applications in minutes26 versions - Latest release: over 1 year ago - 29 thousand downloads total - 763 stars on GitHub - 1 maintainer
processors-rs 0.6.4
Embed anything at lightning speed4 versions - Latest release: 3 months ago - 1.98 thousand downloads total - 700 stars on GitHub - 1 maintainer
embed_anything 0.6.4
Embed anything at lightning speed16 versions - Latest release: 3 months ago - 13.3 thousand downloads total - 700 stars on GitHub - 1 maintainer
inference_core 0.1.0
Feakin is a architecture design and visual collaboration tool. This is the parser for Feakin.1 version - Latest release: almost 2 years ago - 1.37 thousand downloads total - 45 stars on GitHub - 2 maintainers
enfer_core 0.1.2
Feakin is a architecture design and visual collaboration tool. This is the parser for Feakin.3 versions - Latest release: almost 2 years ago - 3.94 thousand downloads total - 45 stars on GitHub - 1 maintainer
enfer 0.1.0
Feakin is a architecture design and visual collaboration tool. This is the parser for Feakin.1 version - Latest release: almost 2 years ago - 1.34 thousand downloads total - 45 stars on GitHub - 1 maintainer
tin 0.3.0
tin: a statically structurally typed embeddable programming language6 versions - Latest release: over 6 years ago - 10.1 thousand downloads total - 64 stars on GitHub - 1 maintainer
bitnet-inference 0.1.2
High-performance inference engine for BitNet models3 versions - Latest release: 7 days ago - 528 downloads total - 1 stars on GitHub - 1 maintainer
paddle-inference-rs 0.1.0
Rust bindings for PaddlePaddle inference library1 version - Latest release: 8 days ago - 0 downloads total - 1 maintainer
cervo 0.8.0
High level APIs for RL in games.12 versions - Latest release: 3 months ago - 1 dependent package - 1 dependent repositories - 15 thousand downloads total - 43 stars on GitHub - 1 maintainer
ask-bayes 0.2.1 💰
CLI tool for Bayesian inference6 versions - Latest release: over 3 years ago - 7.97 thousand downloads total - 1 stars on GitHub - 1 maintainer
specs-visitor-derive 0.3.0
A procedural macro for deriving traits in specs-derive.4 versions - Latest release: over 6 years ago - 2 dependent packages - 1 dependent repositories - 7.26 thousand downloads total - 64 stars on GitHub - 1 maintainer
cervo-cli 0.8.0
High level APIs for RL in games.8 versions - Latest release: 3 months ago - 7.14 thousand downloads total - 43 stars on GitHub - 1 maintainer
orkhon 0.2.3 💰
Machine Learning Inference Framework and Server Runtime8 versions - Latest release: over 4 years ago - 25.7 thousand downloads total - 145 stars on GitHub - 1 maintainer
liquid-edge 0.2.1
Minimal edge inference runtime for LLMs2 versions - Latest release: about 1 month ago - 418 downloads total - 59 stars on GitHub - 1 maintainer
typeql 3.2.0
TypeQL Language for Rust24 versions - Latest release: 4 months ago - 1 dependent package - 20.8 thousand downloads total - 232 stars on GitHub - 1 maintainer
inference-gateway-sdk 0.11.0
Rust SDK for interacting with various language models through the Inference Gateway38 versions - Latest release: about 1 month ago - 21.3 thousand downloads total - 2 stars on GitHub - 1 maintainer
cervo-onnx 0.8.0
Extends cervo with ONNX support from tract-onnx.12 versions - Latest release: 3 months ago - 2 dependent packages - 1 dependent repositories - 57.5 thousand downloads total - 43 stars on GitHub - 1 maintainer
wat-fmt 0.0.8 💰
A pretty formatter for WebAssembly Text Format1 version - Latest release: 5 months ago - 532 downloads total - 2 stars on GitHub - 1 maintainer
inf-wasmparser 0.0.8 💰
A simple event-driven library for parsing Inferara non-deterministic WASM extension WebAssembly b...4 versions - Latest release: 5 months ago - 2.17 thousand downloads total - 2 stars on GitHub - 1 maintainer
inf-wast 0.0.8 💰
inf-wasm-tool inf-wast crate fork with Inference non-detenministic instructions extension8 versions - Latest release: 5 months ago - 4.72 thousand downloads total - 2 stars on GitHub - 1 maintainer
tree-sitter-inference 0.0.35
Inference grammar for tree-sitter35 versions - Latest release: 13 days ago - 26.1 thousand downloads total - 1 maintainer
curtana 0.1.1
Simplified zero-cost wrapper over llama.cpp powered by lama-cpp-2.3 versions - Latest release: about 2 months ago - 1.01 thousand downloads total - 1 stars on GitHub - 1 maintainer
bitnet-cpp-sys 0.0.5
Low Level Bindings to bitnet.cpp5 versions - Latest release: 8 months ago - 5.39 thousand downloads total - 1 maintainer
lmonade-runtime 0.1.0-alpha.2
Actor-based runtime for LLM inference orchestration and resource management2 versions - Latest release: 15 days ago - 116 downloads total - 1 maintainer
lmonade 0.1.0-alpha.2
LLM inference engine - main crate with CLI and re-exports2 versions - Latest release: 15 days ago - 112 downloads total - 1 maintainer
lllllxxxxx 0.0.7
TypeQL Grammar for Rust7 versions - Latest release: over 3 years ago - 8.39 thousand downloads total - 233 stars on GitHub - 1 maintainer
cerebras-rs 0.0.2
High-performance Rust SDK for Cerebras Inference API - Low-latency AI model inference powered by ...2 versions - Latest release: 3 months ago - 647 downloads total - 1 maintainer
leaves-rs 0.0.1
Pure Rust implementation of prediction part for GBRT (Gradient Boosting Regression Trees) models ...1 version - Latest release: 10 months ago - 806 downloads total - 0 stars on GitHub - 1 maintainer
moonlite 0.1.0
A machine learning framework for efficient, cost-effective, scalable inference, and distributed, ...1 version - Latest release: 6 months ago - 553 downloads total - 0 stars on GitHub - 1 maintainer
bayes 0.0.1
Composable abstractions to build probabilistic models and inference algorithms1 version - Latest release: over 5 years ago - 1.63 thousand downloads total - 1 maintainer
genson-rs 0.2.0
Extremely fast JSON Schema inference engine built in Rust2 versions - Latest release: over 1 year ago - 2.77 thousand downloads total - 81 stars on GitHub - 1 maintainer
wasi-nn-safe 0.1.0
High-level and Safe Rust API bindings for wasi-nn system calls7 versions - Latest release: about 2 years ago - 8.24 thousand downloads total - 0 stars on GitHub - 1 maintainer
tin-lang 0.2.2
tin: a statically structurally typed embeddable programming language4 versions - Latest release: over 6 years ago - 5.72 thousand downloads total - 64 stars on GitHub - 1 maintainer
ai_kit 0.1.0
Collection of classic AI algorithms with convenient interfaces1 version - Latest release: about 8 years ago - 2.13 thousand downloads total - 13 stars on GitHub - 1 maintainer
fashion-clip-rs 0.1.5
A complete(grpc service and lib) Rust inference with multilingual embedding support. This version...5 versions - Latest release: over 1 year ago - 6.17 thousand downloads total - 39 stars on GitHub - 1 maintainer
asdi 0.2.5
Another Simplistic Datalog Implementation (in Rust)7 versions - Latest release: over 3 years ago - 9.17 thousand downloads total - 4 stars on GitHub - 1 maintainer
triton-distributed 0.1.2+cb8ea1b
Distributed GenAI Inference Framework2 versions - Latest release: 7 months ago - 1.37 thousand downloads total - 51 stars on GitHub - 2 maintainers
triton-client 0.2.0
A client for interfacing with NVIDIA Triton inference server.3 versions - Latest release: about 3 years ago - 7.42 thousand downloads total - 30 stars on GitHub - 2 maintainers
llmnop 0.2.0
A command-line tool for benchmarking the performance of LLM inference endpoints.2 versions - Latest release: about 2 months ago - 664 downloads total - 3 stars on GitHub - 1 maintainer
typeql_lang
TypeQL Language for Rust1 version - Latest release: 23 days ago - 1 dependent package - 1.03 thousand downloads total - 231 stars on GitHub - 2 maintainers
bitnet-cpp 0.0.5
bitnet.cpp bindings for Rust4 versions - Latest release: 8 months ago - 3.08 thousand downloads total - 1 maintainer
onnxruntime-sys-patch 0.0.14
Unsafe wrapper around Microsoft's ONNX Runtime1 version - Latest release: over 3 years ago - 1.46 thousand downloads total - 305 stars on GitHub - 1 maintainer
onnxruntime-sys-ng 1.16.1
Unsafe wrapper around Microsoft's ONNX Runtime2 versions - Latest release: almost 2 years ago - 1 dependent package - 3.22 thousand downloads total - 305 stars on GitHub - 1 maintainer
Top 9.0% on crates.io
13 versions - Latest release: about 4 years ago - 2 dependent packages - 9 dependent repositories - 72.9 thousand downloads total - 305 stars on GitHub - 1 maintainer
onnxruntime-sys 0.0.14
Unsafe wrapper around Microsoft's ONNX Runtime13 versions - Latest release: about 4 years ago - 2 dependent packages - 9 dependent repositories - 72.9 thousand downloads total - 305 stars on GitHub - 1 maintainer
onnxruntime 0.0.14
Wrapper around Microsoft's ONNX Runtime13 versions - Latest release: about 4 years ago - 2 dependent packages - 5 dependent repositories - 61.3 thousand downloads total - 305 stars on GitHub - 1 maintainer
onnxruntime-ng 1.16.1
Wrapper around Microsoft's ONNX Runtime2 versions - Latest release: almost 2 years ago - 2.65 thousand downloads total - 305 stars on GitHub - 1 maintainer
programinduction 0.9.0
A library for program induction and learning representations.37 versions - Latest release: over 1 year ago - 3 dependent repositories - 51.6 thousand downloads total - 31 stars on GitHub - 2 maintainers
tritonserver-rs 0.4.1
Pefrorm easy and efficient ML models inference7 versions - Latest release: about 2 months ago - 4.3 thousand downloads total - 25 stars on GitHub - 1 maintainer
streamfer 0.1.0
Streaming inference server1 version - Latest release: almost 2 years ago - 1.28 thousand downloads total - 1 maintainer
ushi 0.1.1
High-performance LLM inference server with llama.cpp FFI bindings2 versions - Latest release: 26 days ago - 0 downloads total - 1 maintainer
lmcpp 0.1.1
Rust bindings for llama.cpp's server with managed toolchain, typed endpoints, and UDS/HTTP support2 versions - Latest release: about 1 month ago - 197 downloads total - 208 stars on GitHub - 1 maintainer
round-multiple 0.1.0
LLM inference in Rust1 version - Latest release: about 1 month ago - 0 downloads total - 0 stars on GitHub - 1 maintainer
llmterm 0.2.3 💰
Your friendly local LLM terminal companion2 versions - Latest release: 6 months ago - 1.02 thousand downloads total - 8 stars on GitHub - 1 maintainer
bayes-rs 0.1.0
A Rust library for Bayesian inference with MCMC samplers1 version - Latest release: 2 months ago - 254 downloads total - 1 maintainer
mlua-tract 0.1.2
Lua bindings for tract, the tiny, no-nonsense, self-contained, Rust-based Tensorflow and ONNX inf...3 versions - Latest release: over 1 year ago - 4.09 thousand downloads total - 0 stars on gitlab.com - 1 maintainer
type_eq 0.1.2
Tiny utils for type-level constraints and inference3 versions - Latest release: about 7 years ago - 4.3 thousand downloads total - 0 stars on GitHub - 1 maintainer
llama-rust 0.1.0
LLM inference in Rust1 version - Latest release: 2 months ago - 242 downloads total - 1 stars on GitHub - 1 maintainer
inference-gateway 0.1.0 removed
Rust SDK for interacting with various language models through the Inference Gateway1 version - Latest release: 8 months ago - 0 downloads total - 0 stars on GitHub - 1 maintainer
Related Keywords
rust
32
llm
19
onnx
18
machine-learning
18
ai
16
onnxruntime
12
llama
9
onnx-runtime
7
bindings
7
ml
7
machinelearning
7
language
6
neuralnetworks
5
onnxruntime-sys
5
candle
5
rust-wrapper
5
deep-learning
4
edge
4
async
4
structural-typing
4
jit-compiler
4
jit
4
embeddable
4
logic
4
polymorphism
4
wasm
4
model
4
transformers
4
huggingface
4
python
3
embedding-models
3
webassembly
3
api-bindings
3
sys
3
formal-specification
3
formal-methods
3
runtime
3
bitnet
3
bayesian
3
typeql
3
reasoning
3
query-language
3
query
3
strongly-typed
3
polymorphic
3
database
3
typedb
3
type-system
3
inference-gateway
3
gateway
3
statistics
3
colpali
2
learning
2
high-performance
2
hybrid
2
indexing
2
information-retrieval
2
ingestion
2
large-language-models
2
bitnet-cpp
2
production
2
rag
2
rust-lang
2
vector-database
2
embedding
2
tensorflow
2
sdk
2
parser
2
rust-sdk
2
vector-search
2
vision-language-model
2
server
2
type
2
gpu
2
cuda
2
graph
2
neural-network
2
ai-training
2
fine-tuning
2
cpp
2
triton
2
nvidia
2
inference-server
2
gguf
2
fashion-clip
1
pcfg
1
distributed
1
gradient-boosting
1
lightgbm
1
program-induction
1
representation-learning
1
actor
1
assistant
1
kalosm
1
local-llm
1
shell
1
grakn
1
ggml-rs
1
terminal
1
knowledgebase
1