crates.io "data-engineering" keyword
evolution 1.3.0
Efficiently evolve your old fixed-length data files into modern file formats.4 versions - Latest release: over 1 year ago - 4.58 thousand downloads total - 7 stars on GitHub - 1 maintainer
ceres-server 0.3.5
REST API server for Ceres harvesting, embedding, and search workflows5 versions - Latest release: 9 days ago - 73 downloads total - 10 stars on GitHub - 1 maintainer
pipebase 0.2.0
A tokio based runtime library for data integration app13 versions - Latest release: over 4 years ago - 19 dependent packages - 28.1 thousand downloads total - 8 stars on GitHub - 1 maintainer
dink 0.1.0
Threaded CDC to Delta lake, written in Rust, just like God intended it.1 version - Latest release: over 2 years ago - 1.52 thousand downloads total - 1 maintainer
pipes3 0.1.5
A pipebase plugin using aws s3 rust sdk1 version - Latest release: about 4 years ago - 1.82 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipebuilder_common 0.2.2
lib for pipebuilder components12 versions - Latest release: about 4 years ago - 3 dependent packages - 18.5 thousand downloads total - 1 stars on GitHub - 1 maintainer
simplygo 0.2.0
Simplygo SDK for Rust.2 versions - Latest release: almost 2 years ago - 3.78 thousand downloads total - 11 stars on GitHub - 1 maintainer
pq-parquet 1.0.5
The jq of Parquet. Inspect, transform, and operate on Parquet files from your terminal. S3, GCS, ...4 versions - Latest release: 7 days ago - 46 downloads total - 1 maintainer
elusion 8.3.0 💰
Elusion is a modern DataFrame / Data Engineering / Data Analysis library that combines the famili...106 versions - Latest release: 7 days ago - 60.7 thousand downloads total - 125 stars on GitHub - 1 maintainer
tl-stream 0.3.1
Streaming and pipeline engine for ThinkingLanguage (Phase 4)3 versions - Latest release: 28 days ago - 111 downloads total - 0 stars on GitHub - 1 maintainer
tl-ai 0.3.1
AI/ML integration for ThinkingLanguage — tensors, training, inference, LLM3 versions - Latest release: 28 days ago - 128 downloads total - 0 stars on GitHub - 1 maintainer
tl-data 0.3.1
Data engine for ThinkingLanguage — DataFusion, connectors, I/O3 versions - Latest release: 28 days ago - 111 downloads total - 0 stars on GitHub - 1 maintainer
tl-lexer 0.3.1
Lexer/tokenizer for ThinkingLanguage3 versions - Latest release: 28 days ago - 155 downloads total - 1 maintainer
thinkinglanguage-types 0.3.1
ThinkingLanguage — Type system and type checker3 versions - Latest release: 28 days ago - 80 downloads total - 1 maintainer
tl-ast 0.3.1
Abstract Syntax Tree definitions for ThinkingLanguage3 versions - Latest release: 28 days ago - 221 downloads total - 1 maintainer
tl-errors 0.3.1
Error types and diagnostics for the ThinkingLanguage compiler3 versions - Latest release: 28 days ago - 250 downloads total - 1 maintainer
tl-ir 0.3.1
Intermediate Representation for ThinkingLanguage table pipe optimization3 versions - Latest release: 28 days ago - 110 downloads total - 1 maintainer
tl-parser 0.3.1
Recursive descent parser for ThinkingLanguage3 versions - Latest release: 28 days ago - 138 downloads total - 1 maintainer
cargo-pipe 0.2.2
A command-line tool to generate and build data integration app using pipebase framework22 versions - Latest release: about 4 years ago - 27.8 thousand downloads total - 8 stars on GitHub - 1 maintainer
piperedis 0.1.6
A pipebase plugin using redis rust client7 versions - Latest release: over 4 years ago - 9.24 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipe-audit-core 0.2.1
Universal data validation and compliance audit engine11 versions - Latest release: 5 months ago - 2.43 thousand downloads total - 1 stars on GitHub - 1 maintainer
pipeamqp 0.1.3
A pipebase plugin using lapin4 versions - Latest release: over 4 years ago - 5.64 thousand downloads total - 8 stars on GitHub - 1 maintainer
rigatoni-core 0.2.0
Core traits, pipeline orchestration, and MongoDB integration for Rigatoni CDC/Data Replication fr...6 versions - Latest release: 4 months ago - 386 downloads total - 0 stars on GitHub - 1 maintainer
pipepsql 0.2.2
A pipebase plugin using tokio_postgres9 versions - Latest release: over 4 years ago - 11.6 thousand downloads total - 8 stars on GitHub - 1 maintainer
padder 2.1.0
A highly efficient Rust crate for padding data during runtime.8 versions - Latest release: 8 months ago - 10.3 thousand downloads total - 3 stars on GitHub - 1 maintainer
solrcopy 0.9.2 💰
Command line tool useful for migration, transformations, backup, and restore of documents stored ...5 versions - Latest release: 13 days ago - 550 downloads total - 9 stars on GitHub - 1 maintainer
timeseries-table-datafusion 0.1.2
DataFusion SQL integration for timeseries-table-core tables (TableProvider + segment pruning).2 versions - Latest release: about 2 months ago - 33 downloads total - 3 stars on GitHub - 1 maintainer
xvc-ecs 0.7.0 💰
Entity-Component System for Xvc32 versions - Latest release: 3 months ago - 6 dependent packages - 1 dependent repositories - 34 thousand downloads total - 29 stars on GitHub - 1 maintainer
thepipelinetool 0.2.7
readers used for thepipelinetool18 versions - Latest release: almost 2 years ago - 1 dependent package - 21.4 thousand downloads total - 18 stars on GitHub - 1 maintainer
pipebuilder 0.2.2
CI for pipebase application12 versions - Latest release: about 4 years ago - 14.5 thousand downloads total - 1 stars on GitHub - 1 maintainer
hdbconnect-arrow 0.3.9
Apache Arrow integration for hdbconnect SAP HANA driver18 versions - Latest release: 18 days ago - 241 downloads total - 1 stars on GitHub - 1 maintainer
hdbconnect-mcp 0.3.9
MCP server for SAP HANA database8 versions - Latest release: 18 days ago - 91 downloads total - 1 stars on GitHub - 1 maintainer
timeseries-table-cli 0.1.2
Command-line tool for creating, managing, and querying time-series tables (powered by DataFusion).3 versions - Latest release: about 2 months ago - 75 downloads total - 12 stars on GitHub - 1 maintainer
timeseries-table-core 0.2.2
Core engine for a Rust-native log-structured time-series table format (metadata log + coverage bi...2 versions - Latest release: about 2 months ago - 116 downloads total - 3 stars on GitHub - 1 maintainer
timeseries-table-format 0.1.4
Append-only time-series table format with gap/overlap tracking3 versions - Latest release: about 2 months ago - 75 downloads total - 3 stars on GitHub - 1 maintainer
magnetise 0.1.2
A Rust library to asses the similarity between SQL queries.3 versions - Latest release: over 1 year ago - 3.19 thousand downloads total - 2 stars on GitHub - 1 maintainer
thepipelinetool_core 0.2.7
An *experimental* pipeline orchestration tool drawing on concepts from Apache Airflow7 versions - Latest release: almost 2 years ago - 1 dependent package - 8.63 thousand downloads total - 33 stars on GitHub - 1 maintainer
xvc-logging 0.7.0 💰
Logging crate for Xvc31 versions - Latest release: 3 months ago - 10 dependent packages - 1 dependent repositories - 36.7 thousand downloads total - 29 stars on GitHub - 1 maintainer
xvc-pipeline 0.7.0 💰
Xvc data pipeline management31 versions - Latest release: 3 months ago - 2 dependent packages - 1 dependent repositories - 30.6 thousand downloads total - 29 stars on GitHub - 1 maintainer
datamixer 0.0.1
Datamixer client library (upcoming)1 version - Latest release: about 3 years ago - 1.5 thousand downloads total - 0 stars on GitHub - 1 maintainer
pipe-audit 0.2.1
Command-line interface for the pipa audit engine12 versions - Latest release: 5 months ago - 2.42 thousand downloads total - 0 stars on GitHub - 1 maintainer
rusty-logging 0.6.0 💰
Logging for OpsML3 versions - Latest release: 3 months ago - 22.5 thousand downloads total - 21 stars on GitHub - 1 maintainer
xvc-storage 0.7.0 💰
Xvc remote and local storage management33 versions - Latest release: 3 months ago - 3 dependent packages - 1 dependent repositories - 33.4 thousand downloads total - 29 stars on GitHub - 1 maintainer
xvc-walker 0.7.0 💰
Xvc parallel file system walker with ignore features36 versions - Latest release: 3 months ago - 7 dependent packages - 1 dependent repositories - 39.7 thousand downloads total - 29 stars on GitHub - 1 maintainer
xvc-config 0.7.0 💰
Xvc configuration management33 versions - Latest release: 3 months ago - 6 dependent packages - 1 dependent repositories - 34.9 thousand downloads total - 29 stars on GitHub - 1 maintainer
xvc-core 0.7.0 💰
Xvc core for common elements for all commands34 versions - Latest release: 3 months ago - 5 dependent packages - 1 dependent repositories - 35.2 thousand downloads total - 29 stars on GitHub - 1 maintainer
pipegen 0.2.2
A generator for data integration app using pipebase framework21 versions - Latest release: about 4 years ago - 2 dependent packages - 32.3 thousand downloads total - 8 stars on GitHub - 1 maintainer
xvc-test-helper 0.7.0 💰
Test helper command for Xvc32 versions - Latest release: 3 months ago - 7 dependent packages - 30.9 thousand downloads total - 29 stars on GitHub - 1 maintainer
pipesqs 0.1.5
A pipebase plugin using aws sqs rust sdk1 version - Latest release: about 4 years ago - 1.83 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipederive 0.2.1
Proc macros for data integration app using pipebase framework7 versions - Latest release: about 4 years ago - 1 dependent package - 1 dependent repositories - 21.3 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipemysql 0.2.3
A pipebase plugin using mysql_async10 versions - Latest release: about 4 years ago - 12.7 thousand downloads total - 8 stars on GitHub - 1 maintainer
protoc-gen-fake 0.3.3
A protobuf compiler plugin to generate fake data based on protobuf schema definitions.5 versions - Latest release: 2 months ago - 82 downloads total - 1 maintainer
pipebytes 0.1.5
A pipebase plugin using byteorder6 versions - Latest release: over 4 years ago - 2 dependent packages - 9.44 thousand downloads total - 8 stars on GitHub - 1 maintainer
czv 0.0.2 💰
Rust library for performing CSV-related operations for data engineering and analysis.3 versions - Latest release: almost 2 years ago - 4.16 thousand downloads total - 1 stars on GitHub - 1 maintainer
pipedynamodb 0.1.5
A pipebase plugin using aws dynamodb rust sdk1 version - Latest release: about 4 years ago - 1.85 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipekafka 0.2.3
A pipebase plugin using rdkafka8 versions - Latest release: over 4 years ago - 10.3 thousand downloads total - 8 stars on GitHub - 1 maintainer
xvc-workflow-tests 0.6.6 💰
Integration test suite for Xvc14 versions - Latest release: about 2 years ago - 18.1 thousand downloads total - 29 stars on GitHub - 1 maintainer
pipereqwest 0.1.5
A pipebase plugin using reqwest6 versions - Latest release: over 4 years ago - 7.96 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipewarp 0.1.5
A pipebase plugin using warp6 versions - Latest release: over 4 years ago - 8.17 thousand downloads total - 9 stars on GitHub - 1 maintainer
pipemqtt 0.1.5
A pipebase plugin using rumqttc5 versions - Latest release: over 4 years ago - 6.77 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipejson 0.1.6
A pipebase plugin using json serde7 versions - Latest release: over 4 years ago - 9.27 thousand downloads total - 8 stars on GitHub - 1 maintainer
statespace 0.1.7
Statespace command-line interface4 versions - Latest release: 25 days ago - 36 downloads total - 820 stars on GitHub - 1 maintainer
statespace-templates 0.1.7
Shared templates (AGENTS.md, favicon, page.html) for Statespace servers3 versions - Latest release: 25 days ago - 37 downloads total - 820 stars on GitHub - 1 maintainer
statespace-server 0.1.7
Open-source AI tool execution runtime - HTTP server library4 versions - Latest release: 25 days ago - 52 downloads total - 820 stars on GitHub - 1 maintainer
statespace-tool-runtime 0.1.7
Core tool execution runtime for Statespace5 versions - Latest release: 25 days ago - 80 downloads total - 820 stars on GitHub - 1 maintainer
pipebuilder_mock 0.1.0
pipebuilder api mock server1 version - Latest release: about 4 years ago - 1.88 thousand downloads total - 1 stars on GitHub - 1 maintainer
qsv 16.1.0
A Blazing-Fast Data-wrangling toolkit.193 versions - Latest release: about 2 months ago - 242 thousand downloads total - 3,538 stars on GitHub - 1 maintainer
pipekube 0.1.5
A pipebase plugin using kubernetes rust sdk6 versions - Latest release: over 4 years ago - 7.98 thousand downloads total - 8 stars on GitHub - 1 maintainer
risingwave 0.1.0
A placeholder repo for RisingWave.1 version - Latest release: about 4 years ago - 1.62 thousand downloads total - 6,125 stars on GitHub - 1 maintainer
pipesns 0.1.5
A pipebase plugin using aws sns rust sdk1 version - Latest release: about 4 years ago - 1.86 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipeavro 0.1.4
A pipebase plugin using avro-rs5 versions - Latest release: over 4 years ago - 6.45 thousand downloads total - 8 stars on GitHub - 1 maintainer
xvc-file 0.7.0 💰
File tracking, versioning, upload and download functions for Xvc33 versions - Latest release: 3 months ago - 3 dependent packages - 1 dependent repositories - 33 thousand downloads total - 29 stars on GitHub - 1 maintainer
tl-llvm 0.3.1
LLVM backend for ThinkingLanguage (Phase 30)2 versions - Latest release: 28 days ago - 12 downloads total - 0 stars on GitHub - 1 maintainer
tl-package 0.3.1
Package manager for ThinkingLanguage3 versions - Latest release: 28 days ago - 43 downloads total - 0 stars on GitHub - 1 maintainer
tl-compiler 0.3.1
Bytecode compiler and VM for ThinkingLanguage (Phase 2)3 versions - Latest release: 28 days ago - 36 downloads total - 0 stars on GitHub - 1 maintainer
tl-lsp 0.3.1
Language Server Protocol implementation for ThinkingLanguage3 versions - Latest release: 28 days ago - 30 downloads total - 0 stars on GitHub - 1 maintainer
tl-gpu 0.3.1
GPU tensor compute for ThinkingLanguage via wgpu3 versions - Latest release: 28 days ago - 33 downloads total - 0 stars on GitHub - 1 maintainer
tl-interpreter 0.3.1
Tree-walking interpreter for ThinkingLanguage (Phase 0)3 versions - Latest release: 28 days ago - 29 downloads total - 0 stars on GitHub - 1 maintainer
thinkinglanguage 0.3.1
CLI entry point for ThinkingLanguage (tl run, tl shell)3 versions - Latest release: 28 days ago - 22 downloads total - 0 stars on GitHub - 1 maintainer
specds 0.1.0 removed
A spec-driven data science pipeline generator using LLMs1 version - Latest release: 9 months ago - 518 downloads total - 1 stars on GitHub - 1 maintainer
firelink 0.1.0
🔥 Efficiently linking CDC with Delta lake, all in Rust, just like God intended it.1 version - Latest release: over 2 years ago - 1.52 thousand downloads total - 1 maintainer
time_series_data_packer_rs 0.4.1
Time series data packer written in Rust language for data intensive IoT and IIoT projects.2 versions - Latest release: 4 months ago - 67 downloads total - 0 stars on GitHub - 1 maintainer
pipecsv 0.1.5
A pipebase plugin using csv serde6 versions - Latest release: over 4 years ago - 8.05 thousand downloads total - 8 stars on GitHub - 1 maintainer
xvc 0.7.0 💰
An MLOps tool to manage data files and pipelines on top of Git31 versions - Latest release: 3 months ago - 1 dependent package - 1 dependent repositories - 33.7 thousand downloads total - 29 stars on GitHub - 1 maintainer
pipecql 0.2.2
A pipebase plugin using scylla9 versions - Latest release: over 4 years ago - 11.6 thousand downloads total - 9 stars on GitHub - 1 maintainer
piperocksdb 0.1.5
A pipebase plugin using rocksdb rust wrapper6 versions - Latest release: over 4 years ago - 8.06 thousand downloads total - 9 stars on GitHub - 1 maintainer
Related Keywords
rust
29
declarative-pipeline
23
api-integration
23
pipebase
22
etl
21
ai
21
pipeline
20
pipeware
19
data-science
18
data
17
machine-learning
16
language
16
mlops
12
sql
12
git
11
file
11
devops
11
versioning
11
command-line-tool
11
machine-learning-engineering
11
data-pipelines
11
database
11
analytics
11
arrow
8
parquet
7
mcp
6
artificial-intelligence
5
time-series
5
stocks
4
stock-analysis
4
roaring-bitmaps
4
lakehouse
4
datafusion
4
data-lake
4
aws
4
webapp
4
markdown
4
polars
4
information-retrieval
4
context-engineering
4
agent
4
ci
3
kafka
3
postgresql
3
csv
3
cdc
3
async
3
continuous-integration
3
pandas
3
delta-lake
3
change-data-capture
3
apache-arrow
3
python
3
sap-hana
2
zero-copy
2
serde
2
tools
2
agents
2
pyo3
2
airflow
2
batch-processing
2
dag
2
orchestration
2
pipelines
2
scheduler
2
self-hosted
2
workflow
2
hana
2
sap
2
db-api
2
high-performance
2
performance
2
model-context-protocol
2
compliance
2
ckan
2
pipebuilder
2
metadata
2
data-visualization
2
data-analysis
2
data-quality
2
delta
2
dataframe
2
cli
2
postgres
2
streaming
2
iceberg
1
arrow-rs
1
apache-parquet
1
concurrency
1
geocode
1
opendata
1
data-wrangling
1
dcat
1
excel
1
rocksdb
1
gemini-api
1
harvester
1
json
1
mqtt
1
warp
1