crates.io "data-engineering" keyword
View the packages on the crates.io package registry that are tagged with the "data-engineering" keyword.
qsv 6.0.1
A Blazing-Fast Data-wrangling toolkit.177 versions - Latest release: 13 days ago - 211 thousand downloads total - 3,020 stars on GitHub - 1 maintainer
padder 2.0.0
A highly efficient Rust crate for padding data during runtime.6 versions - Latest release: 2 days ago - 7.49 thousand downloads total - 2 stars on GitHub - 1 maintainer
elusion 3.12.4
Elusion is a modern DataFrame / Data Engineering / Data Analysis library that combines the famili...68 versions - Latest release: 4 days ago - 39.7 thousand downloads total - 80 stars on GitHub - 1 maintainer
specds 0.1.0
A spec-driven data science pipeline generator using LLMs1 version - Latest release: 6 days ago - 127 downloads total - 1 stars on GitHub - 1 maintainer
thepipelinetool 0.2.7
readers used for thepipelinetool18 versions - Latest release: over 1 year ago - 1 dependent package - 18.4 thousand downloads total - 18 stars on GitHub - 1 maintainer
pipederive 0.2.1
Proc macros for data integration app using pipebase framework7 versions - Latest release: over 3 years ago - 1 dependent package - 1 dependent repositories - 18.3 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipekube 0.1.5
A pipebase plugin using kubernetes rust sdk6 versions - Latest release: over 3 years ago - 7.08 thousand downloads total - 8 stars on GitHub - 1 maintainer
magnetise 0.1.2
A Rust library to asses the similarity between SQL queries.3 versions - Latest release: 11 months ago - 2.68 thousand downloads total - 2 stars on GitHub - 1 maintainer
cargo-pipe 0.2.2
A command-line tool to generate and build data integration app using pipebase framework22 versions - Latest release: over 3 years ago - 24.7 thousand downloads total - 8 stars on GitHub - 1 maintainer
dink 0.1.0
Threaded CDC to Delta lake, written in Rust, just like God intended it.1 version - Latest release: over 1 year ago - 1.3 thousand downloads total - 1 maintainer
pipemqtt 0.1.5
A pipebase plugin using rumqttc5 versions - Latest release: over 3 years ago - 6.01 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipecsv 0.1.5
A pipebase plugin using csv serde6 versions - Latest release: over 3 years ago - 7.15 thousand downloads total - 8 stars on GitHub - 1 maintainer
czv 0.0.2 💰
Rust library for performing CSV-related operations for data engineering and analysis.3 versions - Latest release: about 1 year ago - 3.47 thousand downloads total - 0 stars on GitHub - 1 maintainer
pipecql 0.2.2
A pipebase plugin using scylla9 versions - Latest release: over 3 years ago - 10.3 thousand downloads total - 9 stars on GitHub - 1 maintainer
pipewarp 0.1.5
A pipebase plugin using warp6 versions - Latest release: over 3 years ago - 7.26 thousand downloads total - 9 stars on GitHub - 1 maintainer
pipekafka 0.2.3
A pipebase plugin using rdkafka8 versions - Latest release: over 3 years ago - 9.17 thousand downloads total - 8 stars on GitHub - 1 maintainer
piperocksdb 0.1.5
A pipebase plugin using rocksdb rust wrapper6 versions - Latest release: over 3 years ago - 7.16 thousand downloads total - 9 stars on GitHub - 1 maintainer
piperedis 0.1.6
A pipebase plugin using redis rust client7 versions - Latest release: over 3 years ago - 8.21 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipebase 0.2.0
A tokio based runtime library for data integration app13 versions - Latest release: over 3 years ago - 19 dependent packages - 24.2 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipesqs 0.1.5
A pipebase plugin using aws sqs rust sdk1 version - Latest release: over 3 years ago - 1.6 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipegen 0.2.2
A generator for data integration app using pipebase framework21 versions - Latest release: over 3 years ago - 2 dependent packages - 28.7 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipepsql 0.2.2
A pipebase plugin using tokio_postgres9 versions - Latest release: over 3 years ago - 10.3 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipeavro 0.1.4
A pipebase plugin using avro-rs5 versions - Latest release: over 3 years ago - 5.67 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipereqwest 0.1.5
A pipebase plugin using reqwest6 versions - Latest release: over 3 years ago - 7.05 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipemysql 0.2.3
A pipebase plugin using mysql_async10 versions - Latest release: over 3 years ago - 11.4 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipejson 0.1.6
A pipebase plugin using json serde7 versions - Latest release: over 3 years ago - 8.25 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipedynamodb 0.1.5
A pipebase plugin using aws dynamodb rust sdk1 version - Latest release: over 3 years ago - 1.62 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipesns 0.1.5
A pipebase plugin using aws sns rust sdk1 version - Latest release: over 3 years ago - 1.63 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipes3 0.1.5
A pipebase plugin using aws s3 rust sdk1 version - Latest release: over 3 years ago - 1.58 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipebytes 0.1.5
A pipebase plugin using byteorder6 versions - Latest release: over 3 years ago - 2 dependent packages - 8.28 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipeamqp 0.1.3
A pipebase plugin using lapin4 versions - Latest release: over 3 years ago - 5 thousand downloads total - 8 stars on GitHub - 1 maintainer
pipebuilder_mock 0.1.0
pipebuilder api mock server1 version - Latest release: over 3 years ago - 1.65 thousand downloads total - 1 stars on GitHub - 1 maintainer
datamixer 0.0.1
Datamixer client library (upcoming)1 version - Latest release: over 2 years ago - 1.27 thousand downloads total - 0 stars on GitHub - 1 maintainer
simplygo 0.2.0
Simplygo SDK for Rust.2 versions - Latest release: about 1 year ago - 3.3 thousand downloads total - 11 stars on GitHub - 1 maintainer
thepipelinetool_core 0.2.7
An *experimental* pipeline orchestration tool drawing on concepts from Apache Airflow7 versions - Latest release: over 1 year ago - 1 dependent package - 7.29 thousand downloads total - 30 stars on GitHub - 1 maintainer
pipebuilder_common 0.2.2
lib for pipebuilder components12 versions - Latest release: over 3 years ago - 3 dependent packages - 16.1 thousand downloads total - 1 stars on GitHub - 1 maintainer
evolution 1.3.0
Efficiently evolve your old fixed-length data files into modern file formats.4 versions - Latest release: 10 months ago - 3.87 thousand downloads total - 6 stars on GitHub - 1 maintainer
firelink 0.1.0
🔥 Efficiently linking CDC with Delta lake, all in Rust, just like God intended it.1 version - Latest release: over 1 year ago - 1.28 thousand downloads total - 1 maintainer
fusiondata 0.1.0
Rust libraries of The fusiondata1 version - Latest release: 11 months ago - 1 thousand downloads total - 9 stars on GitHub - 1 maintainer
rusty-logging 0.5.0 💰
Logging for OpsML2 versions - Latest release: 4 months ago - 7.56 thousand downloads total - 11 stars on GitHub - 1 maintainer
pipebuilder 0.2.2
CI for pipebase application12 versions - Latest release: over 3 years ago - 12.6 thousand downloads total - 1 stars on GitHub - 1 maintainer
xvc-config 0.6.17 💰
Xvc configuration management29 versions - Latest release: 3 months ago - 6 dependent packages - 1 dependent repositories - 29.1 thousand downloads total - 29 stars on GitHub - 1 maintainer
xvc-pipeline 0.6.17 💰
Xvc data pipeline management27 versions - Latest release: 3 months ago - 2 dependent packages - 1 dependent repositories - 25.3 thousand downloads total - 29 stars on GitHub - 1 maintainer
xvc-walker 0.6.17 💰
Xvc parallel file system walker with ignore features32 versions - Latest release: 3 months ago - 7 dependent packages - 1 dependent repositories - 32.5 thousand downloads total - 29 stars on GitHub - 1 maintainer
xvc-workflow-tests 0.6.6 💰
Integration test suite for Xvc14 versions - Latest release: over 1 year ago - 15.4 thousand downloads total - 29 stars on GitHub - 1 maintainer
xvc-ecs 0.6.17 💰
Entity-Component System for Xvc28 versions - Latest release: 3 months ago - 6 dependent packages - 1 dependent repositories - 27.6 thousand downloads total - 29 stars on GitHub - 1 maintainer
xvc-core 0.6.17 💰
Xvc core for common elements for all commands30 versions - Latest release: 3 months ago - 5 dependent packages - 1 dependent repositories - 28.8 thousand downloads total - 29 stars on GitHub - 1 maintainer
xvc-storage 0.6.17 💰
Xvc remote and local storage management29 versions - Latest release: 3 months ago - 3 dependent packages - 1 dependent repositories - 27.9 thousand downloads total - 29 stars on GitHub - 1 maintainer
xvc-file 0.6.17 💰
File tracking, versioning, upload and download functions for Xvc29 versions - Latest release: 3 months ago - 3 dependent packages - 1 dependent repositories - 27.7 thousand downloads total - 29 stars on GitHub - 1 maintainer
xvc-test-helper 0.6.17 💰
Test helper command for Xvc28 versions - Latest release: 3 months ago - 7 dependent packages - 25.6 thousand downloads total - 29 stars on GitHub - 1 maintainer
xvc 0.6.17 💰
An MLOps tool to manage data files and pipelines on top of Git27 versions - Latest release: 3 months ago - 1 dependent package - 1 dependent repositories - 27.4 thousand downloads total - 29 stars on GitHub - 1 maintainer
xvc-logging 0.6.17 💰
Logging crate for Xvc27 versions - Latest release: 3 months ago - 10 dependent packages - 1 dependent repositories - 30 thousand downloads total - 29 stars on GitHub - 1 maintainer
risingwave 0.1.0
A placeholder repo for RisingWave.1 version - Latest release: over 3 years ago - 1.28 thousand downloads total - 6,125 stars on GitHub - 1 maintainer
Related Keywords
declarative-pipeline
23
api-integration
23
pipebase
22
pipeware
19
rust
17
data
15
data-science
14
machine-learning
12
mlops
12
command-line-tool
11
data-pipelines
11
machine-learning-engineering
11
versioning
11
git
11
file
11
devops
11
etl
5
sql
4
aws
4
scheduler
3
pipeline
3
delta-lake
3
kafka
3
ci
3
continuous-integration
3
postgresql
3
csv
3
serde
2
python
2
cdc
2
change-data-capture
2
postgres
2
delta
2
pipebuilder
2
parquet
2
polars
2
workflow
2
self-hosted
2
dataframe
2
analytics
2
data-analysis
2
data-visualization
2
airflow
2
batch-processing
2
dag
2
orchestration
2
pipelines
2
data-governance
1
spark-streaming
1
arrow-rs
1
streaming
1
apache-parquet
1
iceberg
1
apache-arrow
1
arrow
1
concurrency
1
web-scraping
1
superset
1
prefect
1
pandas
1
duckdb
1
stream-processing
1
dbt
1
data-pipeline
1
dashboard
1
database
1
big-data
1
cloud-native
1
distributed-database
1
flink
1
ksqldb
1
materialized-view
1
python3
1
postgresql-database
1
real-time
1
open-source
1
observability
1
monitoring
1
model-deployment
1
serverless
1
spark
1
governance
1
deep-learning
1
artificial-intelligence
1
ai
1
visualization
1
azure
1
library
1
mqtt
1
similarity
1
k8s
1
integration
1
declarative
1
codegen
1
engineering
1
analysis
1
formatting
1
timeseries
1
statistics
1
sqlite
1