conda-forge.org "data-engineering" keyword
awswrangler 2.17.0
An open-source Python package that extends the power of Pandas library to AWS connecting DataFram...81 versions - Latest release: over 3 years ago - 1 dependent repositories - 813 thousand downloads total - 4,070 stars on GitHub
Top 4.9% on conda-forge.org
133 versions - Latest release: over 3 years ago - 8 dependent packages - 41 dependent repositories - 1.1 million downloads total - 20,629 stars on GitHub
prefect 2.6.7
Prefect is a workflow management system, designed for modern infrastructure and powered by the op...133 versions - Latest release: over 3 years ago - 8 dependent packages - 41 dependent repositories - 1.1 million downloads total - 20,629 stars on GitHub
pyspark-test 0.2.0
Testing library for pyspark, inspired from pandas testing module but for pyspark, to help users w...2 versions - Latest release: over 4 years ago - 12.7 thousand downloads total - 20 stars on GitHub
xontrib-pipeliner 0.3.2 💰
Let your pipe lines flow thru the Python code in xonsh.5 versions - Latest release: over 5 years ago - 15.8 thousand downloads total - 60 stars on GitHub
Top 4.8% on conda-forge.org
42 versions - Latest release: over 3 years ago - 120 dependent packages - 3 dependent repositories - 1.94 million downloads total - 42,884 stars on GitHub
airflow 2.4.3
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...42 versions - Latest release: over 3 years ago - 120 dependent packages - 3 dependent repositories - 1.94 million downloads total - 42,884 stars on GitHub
Top 7.0% on conda-forge.org
4 versions - Latest release: over 3 years ago - 23 dependent packages - 1 dependent repositories - 253 thousand downloads total - 43,247 stars on GitHub
apache-airflow-providers-common-sql 1.3.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows4 versions - Latest release: over 3 years ago - 23 dependent packages - 1 dependent repositories - 253 thousand downloads total - 43,247 stars on GitHub
airflow-with-kerberos 2.4.3
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...25 versions - Latest release: over 3 years ago - 641 thousand downloads total - 42,884 stars on GitHub
airflow-with-jdbc 1.10.11
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...15 versions - Latest release: over 5 years ago - 131 thousand downloads total - 42,884 stars on GitHub
airflow-with-virtualenv 2.4.3
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...27 versions - Latest release: over 3 years ago - 681 thousand downloads total - 42,884 stars on GitHub
airflow-with-salesforce 1.10.15
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...19 versions - Latest release: about 5 years ago - 179 thousand downloads total - 42,884 stars on GitHub
airflow-with-cncf-kubernetes 2.4.3
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...10 versions - Latest release: over 3 years ago - 338 thousand downloads total - 42,884 stars on GitHub
airflow-with-sentry 2.4.3
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...27 versions - Latest release: over 3 years ago - 681 thousand downloads total - 42,884 stars on GitHub
airflow-with-cloudant 1.10.15
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...19 versions - Latest release: about 5 years ago - 179 thousand downloads total - 42,884 stars on GitHub
airflow-with-vertica 1.10.15
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...19 versions - Latest release: about 5 years ago - 180 thousand downloads total - 42,884 stars on GitHub
airflow-with-docker 1.10.11
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...15 versions - Latest release: over 5 years ago - 131 thousand downloads total - 42,884 stars on GitHub
airflow-with-dask 2.4.3
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...40 versions - Latest release: over 3 years ago - 612 thousand downloads total - 42,884 stars on GitHub
superset 2.0.0
Apache Superset is a Data Visualization and Data Exploration Platform12 versions - Latest release: over 3 years ago - 96.3 thousand downloads total - 58,575 stars on GitHub
redun 0.8.16
Yet another redundant workflow engine6 versions - Latest release: over 3 years ago - 34.2 thousand downloads total - 577 stars on GitHub
Top 5.2% on conda-forge.org
102 versions - Latest release: over 3 years ago - 19 dependent packages - 29 dependent repositories - 9,015 stars on GitHub
xonsh 0.13.3 💰
Xonsh is a Python-powered, cross-platform, Unix-gazing shell shell language and command prompt. T...102 versions - Latest release: over 3 years ago - 19 dependent packages - 29 dependent repositories - 9,015 stars on GitHub
apache-airflow-providers-jenkins 3.0.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows11 versions - Latest release: almost 4 years ago - 59.4 thousand downloads total - 44,599 stars on GitHub
airflow-with-kubernetes 1.10.15
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...18 versions - Latest release: about 5 years ago - 158 thousand downloads total - 42,529 stars on GitHub
pyjanitor 0.23.1
Clean APIs for data cleaning. Python implementation of R package Janitor51 versions - Latest release: over 3 years ago - 1 dependent package - 14 dependent repositories - 1,457 stars on GitHub
airflow-with-databricks 1.10.15
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...19 versions - Latest release: about 5 years ago - 42,783 stars on GitHub
apache-airflow-providers-imap 3.0.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows8 versions - Latest release: almost 4 years ago - 2 dependent packages - 1 dependent repositories - 230 thousand downloads total - 42,884 stars on GitHub
apache-airflow-providers-jira 3.0.1
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows9 versions - Latest release: over 3 years ago - 25.9 thousand downloads total - 42,884 stars on GitHub
apache-airflow-providers-apache-druid 3.2.1
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows14 versions - Latest release: over 3 years ago - 67.5 thousand downloads total - 42,884 stars on GitHub
r-uptasticsearch 0.4.0
An Elasticsearch client tailored to data science workflows.1 version - Latest release: almost 6 years ago - 39.6 thousand downloads total - 49 stars on GitHub
airflow-with-postgres 1.10.15
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...19 versions - Latest release: about 5 years ago - 185 thousand downloads total - 42,884 stars on GitHub
gspread-pandas 2.2.3
A package to easily open an instance of a Google spreadsheet and interact with worksheets through...3 versions - Latest release: almost 6 years ago - 404 stars on GitHub
apache-airflow-providers-singularity 3.0.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows7 versions - Latest release: almost 4 years ago - 42,884 stars on GitHub
sf-hamilton 1.11.0
A scalable general purpose micro-framework for defining dataflows. You can use it to build datafr...12 versions - Latest release: over 3 years ago - 688 stars on GitHub
ploomber-scaffold 0.3.1
The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️10 versions - Latest release: about 4 years ago - 1 dependent package - 29.3 thousand downloads total - 3,604 stars on GitHub
airflow-with-cassandra 1.10.15
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...18 versions - Latest release: about 5 years ago - 42,884 stars on GitHub
apache-airflow-providers-apache-pig 3.0.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows6 versions - Latest release: almost 4 years ago - 42,783 stars on GitHub
apache-airflow-providers-yandex 3.1.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows8 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
airflow-with-atlas 2.2.4
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...17 versions - Latest release: about 4 years ago - 42,884 stars on GitHub
airflow-with-azure_blob_storage 1.10.15
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...18 versions - Latest release: about 5 years ago - 42,884 stars on GitHub
airflow-with-crypto 1.10.15
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...19 versions - Latest release: about 5 years ago - 42,884 stars on GitHub
airflow-with-mysql 1.10.11
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...14 versions - Latest release: over 5 years ago - 42,884 stars on GitHub
ploomber 0.21.7
The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️60 versions - Latest release: over 3 years ago - 2 dependent repositories - 3,604 stars on GitHub
Top 8.9% on conda-forge.org
22 versions - Latest release: over 3 years ago - 70 dependent packages - 42,884 stars on GitHub
apache-airflow 2.4.3
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...22 versions - Latest release: over 3 years ago - 70 dependent packages - 42,884 stars on GitHub
Top 8.3% on conda-forge.org
10 versions - Latest release: over 3 years ago - 7 dependent packages - 1 dependent repositories - 42,884 stars on GitHub
apache-airflow-providers-http 4.0.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows10 versions - Latest release: over 3 years ago - 7 dependent packages - 1 dependent repositories - 42,884 stars on GitHub
dagster-graphql 1.0.17
Dagster is a system for building modern data applications. Combining an elegant programming model...114 versions - Latest release: over 3 years ago - 6 dependent packages - 14,215 stars on GitHub
apache-airflow-providers-apache-hdfs 3.1.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows10 versions - Latest release: over 3 years ago - 2 dependent packages - 42,884 stars on GitHub
apache-airflow-providers-ftp 3.1.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows9 versions - Latest release: over 3 years ago - 2 dependent packages - 1 dependent repositories - 42,884 stars on GitHub
great-expectations 0.15.32
Great Expectations helps teams save time and promote analytic integrity by offering a unique appr...144 versions - Latest release: over 3 years ago - 2 dependent packages - 1 dependent repositories - 10,884 stars on GitHub
dagster-snowflake 1.0.17
Dagster is a system for building modern data applications. Combining an elegant programming model...108 versions - Latest release: over 3 years ago - 1 dependent package - 14,215 stars on GitHub
apache-airflow-providers-postgres 5.2.2
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows15 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
airflow-with-apache-beam 2.2.4
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...12 versions - Latest release: about 4 years ago - 42,884 stars on GitHub
apache-airflow-providers-salesforce 5.1.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows13 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
apache-airflow-providers-qubole 3.2.1
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows11 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
apache-airflow-providers-presto 4.0.1
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows12 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
airflow-with-ssh 1.10.15
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...19 versions - Latest release: about 5 years ago - 42,884 stars on GitHub
apache-airflow-providers-grpc 3.0.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows7 versions - Latest release: almost 4 years ago - 42,884 stars on GitHub
airflow-with-winrm 1.10.15
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...18 versions - Latest release: about 5 years ago - 42,884 stars on GitHub
apache-airflow-providers-papermill 3.0.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows9 versions - Latest release: almost 4 years ago - 42,884 stars on GitHub
apache-airflow-providers-hashicorp 3.1.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows11 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
apache-airflow-providers-segment 3.0.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows6 versions - Latest release: almost 4 years ago - 42,884 stars on GitHub
airflow-with-statsd 2.4.3
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...41 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
apache-airflow-providers-microsoft-azure 4.0.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows16 versions - Latest release: almost 4 years ago - 42,884 stars on GitHub
apache-airflow-providers-slack 6.0.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows11 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
apache-airflow-providers-asana 2.0.1
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows3 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
apache-airflow-providers-apache-beam 4.0.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows10 versions - Latest release: almost 4 years ago - 42,884 stars on GitHub
dagster-ge 1.0.17
Dagster is a system for building modern data applications. Combining an elegant programming model...103 versions - Latest release: over 3 years ago - 14,215 stars on GitHub
airflow-with-celery 1.10.15
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...19 versions - Latest release: about 5 years ago - 42,884 stars on GitHub
airflow-with-hashicorp 1.10.15
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...5 versions - Latest release: about 5 years ago - 42,884 stars on GitHub
apache-airflow-providers-celery 3.0.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows7 versions - Latest release: almost 4 years ago - 42,783 stars on GitHub
apache-airflow-providers-sendgrid 3.0.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows7 versions - Latest release: almost 4 years ago - 42,884 stars on GitHub
apache-airflow-providers-amazon 6.0.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows18 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
apache-airflow-providers-elasticsearch 4.2.1
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows16 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
apache-airflow-providers-microsoft-mssql 3.2.1
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows11 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
apache-airflow-providers-dingding 3.0.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows7 versions - Latest release: almost 4 years ago - 42,884 stars on GitHub
apache-airflow-providers-zendesk 4.0.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows7 versions - Latest release: almost 4 years ago - 42,884 stars on GitHub
airflow-with-elasticsearch 1.10.11
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...14 versions - Latest release: over 5 years ago - 42,884 stars on GitHub
apache-airflow-providers-jdbc 3.2.1
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows10 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
dagstermill 1.0.17
Dagster is a system for building modern data applications. Combining an elegant programming model...112 versions - Latest release: over 3 years ago - 14,300 stars on GitHub
apache-airflow-providers-neo4j 3.1.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows10 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
apache-airflow-providers-alibaba 2.1.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows4 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
dagster-papertrail 1.0.17
Dagster is a system for building modern data applications. Combining an elegant programming model...114 versions - Latest release: over 3 years ago - 14,215 stars on GitHub
dagster-docker 1.0.17
Dagster is a system for building modern data applications. Combining an elegant programming model...86 versions - Latest release: over 3 years ago - 14,300 stars on GitHub
dagster_dask 0.6.5
An orchestration platform for the development, production, and observation of data assets.1 version - Latest release: over 6 years ago - 14,215 stars on GitHub
apache-airflow-providers-airbyte 3.1.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows3 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
apache-airflow-providers-github 2.1.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows3 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
airflow-with-sendgrid 1.10.15
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...18 versions - Latest release: about 5 years ago - 42,884 stars on GitHub
airflow-with-jira 1.10.15
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...19 versions - Latest release: about 5 years ago - 42,884 stars on GitHub
airflow-with-azure-mgmt-containerinstance 1.10.11
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...11 versions - Latest release: over 5 years ago - 42,884 stars on GitHub
airflow-with-leveldb 2.4.3
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...19 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
apache-airflow-providers-apache-livy 3.1.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows9 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
apache-airflow-providers-dbt-cloud 2.2.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows5 versions - Latest release: over 3 years ago - 43,662 stars on GitHub
dagster-twilio 1.0.17
Dagster is a system for building modern data applications. Combining an elegant programming model...114 versions - Latest release: over 3 years ago - 14,215 stars on GitHub
airflow-with-azure_cosmos 1.10.15
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...16 versions - Latest release: about 5 years ago - 42,884 stars on GitHub
dagster-cron 0.11.15
An orchestration platform for the development, production, and observation of data assets.44 versions - Latest release: over 4 years ago - 14,215 stars on GitHub
apache-airflow-providers-cncf-kubernetes 4.4.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows20 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
airflow-with-samba 1.10.15
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...19 versions - Latest release: about 5 years ago - 42,884 stars on GitHub
dagster_pagerduty 0.6.4
An orchestration platform for the development, production, and observation of data assets.1 version - Latest release: over 6 years ago - 14,215 stars on GitHub
apache-airflow-providers-apache-spark 3.0.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows10 versions - Latest release: almost 4 years ago - 42,884 stars on GitHub
apache-airflow-providers-mongo 3.0.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows8 versions - Latest release: almost 4 years ago - 42,884 stars on GitHub
apache-airflow-providers-oracle 3.4.0
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows13 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
airflow-with-jenkins 1.10.15
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...18 versions - Latest release: about 5 years ago - 42,884 stars on GitHub
airflow-with-password 2.4.3
Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler...41 versions - Latest release: over 3 years ago - 42,884 stars on GitHub
Related Keywords
python
197
data-science
196
etl
188
mlops
186
workflow
185
orchestration
184
data-integration
183
data-pipelines
183
scheduler
182
data-orchestrator
182
machine-learning
126
workflow-engine
123
elt
122
dag
122
apache
122
automation
122
workflow-orchestration
121
apache-airflow
121
airflow
121
analytics
62
hacktoberfest
62
dagster
61
metadata
61
workflow-automation
61
data
7
pipeline
4
pipelines
4
pandas
4
dataframe
3
data-quality
3
data-versioning
2
dataquality
2
ml
2
artificial-intelligence
2
aws
2
sql
2
data-version-control
2
data-analytics
2
data-analysis
2
mysql
2
parquet
2
xonsh
2
shell
2
serialization
2
dbt
2
vscode
2
spark
2
pycharm
2
jupyter
2
papermill
2
jupyter-notebooks
2
notebooks
2
transformation
1
data-unit-tests
1
data-profiling
1
data-profilers
1
cleandata
1
stitch-fix
1
software-engineering
1
numpy
1
lineage
1
hamiltonian
1
hamilton
1
featurization
1
feature-engineering
1
etl-pipeline
1
etl-framework
1
data-platform
1
reverse-etl
1
features
1
feature-store
1
big-data
1
trino
1
snowflake
1
rdbms
1
postgresql
1
postgres
1
oracle-database
1
dataengineering
1
databricks-sql
1
database
1
data-quality-monitoring
1
data-diffing
1
zarr
1
xarray
1
cloud
1
pipeline-tests
1
pipeline-testing
1
pipeline-debt
1
exploratorydataanalysis
1
exploratory-data-analysis
1
exploratory-analysis
1
eda
1
dataunittest
1
datacleaning
1
datacleaner
1
data-viz
1
data-visualization
1
business-intelligence
1
business-analytics
1