An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

pypi.org "apache-spark" keyword

Top 0.2% on pypi.org
mlflow 3.17.1
MLflow is an open source platform for the complete machine learning lifecycle
191 versions - Latest release: 8 days ago - 360 dependent packages - 5,089 dependent repositories - 19.9 million downloads last month - 28,439 stars on GitHub - 1 maintainer
odctl 1.1.0
Orchestrator for the Open Data Stack
26 versions - Latest release: 5 days ago - 1.4 thousand downloads last month - 13 stars on GitHub - 1 maintainer
Top 3.3% on pypi.org
synapseml 1.1.3
Synapse Machine Learning
31 versions - Latest release: 7 months ago - 2 dependent packages - 3 dependent repositories - 1.16 million downloads last month - 5,248 stars on GitHub - 1 maintainer
dsgrid-toolkit 0.4.0
Python API for accessing demand-side grid model (dsgrid) datasets
7 versions - Latest release: 7 months ago - 77 downloads last month - 34 stars on GitHub - 1 maintainer
lakefs 0.16.0
lakeFS Python SDK Wrapper
32 versions - Latest release: 6 months ago - 1 dependent package - 400 thousand downloads last month - 5,556 stars on GitHub - 1 maintainer
lakefs_sdk_async 1.53.0
lakeFS API
1 version - Latest release: about 1 year ago - 10 downloads last month - 5,556 stars on GitHub - 1 maintainer
Top 6.3% on pypi.org
lakefs-sdk 1.88.0
lakeFS API
119 versions - Latest release: 20 days ago - 3 dependent packages - 1 dependent repositories - 507 thousand downloads last month - 5,556 stars on GitHub - 1 maintainer
Top 3.2% on pypi.org
lakefs-client 1.44.0
[legacy] lakeFS API
178 versions - Latest release: almost 2 years ago - 4 dependent packages - 5 dependent repositories - 7.6 thousand downloads last month - 5,556 stars on GitHub - 1 maintainer
mlflow-devlibx 1.22.8
MLflow: A Platform for ML Development and Productionization
9 versions - Latest release: over 4 years ago - 1 dependent repositories - 39 downloads last month - 28,439 stars on GitHub - 1 maintainer
mlflow-tmp 2.2.26
MLflow: A Platform for ML Development and Productionization
25 versions - Latest release: over 3 years ago - 40 downloads last month - 28,439 stars on GitHub - 1 maintainer
mlflow-by-ckl 2.81.0
MLflow: A Platform for ML Development and Productionization
44 versions - Latest release: over 2 years ago - 21 downloads last month - 28,439 stars on GitHub - 2 maintainers
mlflow-stonewise 1.30.1
MLflow: A Platform for ML Development and Productionization
1 version - Latest release: almost 4 years ago - 12 downloads last month - 28,439 stars on GitHub - 1 maintainer
mlflow-by-johnsnowlabs-v2 2.44.0
MLflow: A Platform for ML Development and Productionization
9 versions - Latest release: about 3 years ago - 28 downloads last month - 28,439 stars on GitHub - 1 maintainer
mlflow-no-ssl 2.16.3
MLflow is an open source platform for the complete machine learning lifecycle
1 version - Latest release: about 2 years ago - 11 downloads last month - 28,439 stars on GitHub - 1 maintainer
mlflow-by-johnsnowlabs 2.40.0
MLflow is an open source platform for the complete machine learning lifecycle
41 versions - Latest release: about 3 years ago - 367 downloads last month - 28,439 stars on GitHub - 3 maintainers
qubole-ml 1.9.1
MLflow: An ML Workflow Tool
2 versions - Latest release: about 6 years ago - 1 dependent repositories - 6 downloads last month - 28,439 stars on GitHub - 1 maintainer
lmcmlflow 1.17.1
MLflow: A Platform for ML Development and Productionization
3 versions - Latest release: over 5 years ago - 1 dependent repositories - 5 downloads last month - 28,439 stars on GitHub - 1 maintainer
mlflow-pri 3.4.1.post18
MLflow is an open source platform for the complete machine learning lifecycle
23 versions - Latest release: 15 days ago - 273 downloads last month - 28,439 stars on GitHub - 2 maintainers
mlflow-ste 1.10.1.dev0
MLflow: An ML Workflow Tool
1 version - Latest release: about 6 years ago - 1 dependent repositories - 10 downloads last month - 28,439 stars on GitHub - 1 maintainer
Top 0.6% on pypi.org
mlflow-skinny 3.17.1
MLflow is an open source platform for the complete machine learning lifecycle
151 versions - Latest release: 8 days ago - 49 dependent packages - 70 dependent repositories - 25 million downloads last month - 28,439 stars on GitHub - 1 maintainer
mlflow-tracing 3.17.1
MLflow Tracing SDK is an open-source, lightweight Python package that only includes the minimum s...
46 versions - Latest release: 8 days ago - 14.8 million downloads last month - 28,439 stars on GitHub - 1 maintainer
Top 1.9% on pypi.org
graphframes 0.6
GraphFrames: DataFrame-based Graphs
1 version - Latest release: almost 8 years ago - 3 dependent packages - 36 dependent repositories - 1.65 million downloads last month - 1,211 stars on GitHub - 1 maintainer
graphframes-py 0.12.3
GraphFrames: Graph Processing Framework for Apache Spark
12 versions - Latest release: 15 days ago - 1.45 million downloads last month - 1,211 stars on GitHub - 3 maintainers
graphframes-latest 0.8.3
GraphFrames: DataFrame-based Graphs
1 version - Latest release: about 3 years ago - 7.68 thousand downloads last month - 1,211 stars on GitHub - 1 maintainer
jupyterlab-pipelines 0.1.0
Export Jupyter notebooks to Airflow DAGs, dbt models, and Spark jobs on ilum
1 version - Latest release: 4 months ago - 11 downloads last month - 0 stars on GitHub - 1 maintainer
kubeflow-spark-api 2.4.0
Kubeflow Spark Operator API models for Kubernetes resources to interact with Kubeflow APIs.
2 versions - Latest release: 8 months ago - 2 thousand downloads last month - 3,154 stars on GitHub - 1 maintainer
pybda 0.1.0
Analysis of big biological data sets for distributed HPC clusters.
6 versions - Latest release: about 7 years ago - 1 dependent repositories - 46 downloads last month - 9 stars on GitHub - 1 maintainer
adbc-driver-spark 0.3.0
ADBC (Arrow Database Connectivity) driver for Apache Spark Connect
3 versions - Latest release: 3 months ago - 1 stars on GitHub - 1 maintainer
mcp-apache-spark-history-server 1.0.2
Model Context Protocol (MCP) server for Apache Spark History Server with job comparison and analy...
12 versions - Latest release: 4 days ago - 654 downloads last month - 205 stars on GitHub - 3 maintainers
khaos-cli 0.7.1
Kafka data generator and load testing tool - generate fake messages, simulate producers and consu...
14 versions - Latest release: 10 months ago - 43 downloads last month - 164 stars on GitHub - 1 maintainer
pysail 0.7.2
Sail Python library
46 versions - Latest release: 22 days ago - 31.3 thousand downloads last month - 3,454 stars on GitHub - 1 maintainer
pyspark-antipattern 0.4.1
A static analysis linter for PySpark — catches performance antipatterns before they reach your cl...
15 versions - Latest release: 6 months ago - 161 downloads last month - 32 stars on GitHub - 1 maintainer
spark-advisor-parser 0.1.21
Spark event log parser with compression support (.gz, .lz4, .snappy, .zstd)
7 versions - Latest release: 7 months ago - 147 downloads last month - 1 maintainer
spark-advisor-storage-connector 0.1.19
Storage connector for reading Spark event logs from HDFS, S3, and GCS
5 versions - Latest release: 7 months ago - 27 downloads last month - 1 maintainer
spark-advisor-mcp 0.1.21
MCP server exposing spark-advisor tools for Claude Desktop and other MCP clients
15 versions - Latest release: 7 months ago - 137 downloads last month - 1 maintainer
spark-advisor-rules 0.1.21
Deterministic rules engine for Spark job analysis
15 versions - Latest release: 7 months ago - 62 downloads last month - 0 stars on GitHub - 1 maintainer
spark-advisor-hs-connector 0.1.21
Spark History Server connector - polls and publishes jobs to NATS
15 versions - Latest release: 7 months ago - 141 downloads last month - 1 maintainer
spark-advisor-analyzer 0.1.21
AI-powered Spark job analyzer - NATS worker service
15 versions - Latest release: 7 months ago - 87 downloads last month - 0 stars on GitHub - 1 maintainer
spark-advisor-models 0.1.21
Shared Pydantic contracts for spark-advisor microservices
15 versions - Latest release: 7 months ago - 1.35 thousand downloads last month - 1 maintainer
spark-advisor-cli 0.1.21
AI-powered Apache Spark job analyzer and configuration advisor
18 versions - Latest release: 7 months ago - 35 downloads last month - 1 maintainer
Top 6.8% on pypi.org
dataproc-templates 0.0.2
Dataproc templates written in Python
2 versions - Latest release: almost 4 years ago - 154 stars on GitHub
google-dataproc-templates 0.1.0
Google Dataproc templates written in Python
13 versions - Latest release: over 3 years ago - 493 downloads last month - 154 stars on GitHub - 2 maintainers
Top 7.5% on pypi.org
feathr 1.0.0
An Enterprise-Grade, High Performance Feature Store
22 versions - Latest release: over 3 years ago - 1 dependent repositories - 202 downloads last month - 1,945 stars on GitHub - 1 maintainer
dagster-slurm-spark 1.3.1
dagster integration with slurm and spark
1 version - Latest release: about 1 year ago - 7 downloads last month - 63 stars on GitHub - 1 maintainer
laktory 0.13.0
An ETL and DataOps framework for building a lakehouse
134 versions - Latest release: 23 days ago - 4.08 thousand downloads last month - 57 stars on GitHub - 1 maintainer
quantum-pipeline 2.4.0
Framework for running, monitoring and analysing quantum algorithms.
16 versions - Latest release: 8 days ago - 64 downloads last month - 11 stars on GitHub - 1 maintainer
delta-explain 0.7.0
Make Delta Lake pruning visible: partition pruning and data skipping diagnostics, as a CLI and a ...
4 versions - Latest release: about 2 months ago - 135 downloads last month - 10 stars on GitHub - 1 maintainer
Top 6.9% on pypi.org
sparkmeasure 0.28.0
Python API for sparkMeasure, a tool for performance troubleshooting of Apache Spark workloads.
16 versions - Latest release: 5 months ago - 1 dependent repositories - 2.49 million downloads last month - 830 stars on GitHub - 1 maintainer
convalesce-emit 1.0.0
Forward a data tool's own output to Convalesce, unchanged.
11 versions - Latest release: 9 days ago - 1.01 thousand downloads last month - 0 stars on GitHub - 1 maintainer
Top 5.0% on pypi.org
sparkit-learn 0.2.6
Scikit-learn on PySpark
5 versions - Latest release: over 11 years ago - 5 dependent repositories - 687 downloads last month - 1,150 stars on GitHub - 3 maintainers
Top 3.1% on pypi.org
spark-sklearn 0.3.0
Integration tools for running scikit-learn on Spark
8 versions - Latest release: over 7 years ago - 14 dependent repositories - 152 thousand downloads last month - 1,072 stars on GitHub - 2 maintainers
pyspark-streaming-base 0.2.1
This project provides a set of base classes that simplify the art of crafting bullet-proof Spark ...
6 versions - Latest release: 7 months ago - 30 downloads last month - 5 stars on GitHub - 1 maintainer
pydataio 1.1.1
A scalable framework for data input and output operations in Spark applications
5 versions - Latest release: 11 months ago - 24 downloads last month - 3 stars on GitHub - 1 maintainer
pyspark-analyzer 5.0.2
A comprehensive PySpark DataFrame profiler for generating detailed statistics and data quality re...
21 versions - Latest release: over 1 year ago - 23 downloads last month - 2 stars on GitHub - 1 maintainer
cleanflow 1.3.3a1
A a framework for cleaning, pre-processing and exploring data in a scalable and distributed manner.
11 versions - Latest release: over 8 years ago - 33 downloads last month - 1 stars on GitHub - 1 maintainer
Top 3.7% on pypi.org
pyspark-stubs 3.0.0 💰
A collection of the Apache Spark stub files
38 versions - Latest release: over 6 years ago - 2 dependent packages - 146 dependent repositories - 116 thousand downloads last month - 118 stars on GitHub - 1 maintainer
eleflow-spark-integrations 0.0.1
The easy and quickly way to connect and integrate the Spark project with many others data sources.
2 versions - Latest release: over 4 years ago - 6 stars on GitHub
spark-connect-proxy 0.0.14 💰
A reverse proxy server which allows secure connectivity to a Spark Connect server
10 versions - Latest release: about 1 year ago - 56 downloads last month - 16 stars on GitHub - 1 maintainer
jsoniq 2.1.8
Python edition of RumbleDB, a JSONiq engine
32 versions - Latest release: 4 months ago - 246 downloads last month - 10 stars on GitHub - 1 maintainer
sparkql 0.10.0
sparkql: Apache Spark SQL DataFrame schema management for sensible humans
20 versions - Latest release: about 3 years ago - 1 dependent repositories - 7.18 thousand downloads last month - 13 stars on GitHub - 1 maintainer
miss-lightgbm-mmlspark 1.0.0
Microsoft ML for Spark
1 version - Latest release: over 2 years ago - 103 downloads last month - 5,247 stars on GitHub - 1 maintainer
sparkdq 0.12.1
A declarative PySpark framework for row- and aggregate-level data quality validation.
24 versions - Latest release: 4 months ago - 1.88 thousand downloads last month - 78 stars on GitHub - 1 maintainer
pyjaws 0.1.7
PyJaws: A Pythonic Way to Define Databricks Jobs and Workflows
10 versions - Latest release: about 3 years ago - 1 dependent repositories - 26 downloads last month - 45 stars on GitHub - 1 maintainer
pyspark3d 0.3.1
Spark extension for processing large-scale 3D data sets
4 versions - Latest release: almost 8 years ago - 1 dependent repositories - 94 downloads last month - 33 stars on GitHub - 1 maintainer
parallel-simulations 0.0.1
Helper class to orchestrate in parallel Monte Carlo simulations for an arbitrary number of models...
1 version - Latest release: over 2 years ago - 8 downloads last month - 0 stars on GitHub - 1 maintainer
pypair 3.0.9 💰
Pairwise association measures of statistical variable types
11 versions - Latest release: about 5 years ago - 1 dependent repositories - 277 downloads last month - 22 stars on GitHub - 1 maintainer
fasttrackml 0.6.0
An experiment tracking server focused on speed and scalability
25 versions - Latest release: over 2 years ago - 4.21 thousand downloads last month - 119 stars on GitHub - 1 maintainer
dead-salmon-brain 0.0.7
Dead Salmon Brain is a cluster computing system for analysing A/B experiments
7 versions - Latest release: over 4 years ago - 1 dependent repositories - 382 downloads last month - 15 stars on GitHub - 1 maintainer
Top 7.6% on pypi.org
sparktorch 0.2.0
Distributed training of PyTorch networks on Apache Spark with ML Pipeline support
11 versions - Latest release: over 3 years ago - 2 dependent repositories - 303 downloads last month - 339 stars on GitHub - 1 maintainer
mlflowcollab 0.0.4
Gebruik MLFlow op een centrale locatie
1 version - Latest release: over 4 years ago - 1 dependent repositories - 8 downloads last month - 2 stars on gitlab.com - 1 maintainer
pyspark-asyncactions 0.0.4 💰
A proof of concept asynchronous actions for PySpark using concurent.futures
4 versions - Latest release: about 6 years ago - 1 dependent repositories - 3.85 thousand downloads last month - 46 stars on GitHub - 1 maintainer
pyspark-connectors 0.3.0
The easy and quickly way to connect and integrate the Spark project with many others data sources.
9 versions - Latest release: over 2 years ago - 58 downloads last month - 6 stars on GitHub - 1 maintainer
pyspark-regression 4.2.4
A tool for regression testing Spark Dataframes in Python
7 versions - Latest release: 7 months ago - 46.8 thousand downloads last month - 3 stars on GitHub - 1 maintainer
patek 0.5.2
A collection of utilities and tools for accelerating pyspark development and productivity.
7 versions - Latest release: over 3 years ago - 88 downloads last month - 0 stars on GitHub - 1 maintainer
tpcds-pyspark 1.0.6
TPCDS_PySpark is a TPC-DS workload generator implemented in Python designed to run at scale using...
6 versions - Latest release: over 1 year ago - 151 downloads last month - 465 stars on GitHub - 1 maintainer
chopin2 1.0.9
Supervised Classification with Hyperdimensional Computing
13 versions - Latest release: over 2 years ago - 1 dependent repositories - 25 downloads last month - 13 stars on GitHub - 1 maintainer
flintrock 2.1.0
A command-line tool for launching Apache Spark clusters.
14 versions - Latest release: almost 3 years ago - 1 dependent repositories - 1.01 thousand downloads last month - 649 stars on GitHub - 1 maintainer
test-cpu-parallel 1.0.5
test-CPU-parallel is a basic CPU workload generator.
2 versions - Latest release: about 3 years ago - 57 downloads last month - 465 stars on GitHub - 1 maintainer
sparkhistogram 0.4
Sparkhistogram contains helper functions for generating data histograms with the Spark DataFrame ...
4 versions - Latest release: over 2 years ago - 1 dependent repositories - 1.58 thousand downloads last month - 464 stars on GitHub - 1 maintainer
jennytest 0.2.9
Data quality and profiling tool powered by Apache Spark.
5 versions - Latest release: over 6 years ago - 39 downloads last month
sparkflow 0.7.0
Deep learning on Spark with Tensorflow
13 versions - Latest release: over 7 years ago - 1 dependent repositories - 31 downloads last month - 296 stars on GitHub - 1 maintainer
blind 0.0.1
Blind Client: The easiest ML tracking library
1 version - Latest release: almost 5 years ago - 2 dependent repositories - 61 downloads last month - 20,464 stars on GitHub - 1 maintainer
Top 6.6% on pypi.org
spylon 0.3.0
Utilities to work with Scala/Java code with py4j
19 versions - Latest release: over 9 years ago - 16 dependent repositories - 2.17 thousand downloads last month - 40 stars on GitHub - 3 maintainers
Top 3.8% on pypi.org
pysparkling 0.6.2
Pure Python implementation of the Spark RDD interface.
69 versions - Latest release: almost 4 years ago - 1 dependent package - 34 dependent repositories - 9.06 thousand downloads last month - 271 stars on GitHub - 2 maintainers
Top 7.8% on pypi.org
dist-keras 0.2.1
Distributed Deep learning with Apache Spark with Keras.
3 versions - Latest release: almost 9 years ago - 1 dependent repositories - 145 downloads last month - 622 stars on GitHub - 1 maintainer
sparkora 0.0.1
Exploratory data analysis toolkit for Pyspark
1 version - Latest release: almost 5 years ago - 1 dependent repositories - 12 downloads last month - 53 stars on GitHub - 1 maintainer
livyc 0.0.14 💰
Apache Livy Client
11 versions - Latest release: over 4 years ago - 55 downloads last month - 3 stars on GitHub - 1 maintainer
gdmix-workflow 0.3.0
Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.
5 versions - Latest release: almost 6 years ago - 1 dependent repositories - 34 downloads last month - 2,912 stars on GitHub - 1 maintainer
dcborow-mmlspark 0.14.dev1
Microsoft ML for Spark
1 version - Latest release: over 6 years ago - 1 dependent repositories - 280 downloads last month - 4,986 stars on GitHub - 1 maintainer
svm 0.1.0
Version manager for Apache Spark
2 versions - Latest release: almost 9 years ago - 14 dependent repositories - 388 downloads last month - 1 stars on GitHub - 1 maintainer
spark-privacy-preserver 0.3.1
Anonymizing Library for Apache Spark
4 versions - Latest release: about 6 years ago - 1 dependent repositories - 26 downloads last month - 31 stars on GitHub - 3 maintainers
pysparql 0.0.6
Query a SPARQL endpoint and manage the result with Spark
2 versions - Latest release: almost 6 years ago - 1 dependent repositories - 114 downloads last month - 0 stars on GitHub - 1 maintainer
spark-pipeline 0.0.4
Data Science oriented tools, mostly for Apache Spark
2 versions - Latest release: over 6 years ago - 1 dependent repositories - 15 downloads last month - 9 stars on GitHub - 1 maintainer
mmtfpyspark 0.3.6
Methods for parallel and distributed analysis and mining of the Protein Data Bank using MMTF and ...
10 versions - Latest release: almost 8 years ago - 1 dependent repositories - 60 downloads last month - 68 stars on GitHub - 2 maintainers
k8s-spark-helper-liangdao-data 0.0.3
Run spark task in k8s
3 versions - Latest release: over 3 years ago - 39 downloads last month - 2,914 stars on GitHub - 1 maintainer
mlflow-databricks-artifacts 2.0.1
Plugin to create and access MLflow-managed artifacts on Databricks
3 versions - Latest release: about 4 years ago - 1 dependent repositories - 679 downloads last month - 20,464 stars on GitHub - 1 maintainer
sparkmanager 0.7.3
A pyspark management framework
21 versions - Latest release: over 6 years ago - 1 dependent repositories - 94 downloads last month - 0 stars on GitHub - 1 maintainer
dbnet 0.2.5
DbNet.
24 versions - Latest release: over 6 years ago - 1 dependent repositories - 76 downloads last month - 7 stars on GitHub - 1 maintainer
exelog 0.0.1
Enabling meticulous logging for Spark Applications
1 version - Latest release: almost 5 years ago - 1 dependent repositories - 21 downloads last month - 5 stars on GitHub - 1 maintainer
Top 3.3% on pypi.org
quinn 0.10.3
Pyspark helper methods to maximize developer efficiency
16 versions - Latest release: over 2 years ago - 3 dependent packages - 11 dependent repositories - 587 thousand downloads last month - 610 stars on GitHub - 1 maintainer