Prefect

Apache Airflow

apache/airflow

Popular

The default open-source orchestrator for authoring and scheduling data pipelines as Python DAGs; v3 since 2025.

Dagster

dagster-io/dagster

Asset-centric orchestrator with lineage, type-checked pipelines and strong local development.

AWS Step Functions

aws.amazon.com

AWS's serverless state-machine orchestrator, commonly used to chain Glue, Lambda and EMR jobs.

Kestra

kestra-io/kestra

Rising

Declarative, YAML-based orchestration platform for data, infrastructure and AI workflows.

Argo Workflows

argoproj/argo-workflows

Kubernetes-native workflow engine for CI, data and ML pipelines defined as DAGs of containers.

Flyte

flyteorg/flyte

Kubernetes-native orchestrator for data and ML pipelines with strong typing and versioning.

Mage

mage-ai/mage-ai

Notebook-style open-source pipeline tool for building, running and monitoring data pipelines.

Luigi

spotify/luigi

Legacy

Spotify's early Python pipeline framework; largely superseded by Airflow, Dagster and Prefect.

Taipy

Avaiga/taipy

Turns Data and AI algorithms into production-ready web applications in no time.