Apache Airflow is an open-source platform for creating, running and monitoring workflows. Users develop Python language scripts that define Directed Acyclic Graphs (DAGs). These DAGs specify dependencies and a sequence of steps (or tasks) to run either on a defined schedule or based on external event triggers. Airflow is popular among developers that need to monitor and manage data pipelines.
Our adapter for Airflow provides seamless integration and automation of Airflow DAG executions into your Tidal-managed processes. With more robust scheduling functionality than the basic features in Airflow, Tidal Automation enables you to include complex DAGs as Airflow jobs within your enterprise workflows and schedule those jobs based on other dependencies and resources. Centralized management frees up developers from monitoring and scheduling tasks so they can focus on higher value activities.
The Tidal adapter for Airflow enables you to:
Airflow DAGs can be viewed directly in Tidal through a built-in DAG viewer.