Integrations
Adapter for Apache® Airflow
Automate Airflow activities with advanced scheduling functionality
Integrate Airflow DAGs into business workflows
Enable Developers to focus on higher value work instead of scheduling tasks
Leverage more advanced scheduling functionality
Orchestrate Airflow DAGs
Airflow is popular among developers that need to manage and monitor data pipelines. Users develop Python language scripts that define Directed Acyclic Graphs (DAGs). These DAGs specify dependencies and a sequence of steps (or tasks) to run either on a defined schedule or based on external event triggers.
Our adapter for Airflow provides seamless integration and automation of Airflow DAG executions in your Tidal-managed processes. With more robust scheduling functionality than the basic features in Airflow, Tidal enables you to include complex DAGs as Airflow jobs within your enterprise workflows and schedule those jobs based on other dependencies and resources. Centralized management frees up developers from monitoring and scheduling tasks so they can focus on higher value activities.
Airflow Directed Acyclic Graph (DAG)
Adapter Capabilities
This adapter offers compatibility with Airflow v1 and v2 as well as Airflow activities embedded in Google Cloud Composer. With our integration, you can:
- Tie Airflow activities into cross-application workflows
- Schedule Airflow jobs based on time, events or dependencies external to a DAG
- Verify the availability of external resources – such as Kubernetes – before starting an Airflow job
- Distribute Airflow jobs across multiple Airflow clusters for improved performance
Tidal can orchestrate Airflow workflows whether they're running on-prem or in the cloud.
How It Works
- The Tidal adapter for Airflow creates a secure connection between the Tidal and Airflow servers.
- The adapter pulls selected DAGs into Tidal so you can define Tidal job and Airflow DAG configuration parameters as well as dependencies between the DAGs and other Tidal jobs.
- The Airflow server runs the requested process.
- Airflow DAG job runs can be controlled and monitored through the Tidal interface as part of schedule execution.
Overview & Demo Video
Tidal can simplify and optimize the scheduling of Airflow jobs and add advanced workflow management capabilities. Learn about the value of the adapter and how to integrate Airflow DAGs into enterprise processes in this short video.
OUR CORE PLATFORM
Tidal Automation
A unified platform for centralized management and control of job scheduling across business, operations and IT processes. Tidal Automation Platform >
