Big Data Pipeline Developer

Adapters for Apache Hadoop

Automate your Big Data workflows

Tidal Hero Grid Image

Automate critical data flows in cross-application processes

Tidal Hero Grid Image

Improve data management across systems and processes

Tidal Hero Grid Image

Turn data into insight for better decision-making

Adapters for Hadoop

Tidal offers a set of adapters to support the various Apache Hadoop software utilities. Get the benefits of centralized management and control plus advanced scheduling functionality by integrating your Hadoop activities in Tidal.

Adapter for Sqoop

Sqoop is a tool designed to transfer data between Hadoop and relational databases. This adapter enables you to automate the various tasks performed by Sqoop – import, transform and export data. The integration enables the following job definitions in Tidal:

  • Code Generation – generate Java classes which interpret imported records
  • Import – import structure data from a relational database management system (RDBMS) to the Hadoop Distributed File System (HDFS)
  • Export – export files from HDFS back to an RDBMS
  • Merge – combine two datasets where entries in one dataset will overwrite entries of an older dataset

Adapter for MapReduce

MapReduce is the programming model in the Hadoop framework that is used to access and process large amounts of data stored in the HDFS. The adapter uses the Hadoop API to submit and monitor MapReduce jobs with the full scheduling capabilities of Tidal. Through this integration, jobs in Tidal divide the input data into independent chunks that are processed by the map tasks in parallel. The adapter serves as the job client to automate the execution of MapReduce jobs as part of Tidal-managed processes.

Adapter for Hive

This adapter enables you to access and manage data stored in the HDFS using Hive’s querly language (HiveQL). The seamless integration with the Tidal platform enables you to define, launch, control and monitor HiveQL commands submitted to Hive via JDBC on a scheduled basis.

HDFS Data Mover Linux Agent

This data mover agent enables you to easily manage file transfers in and out of the Hadoop file system.

Tidal can orchestrate Hadoop workflows whether they're running on-prem or in the cloud.