About Data Pipelines

Data Pipeline Studio (DPS) provides the ability to create no-code and low-code data pipelines within a few clicks. DPS lets you add the required stages, nodes, tools, and connectors to create data pipelines. It provides templates for a variety of combination of tools used in data ingestion, data transformation, and data quality stages of a data pipeline. The task of creating jobs is simplified using templates, as they require minimal inputs for creating and running data pipelines to get the required results.

After creating a data pipeline you can either run the complete pipeline or you can run the individual jobs that are part of the pipeline. You can create versions of the pipeline as you make changes to the pipeline and save them. You can also promote a pipeline from one development stage to another (dev > qa > prod).

Create a Data Pipeline
Run and Monitor a Data Pipeline
Edit Jobs in a Data Pipeline
Handling Pipeline Failures in Data Pipeline Studio
Data Pipeline Versioning and Rollback
Pipeline Version and Wheel Package Version
Promoting Data Pipelines to Different Stages in DPS