Azure Data Factory Pipeline Trigger & Monitor

Overview

This flow orchestrates and monitors an Azure Data Factory (ADF) pipeline end-to-end from Power Automate. On a schedule it triggers a pipeline run, polls the run status in a Do Until loop until the run reaches a terminal state, then routes the outcome: a success card to a Microsoft Teams channel, and on failure/cancellation a Teams card plus a high-importance Outlook email to the data team containing the ADF error detail and a deep link into the ADF monitoring blade. Why it matters: ADF's own alerting is coarse and lives in the ADF portal. Wrapping a run in a flow gives business-friendly start triggers (a schedule, a button, an upstream event) and rich, routed notifications - no one has to sit watching the ADF monitor. The flow ships Off; going live requires only authorizing the three connections and setting the environment-variable values.

Use Case

Data-engineering teams want non-engineers to be able to kick off (or schedule) an ETL pipeline and receive clear, routed success/failure notifications instead of silent runs. This reference flow shows the canonical trigger -> create-run -> poll-until-done -> report pattern with a timeout-guarded Do Until and a Switch that fans out per outcome.

Flow Architecture

Environment Variables

Connectors & Connections

Connector	API name	Actions used
Azure Data Factory	shared_azuredatafactory	CreatePipelineRun GetPipelineRun
Microsoft Teams	shared_teams	PostMessageToConversation
Office 365 Outlook	shared_office365	SendEmailV2

Note — All connections are referenced as solution connection references; the flow is portable between environments as long as a connection is mapped at import time.

Customization Guide

Almost every realistic variant of this flow can be implemented by changing environment variable values. A few cases require small edits inside the flow definition — those are called out explicitly below.

Pipeline parameters

Pass run-time parameters (date window, source path) into Create Pipeline Run via the action's Parameters input (a JSON object), feeding values from the trigger.

On-demand start

Replace the Recurrence trigger with a PowerApps (V2) or manual trigger to let a user kick the pipeline off from an app or button; the rest of the flow is unchanged.

Poll cadence and timeout

Adjust Initialize Poll Interval Seconds (default 60s) and the Do Until limit (count 60 / timeout PT2H) so a hung pipeline cannot loop forever.

Row counts

GetPipelineRun returns status, duration, and message but not rows copied. To report row counts, add an ADF activity-runs query (or read a copy activity output) after the loop and include rowsCopied in the notifications.

Chaining

Add a second Create Pipeline Run that runs only if the first succeeds to build a dependency chain inside one flow.

Routing

Split the Switch default into explicit Failed and Cancelled cases if cancellations should notify differently.

Key Expressions

The flow is intentionally light on Power Fx / WDL gymnastics — the heaviest expressions are the branch-name concatenation and the approval outcome check. They are listed below in the order they appear in the flow.

@or(equals(variables('varRunStatus'),'Succeeded'), equals(variables('varRunStatus'),'Failed'), equals(variables('varRunStatus'),'Cancelled'))

@body('Create_Pipeline_Run')?['runId']

@body('Get_Run_Status')?['status']

@div(variables('varDurationMs'),60000)

@{concat(variables('varMonitorBaseUrl'),'/',variables('varRunId'),'?factory=/subscriptions/',variables('varSubscriptionId'),'/resourceGroups/',variables('varResourceGroup'),'/providers/Microsoft.DataFactory/factories/',variables('varFactoryName'))}

@{guid()}

Customize & download