Data Factory SLA and Run Duration Monitor

Overview

This flow keeps Azure Data Factory pipelines on schedule and within SLA. On an hourly recurrence it pulls recent pipeline runs, re-fetches each run's authoritative metadata, classifies it (failed, over-SLA, on-track, in-progress), writes one snapshot row per run to a governed Dataverse fact table that feeds a Power BI ops dashboard, and — only when something breached — alerts the data team in Microsoft Teams with a per-run summary. Slow or failed pipeline runs delay data without anyone noticing; continuous SLA monitoring surfaces breaches and degradation trends early, and the Dataverse fact table gives Power BI a clean, historical basis for trend reporting. Ships Off; going live requires only authorizing the three connections, supplying the Azure service-principal + Data Factory env vars, and setting the Teams group/channel ids.

Use Case

A data team running scheduled Azure Data Factory pipelines wants assurance that pipelines run on time and within expected duration, an auditable history of every run for trend analysis, and a proactive alert the moment a run fails or blows past its SLA window — without manually watching the ADF monitoring blade.

The flow is ideal for teams that:

• Hourly SLA sweep over recent ADF pipeline runs
• Classifies each run: Run Failed / SLA Exceeded / On Track / In Progress
• Writes one snapshot row per run to a governed Dataverse fact table for Power BI
• Posts a consolidated Teams alert only when a run breaches

Flow Architecture

Environment Variables

Connectors & Connections

Note — All connections are referenced as solution connection references; the flow is portable between environments as long as a connection is mapped at import time.

Customization Guide

Almost every realistic variant of this flow can be implemented by changing environment variable values. A few cases require small edits inside the flow definition — those are called out explicitly below.

Per-pipeline SLAs

Replace the single flowlibs_DefaultSLAMinutes with a JSON map (pipeline -> SLA minutes) and look up each run's threshold in Compose Breach Type.

Missed-run detection

Compare expected scheduled runs against returned runs and alert when a scheduled run never started.

Live Power BI refresh

Once a Power BI connection is authorized in the tenant, add a RefreshDataset action after the loop (bind the dataset id to a new flowlibs_PowerBiDatasetId env var). Today the Dataverse fact table itself is the governed dashboard feed.

Rolling-average anomaly

Trend flowlibs_durationminutes per pipeline and flag runs slower than the rolling average, not just a fixed SLA.

Auto-retry transient failures

Branch on status = Failed with a transient error and call the ADF connector to re-run the pipeline.

Key Expressions

The flow is intentionally light on Power Fx / WDL gymnastics — the heaviest expressions are the branch-name concatenation and the approval outcome check. They are listed below in the order they appear in the flow.

@addHours(utcNow(), mul(-1, int(variables('varLookbackHours'))))

@div(coalesce(body('Get_Pipeline_Run_Details')?['durationInMs'], 0), 60000)

@if(or(equals(status,'Failed'),equals(status,'Cancelled')), 'Run Failed', if(and(equals(status,'Succeeded'),greater(durationMin, int(SLA))), 'SLA Exceeded', if(equals(status,'Succeeded'),'On Track','In Progress')))

@greater(variables('varBreachCount'), 0)