ADO Test Run Failure Summary Report
Triggered daily, this flow queries Azure DevOps for recent test runs, extracts failure details, logs them to an Excel workbook for trend tracking, and sends a formatted Outlook email summary to stakeholders.
Provided as-is, without warranty of any kind. Review and test each pattern in a non-production environment before deploying it to live automations. See our Terms.
Overview
This flow monitors Azure DevOps test runs daily, identifies failures, logs them to an Excel workbook for historical tracking, and emails a formatted failure summary to stakeholders. It ships in the Off state as a portable reference implementation — turning it on requires only connection authorization and environment variable configuration.
Use Case
Development and QA teams running automated tests in Azure DevOps need timely visibility into test failures without manually checking each pipeline. This flow automates failure detection and notification so that test owners can triage failures the morning after they occur, while also building a historical failure trend dataset for analysis.
The flow is ideal for teams that:
- Development teams running automated test suites in Azure DevOps
- QA engineers needing daily failure triage without manual pipeline checks
- Engineering managers tracking test failure trends over time
- Teams that want morning-of-shift visibility into overnight test results
Flow Architecture
Recurrence
RecurrenceRuns daily at 8:00 AM Eastern.
Initialize Variables
Initialize variable (x9, parallel)Binds all environment variables to runtime variables: ADO organization, ADO project, SharePoint site URL, recipient email, Excel drive ID, Excel file path, Excel table name, plus an HTML accumulator and total failure counter.
Get Recent Test Runs
Azure DevOps — HttpRequest (GET)Queries `_apis/test/runs` for test runs updated in the last 24 hours.
Parse Test Runs
Parse JSONExtracts the response into a typed array of test run objects (id, name, totalTests, passedTests, unanalyzedTests, completedDate, state).
For Each Test Run
Apply to eachFor each test run returned by the query: check whether it has failures and, if so, fetch failure details, log them to Excel, and increment the failure counter.
Check If Run Has Failures
If conditionBranches on whether the current test run reports any unanalyzed (failed) tests: `unanalyzedTests > 0`.
- Get Failed Test Results — Azure DevOps HttpRequest (GET) queries `_apis/test/runs/{runId}/results?outcomes=Failed` for failed test cases.
- Parse Failed Results — Parse JSON extracts failed test details (testCaseTitle, errorMessage, stackTrace, outcome).
- For Each Failed Test — Inner Apply to each: for every failed test, append an HTML table row, log a row to the Excel trend workbook, and increment the failure counter.
Environment Variables
| Schema name | Type | Default | Description |
|---|---|---|---|
| flowlibs_AdoOrganization | String | <configure> | Azure DevOps organization name. Set this to the slug used in your ADO URL (https://dev.azure.com/<organization>). |
| flowlibs_AdoProject | String | FlowLibs | Azure DevOps project name to query for test runs. |
| flowlibs_SharePointSiteURL | String | https://your-tenant.sharepoint.com | SharePoint root site URL hosting the Excel trend workbook. |
| flowlibs_TestFailureRecipientEmail | String | alerts@yourcompany.com | Email address that receives the failure summary report. |
| flowlibs_TestFailureExcelDrive | String | <configure> | Microsoft Graph drive ID for the document library that holds the Excel trend workbook. |
| flowlibs_TestFailureExcelFile | String | /Shared Documents/FlowLibs - Test Failure Trends.xlsx | Server-relative path to the Excel workbook used for failure trend logging. |
| flowlibs_TestFailureExcelTable | String | TestFailureTrends | Excel table name (within the workbook) that the flow appends rows to. |
Connectors & Connections
| Connector | API name | Actions used |
|---|---|---|
| Azure DevOps | shared_visualstudioteamservices | HttpRequest (Query test runs and test results) |
| Office 365 Outlook | shared_office365 | SendEmailV2 (Send failure summary email) |
| Excel Online (Business) | shared_excelonlinebusiness | AddRowV2 (Append failure rows to tracking workbook) |
Note — All connections are referenced as solution connection references; the flow is portable between environments as long as a connection is mapped at import time.
Customization Guide
Almost every realistic variant of this flow can be implemented by changing environment variable values. A few cases require small edits inside the flow definition — those are called out explicitly below.
- Update environment variables
- Set flowlibs_AdoOrganization and flowlibs_AdoProject to your ADO org and project. Set flowlibs_TestFailureRecipientEmail to the recipient that should get failure reports. Set the SharePoint URL, Excel drive ID, file path, and table name to point at your tracking workbook.
- Authorize connections
- Open the flow in the designer, click each action that shows Add an authorized connection, and sign in with appropriate credentials for Azure DevOps, Office 365 Outlook, and Excel Online (Business).
- Swap the trigger (optional)
- The Recurrence trigger polls daily. To react in real-time, replace it with an ADO Service Hook webhook or an Azure DevOps OnWorkItemUpdatedV2 trigger pointing to test-related work items. Note: ADO connector triggers require live validation against the ADO project during save.
- Adjust the polling window
- The addDays(utcNow(), -1, ...) expression in Get Recent Test Runs looks back 24 hours. Change -1 to -7 for weekly scans or -1/24 patterns via addHours for tighter windows.
- Filter by test suite
- Append &planId={planId}&suiteId={suiteId} to the ADO API URI in Get Recent Test Runs to scope to a specific test plan or suite.
- Turn the flow on
- The solution ships with the flow Off. Once connections are authorized and env vars are configured, toggle the flow from Off to On.
Key Expressions
The flow is intentionally light on Power Fx / WDL gymnastics — the heaviest expressions are the branch-name concatenation and the approval outcome check. They are listed below in the order they appear in the flow.
EXPR.0124-hour lookback
Generates the ISO timestamp for 24 hours ago — used to scope the ADO test runs query.
EXPR.02Stack trace truncation
Caps stack traces at 200 characters before writing them to the Excel column, with a `N/A` fallback when null.
EXPR.03Failure count gate
Gates the email send on whether any failures were recorded during the run.
EXPR.04HTML body assembly
Accumulates one HTML table row per failure inside `varFailureSummaryHtml`, which is then dropped into the Compose action for the email body.
Comments
Sign in to join the conversation.
Sign inNo comments yet. Be the first to share your experience with this flow.