Data Lake Curated Dataset to Power BI Push

Overview

This flow closes the lake-to-dashboard gap from Azure Data Lake. On an hourly schedule it lists and reads the aggregated curated-zone mart files, totals the data rows across them, validates that total against a configurable floor, and - only when the data looks sane - triggers a Power BI dataset refresh so the dashboard reflects the latest mart. It posts a success summary or a count-anomaly alert to a Microsoft Teams channel either way.

Why it matters: getting curated data into Power BI usually means gateways and ADF/Synapse pipelines. A flow-driven refresh with a built-in record-count sanity check is a lighter, governed path for pre-aggregated marts - and it refuses to publish when the row count collapses, preventing a broken dashboard.

Ships Off (demo).

Use Case

A data team lands pre-aggregated marts in the curated zone of a Data Lake and wants them visible in Power BI on a schedule, without standing up a separate gateway/pipeline - and with a guardrail that skips the refresh if the mart looks empty or truncated.

Flow Architecture

Environment Variables

Connectors & Connections

Connector	API name	Actions used
Azure Data Lake	shared_azuredatalake	ListFiles ReadFile
Power BI	shared_powerbi	RefreshDataset
Microsoft Teams	shared_teams	PostMessageToConversation

Note — All connections are referenced as solution connection references; the flow is portable between environments as long as a connection is mapped at import time.

Customization Guide

Almost every realistic variant of this flow can be implemented by changing environment variable values. A few cases require small edits inside the flow definition — those are called out explicitly below.

Push rows instead of refresh (AddRows)

For a Power BI push/streaming dataset, replace RefreshDataset with AddRows (groupid + datasetid + tablename, body Payload). Configure a real dataset/table first so the designer can resolve the dynamic body columns.

Incremental loads

Track the last-processed partition/file (a Dataverse row or state file) and have ListFiles / the filter only pick up new partitions.

Multi-dataset fan-out

Loop a list of workspace/dataset ids to refresh several reports from one run.

Stricter validation

Raise the min-rows floor, or compare against the prior run's count to catch a sudden drop rather than just an empty file.

Poll the refresh

After RefreshDataset, add a Do-until on Get Refresh History to confirm success before notifying.

Key Expressions

The flow is intentionally light on Power Fx / WDL gymnastics — the heaviest expressions are the branch-name concatenation and the approval outcome check. They are listed below in the order they appear in the flow.

sub(length(split(trim(replace(string(body('Read_Curated_File')), decodeUriComponent('%0D'), '')), decodeUriComponent('%0A'))), 1)

add(variables('varRowCount'), if(greater(outputs('Compose_Row_Count_For_File'),0), outputs('Compose_Row_Count_For_File'), 0))

greaterOrEquals(variables('varRowCount'), int(variables('varMinExpectedRows')))

coalesce(outputs('List_Curated_Mart_Files')?['body/FileStatuses/FileStatus'], createArray())