Blob Inbound File Processing Pipeline

Overview

This flow turns an Azure Blob Storage drop-zone into a governed, auditable ETL intake. A polling trigger fires once per file dropped into the inbound container; the flow reads the file, validates its format (CSV / JSON / XML), parses it, loads each row into a SQL staging table, moves the original to a processed or failed container (copy then delete), and posts the outcome to a Teams channel. Every file is tagged with a correlation id stamped on every SQL row and the notification.

Why it matters: manual file loads are slow and error-prone. An automated, auditable intake keeps data flowing reliably from partners and systems, quarantines bad files for review, and gives operations a single source of truth for what landed and when.

Ships Off (Stopped).

Use Case

Partners and upstream systems drop data files (CSV/JSON/XML) into a Blob container. Those files must be validated, parsed, and loaded into a database automatically, with successful files archived, malformed files quarantined, and the owning team notified of each outcome.

Flow Architecture

Environment Variables

Connectors & Connections

Note — All connections are referenced as solution connection references; the flow is portable between environments as long as a connection is mapped at import time.

Customization Guide

Almost every realistic variant of this flow can be implemented by changing environment variable values. A few cases require small edits inside the flow definition — those are called out explicitly below.

Schema validation

Replace the extension check with validation against an expected schema contract (required CSV headers, a JSON schema) and quarantine on mismatch.

Typed loads

Swap the generic staging INSERT for a typed PostItem_V2 or a fully-typed INSERT once the source schema is fixed.

Robust CSV parsing

For files with quoted commas or embedded newlines, add a parsing routine or route through a dataflow.

Idempotency

Dedupe re-uploads by checking the staging table for the file name + hash before loading.

Partner SLAs

Pair with a scheduled expected-file watchdog that alerts when a partner's daily file is missing.

Key Expressions

The flow is intentionally light on Power Fx / WDL gymnastics — the heaviest expressions are the branch-name concatenation and the approval outcome check. They are listed below in the order they appear in the flow.

toLower(last(split(triggerBody()?['Name'], '.')))

base64ToString(body('Get_Blob_Content')?['$content'])

skip(split(replace(outputs('Decode_File_Text'), decodeUriComponent('%0D'), ''), decodeUriComponent('%0A')), 1)

concat('/', variables('varProcessedContainer'), '/', triggerBody()?['Name'])