Cosmos DB Change Feed to Downstream Sync
Polling the Cosmos DB change feed on a schedule, the flow processes new and updated documents, projects them into Dataverse/SQL, posts events to Teams, and tracks a continuation token for exactly-once-ish processing. Turns Cosmos changes into reliable downstream updates.
Provided as-is, without warranty of any kind. Review and test each pattern in a non-production environment before deploying it to live automations. See our Terms.
Overview
This flow turns Azure Cosmos DB changes into reliable downstream updates. On a 5-minute schedule it polls the Cosmos change feed using a stored _ts watermark, projects each new or updated document into a Microsoft Dataverse mirror table with an idempotent upsert (keyed on the Cosmos document id), advances the watermark for near exactly-once processing, and posts a run summary to a Microsoft Teams channel. Apps write to Cosmos but analytics and operations often live elsewhere; consuming the change feed keeps downstream systems current without dual-writes or brittle export jobs. The solution ships Off (demo) — going live requires only authorizing the three connections and setting the environment variable values.
Use Case
A Cosmos-backed application needs its document changes mirrored into Dataverse for reporting and automation, while operations wants a heartbeat in Teams each cycle showing how many documents moved. The continuation watermark guarantees each run only reads what changed since the last successful run, so the flow is cheap and resumable. Two Dataverse tables back the flow: a continuation-token state table (one row per feed) and a downstream projection table mirroring each Cosmos document.
Flow Architecture
Every 5 Minutes
RecurrencePolls the Cosmos change feed on a 5-minute schedule.
Initialize Trace & Config
Initialize variableMints a guid() correlation id and binds the Cosmos account/database/container, continuation store, state key, Teams ids, the _ts watermark (default 0), and a synced-count counter.
Get Continuation State
Dataverse - ListRecordsReads the saved watermark row; a condition loads flowlibs_lastts when present, otherwise keeps 0.
Read Change Feed
Cosmos - QueryDocuments_V5Composes SELECT * FROM c WHERE c._ts > {watermark} ORDER BY c._ts ASC and returns changed documents under body/value.
Apply to Each Changed Document
Apply to eachSequentially upserts each document into the Dataverse projection (ListRecords then UpdateRecord or CreateRecord, idempotent on the Cosmos document id) and increments the synced count.
Advance Watermark
Dataverse - UpdateRecord/CreateRecordComputes the new _ts watermark and updates or creates the continuation-token row.
Post Sync Summary to Teams
Teams - PostMessageToConversationPosts documents synced, new watermark, and correlation id to the events channel.
Environment Variables
| Schema name | Type | Default | Description |
|---|---|---|---|
| flowlibs_CosmosAccount | String | REPLACE_WITH_COSMOS_ACCOUNT_NAME | Cosmos DB (database account) name. |
| flowlibs_CosmosDatabase | String | REPLACE_WITH_COSMOS_DATABASE | Cosmos database id. |
| flowlibs_CosmosContainer | String | REPLACE_WITH_COSMOS_CONTAINER | Source container whose change feed is polled. |
| flowlibs_ContinuationStore | String | flowlibs_cosmosfeeds | Dataverse entity set name of the watermark table. |
| flowlibs_TeamsGroupId | String | <your-team-id> | Teams team (group) id for the events channel. |
| flowlibs_TeamsChannelId | String | <your-channel-id> | Teams channel id for the events channel. |
Connectors & Connections
| Connector | API name | Actions used |
|---|---|---|
| Azure Cosmos DB | shared_documentdb | QueryDocuments_V5 |
| Microsoft Dataverse | shared_commondataserviceforapps | ListRecords CreateRecord UpdateRecord |
| Microsoft Teams | shared_teams | PostMessageToConversation |
Note — All connections are referenced as solution connection references; the flow is portable between environments as long as a connection is mapped at import time.
Customization Guide
Almost every realistic variant of this flow can be implemented by changing environment variable values. A few cases require small edits inside the flow definition — those are called out explicitly below.
- Per-type routing
- Branch on a docType field to project different document types into different Dataverse tables or downstream systems.
- Deletes
- The change feed surfaces upserts only; adopt a soft-delete flag in source documents and mirror it as a status, since hard deletes are not emitted.
- Partition key
- The demo stores coalesce(partitionKey, id); set this to your container's actual partition-key path for accurate projection.
- Throughput
- For high-volume containers, shorten the recurrence or hand off to an Azure Function change-feed processor and have this flow orchestrate.
- Payload size
- flowlibs_payload is a Memo column storing the full document JSON; project only the fields you need if documents are large.
Key Expressions
The flow is intentionally light on Power Fx / WDL gymnastics — the heaviest expressions are the branch-name concatenation and the approval outcome check. They are listed below in the order they appear in the flow.
EXPR.01Watermark query
Builds the incremental Cosmos query from the stored watermark.
EXPR.02Load saved watermark
Reads the last saved _ts, defaulting to 0 on first run.
EXPR.03New watermark
Advances to the newest processed _ts, or keeps the prior value.
EXPR.04Idempotency filter
Locates an existing projection row for upsert.
EXPR.05Payload capture
Serializes the full document JSON into the Memo column.
Comments
Sign in to join the conversation.
Sign inNo comments yet. Be the first to share your experience with this flow.