PADData Operationslistdedupedatatable

List Deduplication

Removes duplicate rows from a PAD DataTable based on one or more key columns. Uses a PowerShell hash-set approach for O(n) performance — handles 100K rows in seconds where a nested-loop approach would take minutes. Also includes a simpler native PAD variant for small datasets.

Members OnlySign in or create a free account to view and copy this pattern.

Usage Notes

  • Set varKeyColumn to the column name that defines uniqueness (e.g.
  • or
  • ).
  • For multi-column keys, concatenate them: varKeyColumn =
  • and adjust the PowerShell script accordingly.
  • The native PAD variant (Pattern 1) works well for under ~5,000 rows. Use the PowerShell variant for larger datasets.
  • The output DataTable preserves the first occurrence of each duplicate — later duplicates are dropped.
  • Always run this before writing to databases or SharePoint to avoid constraint violations.

Requirements

  • Power Automate Desktop 2.30+
  • Variables: varSourceData (DataTable), varKeyColumn (Text), varDedupedData (DataTable)