Assert unique
Assert unique — assertUnique
Verify that a set of columns contains no duplicate rows.
The node is pass-through: the dataframe leaves unchanged regardless of the
outcome. The violation is recorded in the run's per-node result and either fails
the run (error mode) or logs a warning and continues (warn mode).
Use cases
- Verify that a join key or primary key is truly unique before a merge.
- Catch accidental fan-out (duplicate rows) introduced by an upstream join.
- Assert that a deduplication step actually worked.
Configuration
| Config key | Type | Required | Description |
|---|---|---|---|
columns | list of strings | No | Column combination that must be unique. Empty (or omitted) checks the whole row across every column. |
mode | "error" | "warn" | No | "error" (default) stops the run; "warn" continues and logs |
Behavior
| Outcome | What happens |
|---|---|
| No duplicates across the specified columns | Run continues; assertion_passed: true |
Duplicates found, mode: "error" | Run fails; error names the columns and duplicate count |
Duplicates found, mode: "warn" | Run continues; warning recorded with duplicate count |
The per-node result in the run detail always includes assertion_passed,
assertion_violation_count (number of duplicate rows), and a sample of up to 5
violating rows.
Generated Python code
_dup_mask = df_1.duplicated(subset=['order_id'], keep=False)
if _dup_mask.any():
raise ValueError(f"assertUnique: {_dup_mask.sum()} duplicate row(s)")
In warn mode the raise is replaced by warnings.warn(...) and execution
continues.
Tips & common mistakes
- Column order doesn't matter.
["a", "b"]and["b", "a"]check the same uniqueness constraint. - All rows with the duplicate key are counted. If
order_id = 7appears three times, the violation count is 3. - Use Remove duplicates if you want to drop duplicate rows instead of asserting they don't exist.