Analysis / Data engineering and quality
Find duplicate business records safely
Identify exact and probable duplicate records, separate legitimate repeated transactions, and prepare evidence for a safe consolidation review.
Data analyst · Analytics engineer
Bring this question into Chat
Your starting prompt.
Inspect [dataset] for exact duplicates and probable duplicate entities using [business key] and [candidate fields]. Separate repeated legitimate transactions from conflicting versions and ambiguous matches. Show duplicate groups, counts, changed fields and estimated metric distortion under explicit consolidation rules. Prepare a proposed deduplication review table with source IDs and confidence reasons; do not delete, merge or overwrite source records.
Copy this starting point, then replace the placeholders with your own inputs.
Method and review
Keep the assumptions visible.
Profile keys, row counts, types, missing values and join cardinality. Retain source lineage and concrete exception records so a correction can be checked.
For this task, check
Fuzzy matches need human review because similar records can represent different real entities.
Before you begin
- Bring exports you are allowed to upload, or data your Workspace can already query.
- Choose the reporting period and agree what each record represents.
Work with permitted files or queryable data. Verify the result against the original records.
Synthetic illustrative example
See the method in practice.
Synthetic input. Illustrative output checked locally; this is not a live Proforma run or customer result. Each entity should have one current balance. Exact duplicate R-2 and later conflicting B balance require a review policy, not automatic deletion.
These invented records explain the method. They are not a recorded Proforma run.
| Record | Entity | Balance | Updated |
|---|---|---|---|
| R-1 | A | 10 | 2026-09-29 |
| R-2 | A | 10 | 2026-09-29 |
| R-3 | B | 20 | 2026-09-29 |
| R-4 | B | 25 | 2026-09-30 |
| Scenario | Included records | Balance total | Review rule |
|---|---|---|---|
| Raw | R-1, R-2, R-3, R-4 | 65 | All supplied rows |
| Remove exact duplicate only | R-1, R-3, R-4 | 55 | R-2 repeats R-1 |
| Latest per entity | R-1, R-4 | 35 | Requires approved current-balance policy |
Method
Compare source IDs, entity key, values and update dates. Removing an exact duplicate differs from resolving conflicting entity versions. The latest-per-entity scenario assumes these are snapshots, not additive transactions.
What the result tells you
Totals of 65, 55 and 35 demonstrate policy sensitivity. None is an approved correction. Repeated legitimate transactions must remain additive; source rows stay unchanged.
Take the next step
Make it your investigation.
Copy the prompt above, create or open your Workspace, and paste it into Chat. Add the inputs you have permission to use and adapt the assumptions with your agent.
Start exploringYour prompt is carried by copy and paste. Signing up does not submit it or start this work automatically.
Or start with your own agent
The setup prompt includes this public task. Your agent will help you connect when needed and use the access you approve.