Highlight duplicate rows in CSV without deleting them
Identify and count duplicate rows in CSV files by exact match or by key column selection.
🛡️ Your rows never leave your browser.
Why Onfyt: Annotates rows with duplicate group IDs and counts in-place rather than silently deleting, preserving your ability to choose which copy to keep.
Interactive Sample Data
| id | name | |
|---|---|---|
| 1 | Grace Kerubo | grace@example.com |
| 2 | Peter Njoroge | peter@example.com |
| 3 | Grace Kerubo | grace@example.com |
How this tool works
Deduplication and duplicate detection are different tasks. Deduplication silently keeps one copy and deletes the rest. Duplicate detection marks every occurrence so a data steward can inspect each group before deciding which record is canonical — the one with the most recent update timestamp, the highest completeness score, or the most reliable source system. Auditing duplicates before acting on them is mandatory when the same customer appears under two IDs with different transaction histories attached to each. DataFix adds a `_duplicate_count` column and a `_duplicate_group` marker to every row without altering your original data, so you can download the annotated file, filter it in Excel, and make deletion decisions manually. Duplicate records in data warehouses and analytics pipelines distort metric calculations, inflate revenue numbers, and trigger duplicate billing or communication events. Identifying duplicate rows requires examining either the entire row across all dimensions or specific composite keys (such as email and signup date). DataFix scans your dataset using hash-indexed comparison in Web Workers, isolating exact row duplicates and near-duplicates. Grouping identical records by key column identifies duplicate customers prior to sending transactional billing emails.
Things that usually go wrong
- Trailing whitespace creates false negatives — two cells that look identical may differ by a space. Run Trim Whitespace first.
- Case differences ('Alice' vs 'alice') create separate groups — normalise case first.
Edge cases we tested
🧪 Case-insensitive email duplicates ('GRACE@corp.com' vs 'grace@corp.com')
Matched when case-insensitive comparison is enabled.
🧪 Duplicates separated by 10,000 rows in large file
In-memory hash map detects match regardless of row distance.
🧪 Three-way duplicate with three distinct IDs
All three tagged with same group ID; count shows 3.
Frequently Asked Questions
Does this keep or remove duplicates?
By default it flags them with a column. Choose 'Remove duplicates' to keep only the first occurrence of each group.
Can I compare all columns?
Yes — leave the column selector empty to compare every column.
Related tools
Validate
Validate CSV
Audit CSV files for structural errors, encoding problems, and row-length inconsistencies.
Open Tool →
Validate
Validate JSON
Parse and validate JSON strings to expose syntax errors at exact line and column positions.
Open Tool →
Validate
Find Missing Values
Audit CSV columns for empty, null, and whitespace-only cells and produce a completeness report.
Open Tool →