Validate

Detect near-duplicate records, typos, and slight spelling variations using Levenshtein distance

Identify subtle duplicates and typos in your CSV using configurable fuzzy matching algorithms.

🛡️ Your rows never leave your browser.
Why Onfyt: Uses Web Worker candidate blocking to run fuzzy comparisons across thousands of rows without crashing the browser tab.

Interactive Sample Data

idcustomer_nameemail
1Jonathan Smithjsmith@example.com
2Jonathon Smithj.smith@example.com
3Grace Kerubograce@corp.com
4Grace Kirubograce.k@corp.com

How this tool works

Exact match deduplication tools fail whenever records contain minor typographical errors, transposed letters, missed vowels, or varied spacing. A customer named 'Jonathon Smith' will escape an exact deduplicator looking for 'Jonathan Smith', resulting in duplicate outreach emails, split support tickets, and degraded reporting accuracy. Fuzzy matching evaluates similarity using edit-distance metrics such as Levenshtein distance, Damerau-Levenshtein transposition analysis, and token sort ratios. By computing the minimum number of single-character edits (insertions, deletions, substitutions) required to transform one string into another, fuzzy algorithms assign a normalized similarity score between 0% and 100%. Pairwise comparison across large datasets requires quadratic O(N²) comparisons, which can freeze ordinary browser tabs. DataFix avoids this bottleneck by implementing candidate blocking and n-gram indexing inside a background Web Worker. Candidate pairs are evaluated against your chosen similarity threshold (typically 80% to 90%), grouping matching rows into cluster IDs for swift human review or automated merging.

Things that usually go wrong

  • A threshold that is too low (60%) flags too many false positives. Start at 85% and lower only if needed.
  • Fuzzy matching is slow for large columns (>10,000 unique values).

Edge cases we tested

🧪 Transposed adjacent characters ('recieve' vs 'receive')

Detected with high similarity using Damerau-Levenshtein calculation.

🧪 Inverted word order ('Smith, John' vs 'John Smith')

Matched correctly when token-based sorting is enabled.

🧪 Very short strings (3 characters or less)

Guarded against false-positive clustering via minimum length thresholds.

Frequently Asked Questions

What algorithm is used?

Levenshtein distance normalised to a 0–100% similarity score.

Can I deduplicate automatically?

Not automatically — you review the pairs and choose which to keep. Automated merging of similar strings risks losing distinct entities.

What is Levenshtein distance?

Levenshtein distance counts the minimum number of single-character edits (insertions, deletions, substitutions) needed to change one string into another. 'Acme Corp' to 'ACME Corporation' has a higher edit distance than 'Acme Corp' to 'Acme Corp.'.

How long does fuzzy matching take on large datasets?

Fuzzy matching is O(n²) — it compares every pair of values. A column with 1,000 unique values produces 500,000 comparisons. This takes 2–10 seconds in-browser depending on your machine.

Related tools

Validate

Validate CSV

Audit CSV files for structural errors, encoding problems, and row-length inconsistencies.

Open Tool →
Validate

Validate JSON

Parse and validate JSON strings to expose syntax errors at exact line and column positions.

Open Tool →
Validate

Find Duplicate Rows

Identify and count duplicate rows in CSV files by exact match or by key column selection.

Open Tool →