Clean

Remove repeated lines from any block of text

Remove repeated lines from any text block, with options for case sensitivity and whitespace handling.

🛡️ Your rows never leave your browser.

Why Onfyt: Trim-before-compare is separate from the kept value — leading/trailing spaces on the retained line are preserved exactly, avoiding silent data changes.

Interactive Sample Data

id	line
1	nairobi
2	mombasa
3	nairobi

How this tool works

Log files, keyword lists, DNS zone exports, and code-generated outputs frequently contain duplicate lines that clutter analysis. A server access log may repeat the same IP address line 500 times. A compiled keyword list may merge several sources, each containing the same 20 seed terms. The engine reads the pasted text line by line and builds a set of seen lines. When a line matches one already in the set, it is skipped. The first-encountered occurrence is kept by default. Options available through your instruction: - **Keep first** (default): keeps the first time each line appears. - **Keep last**: scans the full input first, then keeps only the final occurrence — useful when later lines are more up-to-date. - **Case-insensitive**: 'Apple' and 'apple' and 'APPLE' are treated as the same line. - **Trim whitespace before comparing**: leading and trailing spaces are stripped before the comparison, so ' apple ' and 'apple' match — but the kept value is the original (un-trimmed) text. - **Sort after deduplication**: the result is sorted A–Z after removing duplicates. All processing runs inside your browser tab.

Things that usually go wrong

Lines with invisible Unicode whitespace (non-breaking space) look identical but won't match standard spaces — use 'trim whitespace' mode to normalise them.
Tab-separated lines where only the tab width differs visually will still be treated as different lines — tabs are significant characters.
If you paste CSV and want row-level deduplication, use the 'Remove Duplicate Rows' tool instead — this tool treats each CSV row as a flat string.

Edge cases we tested

🧪 Blank lines treated as duplicates

Multiple consecutive blank lines are collapsed to one blank line (blank is a valid line value).

🧪 Lines differing only in leading spaces (' apple' vs 'apple')

Kept as distinct by default; treated as duplicates only when 'trim whitespace' is specified.

🧪 Keep-last mode with 5 occurrences of the same line

The fifth (final) occurrence is kept; the first four are dropped.

🧪 Case-insensitive with mixed case ('Apple', 'APPLE', 'apple')

Compared in lowercase; first occurrence ('Apple') is kept, the other two dropped.

🧪 Single-line input (no duplicates possible)

Returned unchanged — no lines removed.

Frequently Asked Questions

Does this preserve the original line order?

Yes — lines appear in the order they were first seen. Add 'sorted A–Z' to your instruction if you want alphabetical order instead.

Can I remove duplicates from a CSV column rather than free text?

This tool works on lines. For CSV column deduplication, use 'Remove Duplicate Rows' and specify the key column.

What counts as a 'line'?

Any sequence of characters terminated by a newline (\n, \r\n, or \r). Blank lines and whitespace-only lines are valid lines.

Will it handle very large inputs?

Yes — the engine runs inside your browser tab and handles several megabytes of text without issue.

Related tools

Format

Sort Lines Alphabetically

Sort any block of text lines alphabetically, reverse alphabetically, or numerically.

Open Tool →

Clean

Remove Empty Lines

Remove all blank and whitespace-only lines from any block of text.

Open Tool →

Clean

Remove Extra Spaces

Collapse multiple consecutive spaces into one and strip leading/trailing spaces from any text.

Open Tool →

Clean

Remove Duplicate Rows

Eliminate duplicate rows across your entire dataset or based on specific unique key columns.

Open Tool →