Deduplicate Lines
Remove duplicate lines, keep only duplicates, or count occurrences. Order is preserved.
This tool removes duplicate lines from any list while preserving the original order: the first occurrence of each line stays exactly where it was. Everything runs in your browser, so email lists, log files, and exported data never leave your device. Beyond plain deduplication, it can also isolate only the lines that repeat, or count how often each line appears.
How to use
- Paste your list into the input on the left (or click Paste to insert your clipboard, Sample to load demo data). Each line is one item.
-
Pick an output mode: Unique keeps the
first occurrence of every line,
Only duplicates lists each line that
appears more than once, and Counted
prefixes every line with its occurrence count (like
3× apple). - Tune the matching rules with the checkboxes: Ignore case, Trim whitespace, and Skip empty lines.
- Copy or download the result from the right panel. The label above it tells you how many lines were removed or repeated.
Worked example
Paste this list with the default options:
Apple
apple
banana
banana
Apple
With Trim whitespace on and
Ignore case off, you get three lines
(Apple, apple, banana)
because " banana" is trimmed before comparing and
merges with banana, while Apple and
apple still count as different lines. Turn on
Ignore case and the output shrinks to
Apple and banana: lowercase
apple is now a duplicate, and the kept line uses
the first occurrence's capitalization. In
Counted mode (case still ignored) you get
3× Apple and 2× banana.
Notes and edge cases
- Order is preserved. Lines come out in the order you pasted them, first occurrence winning.
- Trimming affects the output. When the trim option is on, the result contains the trimmed versions of your lines, not the originals with their leading or trailing spaces.
- Only duplicates is handy for log analysis: paste a log excerpt to see which messages recur, one instance each. Counted turns the same list into a quick frequency report.
- Counted output stays in original order. To rank by frequency, paste the result into the sort tool and use numeric sorting.
- With Skip empty lines off, blanks are deduplicated like any other line, and the first blank is kept.
-
Windows (
CRLF) and Unix (LF) line endings are handled automatically.
FAQ
More questions? Browse the full FAQ.
Related tools
After deduplicating, sort the lines alphabetically or numerically to produce a clean, ordered set. Numeric sort is perfect for ranking Counted output by frequency. Or run your cleaned list through the word and character counter to check its size before sending or publishing.