How do I remove duplicate lines from a list?

Paste one item per line and open the Keep unique tab. It keeps the first occurrence of each line, and you can copy the result or download it as .txt.

Can it ignore capital letters and extra spaces?

Yes. Turn on Ignore case, Trim whitespace and Ignore empty lines so that lines differing only in case or spacing count as the same.

Can I load a file instead of pasting text?

Yes, a plain text file such as .txt, .csv, .log or .md. It is read in your browser, not uploaded.

Yes. Every tool is free to use, with no account and no credit card. The work runs on your device rather than on our servers, which is what keeps it free to run.

Does it work on mobile?

Yes. It runs in current mobile browsers on iOS and Android, with nothing to install.

🔍

Free Duplicate Line Finder

Find, count, or remove duplicate lines from a list. Great for cleaning up email lists, audit logs, or finding repeat entries in data dumps.

Runs in your browserNo sign-up, no pop-upsNo login requiredFree forever

Works offline once loaded

This tool finds duplicate lines and CSV rows in text, not duplicate files or photos. To check whether two files are identical, use the file hash checker

Options

Ignore caseTrim whitespaceIgnore empty lines

Input format

Sort

Count format

Input (one item per line)

Occurrence counts

Output will appear here…

Common uses

Dedupe Email ListsClean up email lists before importing into Mailchimp, SendGrid, or other ESPs.
Clean Data Before AnalysisFind duplicate rows in CSV exports or database dumps before analytics work.
Identify Repeats in To-Do ListsSpot duplicate tasks in long to-do lists or project trackers.
Count Word FrequencyPaste a list of words/keywords to see which appear most often.
Find Repeated Commit MessagesPaste a git log to find authors' habitual commit message patterns.

How It Works

Duplicate detection runs entirely in your browser using a hand-written JavaScript Map that counts every line in a single pass — no library, no server, no upload. Results update live as you type.

1
Line splitting and normalization
The tool splits your text on newlines (handling both \r\n and \n), then optionally trims leading/trailing whitespace from each line and drops empty lines — controlled by the checkboxes you set before processing.
2
Single-pass Map counting
Each line is fed into a JavaScript Map keyed by its normalized form (lowercased when ignore-case is on). The Map accumulates an occurrence count per unique key in one linear pass, preserving the first-seen original text.
3
Mode-specific output generation
Depending on the selected tab — Count, Unique, Duplicates Only, or Highlighted — the Map's values are filtered or sorted (by frequency, alphabetically, or first-occurrence order) and formatted as plain text or an inline color-coded view.
4
Copy or download the result
The processed text can be copied to your clipboard, exported as a .txt file, or (in Count mode) copied as CSV with one row per value and its occurrence count — ready to paste into a spreadsheet.

Related Tools

🧹

Duplicate File & Photo Finder

Find duplicate files and similar photos in a folder: exact copies by content, plus resized or re-saved versions of the same photo. Keep the best copy and move the rest aside. Runs in your browser; nothing is uploaded.

Try itFiles stay local

😀