Remove Duplicate Lines
Text Tools
Strip duplicate lines from any text. Case-sensitive matching, whitespace trimming, keep-first or keep-last, and optional sorting.
Runs entirely in your browserAbout Remove Duplicate Lines
Deduplicating lines is one of those small chores that comes up constantly — cleaning up a list of emails, deduping log entries, normalizing a CSV column, or finding the unique items in any line-separated dataset. Doing it by hand is tedious; doing it with a one-liner on the command line works but isn't always handy.
This tool gives you the four options that actually matter: case sensitivity (treat `Apple` and `apple` as the same or different), whitespace trimming (so ` apple` and `apple` match), keep-first or keep-last (which occurrence wins when there are duplicates — matters for ordered data), and optional alphabetical sorting. A live readout shows how many lines were kept and how many removed, so you can confirm the operation did what you expected. Everything runs in your browser.
How to use
- 1
Paste your lines
Drop the text into the input area. One item per line.
- 2
Set the comparison rules
Toggle Case sensitive to differentiate `Apple` from `apple`. Toggle Trim whitespace to ignore leading/trailing spaces. Together these decide what counts as a duplicate.
- 3
Pick keep direction
Keep First preserves the order of first appearance. Keep Last keeps the last occurrence — useful when later entries supersede earlier ones.
- 4
Optionally sort alphabetically
Toggle Sort to alphabetize the output after deduplication.
- 5
Grab the result
Use Copy or Download (.txt). The label shows X kept · Y removed.
Examples
Case-insensitive deduplication
With case sensitivity off, Apple and apple are the same item.
Input
apple
Banana
apple
APPLE
bananaOutput
apple
BananaKeep last, sorted
Last occurrence wins; output is alphabetized.
Input
bravo
alpha
charlie
alpha
deltaOutput
alpha
bravo
charlie
deltaFrequently asked questions
Does it handle very large files?+
Yes — tens of thousands of lines deduplicate instantly. For millions of lines you might notice a brief pause as the browser processes the input.
How is order preserved?+
When Keep First is selected (default), the first occurrence wins and the original order is maintained. Keep Last reverses this. Toggle Sort to alphabetize afterward.
What happens to empty lines?+
Empty lines are treated like any other value — duplicates of empty lines collapse to one empty line. If you don't want any empty lines in the output, trim them first.
Can I deduplicate by a substring?+
Not in this tool — it compares whole lines. For substring-based deduplication (e.g. unique email addresses from a 'Name <email>' list), pre-process the data first.
Is my text sent anywhere?+
No. All processing happens in your browser.
Related tools
Word Counter
Count characters, words, sentences, and paragraphs with live reading-time estimates.
Case Converter
Convert text between upper, lower, title, camelCase, snake_case, kebab-case, and more.
Text Diff Checker
Compare two pieces of text and highlight differences inline or side by side.
Lorem Ipsum Generator
Generate placeholder text in classic, hipster, or corporate dialects.