Skip to content
gohacks.dev
Tooling

Sort, Dedupe and Clean Lines

Sort lines alphabetically or naturally, remove duplicates, keep only duplicated lines, trim, drop empty lines or number them.

Free · No sign-up · Your inputs stay in your browser

Input

Sort, dedupe and clean up lines

Output

Result

7 lines in, 7 out

Sorting uses Intl.Collator, so accented letters sort next to their base letter, unlike a plain byte sort such as LC_ALL=C sort.

Runs in this browser. Nothing you paste is sent to a server.

The shell equivalents

Every operation here has a one-line shell form: sort, sort -r, sort -V for natural order, sort -u or awk '!seen[$0]++' to remove duplicates while keeping order, sort | uniq -d for duplicated lines, and nl or cat -n to number them.

Locale changes the order

GNU sort follows the current locale, so the same file sorts differently under LC_ALL=C, which compares bytes, and under en_US.UTF-8, which compares letters the way a dictionary does. This tool sorts the dictionary way with Intl.Collator. When two scripts disagree about the order of a sorted file, a different locale is the usual cause.

Share

Frequently asked questions

What is a natural sort?
A sort that compares runs of digits as numbers, so web-2 comes before web-10. A plain alphabetical sort puts web-10 first because 1 sorts before 2.
Does removing duplicates keep the original order?
Yes. The first occurrence of each line stays where it was and later copies are removed, like awk '!seen[$0]++'.
Is what I paste sent anywhere?
No. The tool runs entirely in your browser with JavaScript. Nothing you type or paste is sent to our server or stored.

Keep planning

Similar tools