Uniq command is mostly used in combination with sort command, as uniq removes duplicates only from a sorted file. i.e In order for uniq to work, all the duplicate entries should be in the adjacent lines. Following are some common examples.
1. Remove Duplicates from a file using uniq command
When you have an employee file with duplicate entries, you can do the following to remove duplicates.
$ sort namesd.txt | uniq $ sort –u namesd.txt
2. Display Duplicate Row Count Using Uniq
If you want to know how many lines are duplicates, do the following. The first field in the following examples indicates how many duplicates where found for that particular line. So, in this example the lines beginning with Alex and Emma were found twice in the namesd.txt file.
$ sort namesd.txt | uniq –c
2 Alex Jason:200:Sales
2 Emma Thomas:100:Marketing
1 Madison Randy:300:Product Development
1 Nisha Singh:500:Sales
1 Sanjay Gupta:400:Support
3. Display Only Duplicate Rows Using Uniq
The following displays only the entries that are duplicates.
$ sort namesd.txt | uniq –cd
2 Alex Jason:200:Sales
2 Emma Thomas:100:Marketing
Comments on this entry are closed.
…and suppose you are interested in only the first field of a | delimited file – as that is where the key is.
Hi All,
i want to merge two files kun3.txt & kun1.txt and i am able achive this using paste command.
But i want the will contained same account name in column 2 and 4
OUTPUT SEE 4th line is not match with account name some one can help me it fix this.
i want column 2 and column 4 will contained same account name
Finale out put
37009 prod 29449 prod
4929 wamcopnt 4920 wamcopnt
3182 bcrshpnt 2687 pimcopnt
2687 pimcopnt 1402 baram1
Out put of file kun3.txt
XXXXXXXX # head kun3.txt
37009 prod
4929 wamcopnt
3182 bcrshpnt
2687 pimcopnt
2372 brsindex
1955 jpmorganpnt
1578 putnampnt
1402 baram1
1356 imsi3
1191 nlipnt
Out put of file kun1.txt
XXXXXXXX # head kun1.txt
29449 prod
4920 wamcopnt
2687 pimcopnt
1402 baram1
1191 nlipnt
1058 putnampnt
1002 deshawnylct
970 campbscs
916 dbnypnt
912 trwprpnt
XXXXXX#
Out put of file paste command
xxxxxxxxx # paste /tmp/kun3.txt /tmp/kun1.txt | pr -t -e10 | head
37009 prod 29449 prod
4929 wamcopnt 4920 wamcopnt
3182 bcrshpnt 2687 pimcopnt
2687 pimcopnt 1402 baram1
2372 brsindex 1191 nlipnt
1955 jpmorganpnt 1058 putnampnt
1578 putnampnt 1002 deshawnylct
1402 baram1 970 campbscs
1356 imsi3 916 dbnypnt
1191 nlipnt 912 trwprpnt