Sorting two columns with sort and uniq

Question

I have a file that needs sorting in a specific way but I can't figure out how to do it.

I've tried the following command but it doesn't give me what I want:

sort -t" " -k1,1 Original_File | uniq -d > Desired_Output

Original_File:

User1 US
User1 NG
User2 US
User3 US
User4 US
User4 EN
User5 US

Desired_Output:

User1 US
User1 NG
User4 US
User4 EN

Exactly what I put. I want it to show if a "User" has two country codes and then output the users that do. — Roboman1723, Aug 27 '14 at 16:21
So actually you want to find repeated occurrences of User 1? — Tim, Aug 27 '14 at 16:23
@KasiyA other than having the words User3 US, and being asked by him, it is not duplicated in any way. How to find the difference between... is totally different to How to find duplicates from a file. — Tim, Aug 27 '14 at 16:30
@Tim Ok you are right[edited question]. I have deleted above comment. — αғsнιη, Aug 27 '14 at 16:41
Is the order of User4 US and User4 EN important? Could User4 US be the last line? — Volker Siegel, Aug 27 '14 at 16:46

score 3 · Accepted Answer · answered Aug 27 '14 at 16:21

3

You can extract the first column, pick up the duplicates, and grep them back from the file again:

cut -f1 -d' ' Original_File | sort | uniq -d | grep -wFf- Original_File

answered Aug 27 '14 at 16:21

choroba

What does the -wf- do in grep? I know capital 'F' is fgrep – Roboman1723 Aug 27 '14 at 16:22
-w is force PATTERN to match only whole words and -f is obtain PATTERN from FILE. Just there under grep --help – Tim Aug 27 '14 at 16:28
w works for whole words (so User1 doesn't select User11), and f- tells grep to read the patterns from the file -, which is stdin. – choroba Aug 27 '14 at 16:29
This gives me exactly what I wanted! Thanks a lot – Roboman1723 Aug 27 '14 at 16:30

Volker Siegel · Answer 2 · 2014-08-27T17:00:30.130

The example input and output was updated; the examples in the first section use the original example input to show detail examples on variants:

Original input in Original_File:

User1 US
User1 NG
User2 US
User3 US
User4 US
User5 US

You can skip the userN part for uniq with the option -f to skip leading fields - separated by space:

$ sort -t" " -k1,1 Original_File | uniq -f 1 
User1 NG
User1 US

For the same order as in the example output, you can reverse the sort - this will change the "label" values of the unique lines:

$ sort -r -t" " -k1,1 Original_File | uniq -f 1
User5 US
User1 NG

Note the User5 in the first result line. If that's not acceptable, just sort again:

$ sort -t" " -k1,1 Original_File | uniq -f 1 | sort  -t" " -k1,1 -r
User1 US
User1 NG

If the UserN part is not separated by space, but has a fixed length, you can skip it for uniq with the option -s:

$ sort -t" " -k1,1 Original_File | uniq -s 6
User1 NG
User1 US

With the updated example input, this is the command for creating the required sort order:

$ sort -t" " -k1,1 Original_File | uniq -f 1 | sort -t" " -k1,1  -k2,2r
User1 US
User1 NG
User4 US
User4 EN

it sorts the second field to reverse order.

Just pipped me to the post! About to answer that the same way. — Tim, Aug 27 '14 at 16:31
Nearly, but he doesn't want it reversed, he wants it same order as the original. I have edited to clarify. — Tim, Aug 27 '14 at 16:34
He doesn't, but I think it is a reasonable assumption that there will be multiple languages. — Tim, Aug 27 '14 at 17:22

2 Answers2