Sample Header Ad - 728x90

Remove all duplicate word from string using shell script

17 votes
12 answers
41084 views
I have a string like "aaa,aaa,aaa,bbb,bbb,ccc,bbb,ccc" I want to remove duplicate word from string then output will be like "aaa,bbb,ccc" I tried This code Source $ echo "zebra ant spider spider ant zebra ant" | xargs -n1 | sort -u | xargs It is working fine with same value,but when I give my variable value then it is showing all duplicate word also. How can I remove duplicate value. **UPDATE** My question is adding all corresponding value into a single string if user is same .I have data like this -> user name | colour AAA | red AAA | black BBB | red BBB | blue AAA | blue AAA | red CCC | red CCC | red AAA | green AAA | red AAA | black BBB | red BBB | blue AAA | blue AAA | red CCC | red CCC | red AAA | green In coding I fetch all distinct user then I concatenate color string successfully .For that I am using code - while read the records if [ "$c" == "" ]; then #$c I defined global c="$colour1" else c="$c,$colour1" fi When I print this $c variable i get the output (For User AAA) "red,black,blue,red,green,red,black,blue,red,green," I want to remove duplicate color .Then desired output should be like "red,black,blue,green" For this desired output i used above code echo "zebra ant spider spider ant zebra ant" | xargs -n1 | sort -u | xargs but it is displaying the output with duplicate values .Like "red,black,blue,red,green,red,black,blue,red,green," Thanks
Asked by Urvashi (343 rep)
Mar 23, 2017, 12:41 PM
Last activity: Aug 3, 2022, 11:27 PM