uniq
uniq是删除重复内容
1、查看uniq的帮助
-bash-3.2# uniq --help
用法:uniq [选项]... [输入 [输出]]
Discard all but one of successive identical lines from INPUT (or
standard input), writing to OUTPUT (or standard output).
长选项必须用的参数在使用短选项时也是必须的。
-c, --count prefix lines by the number of occurrences
-d, --repeated only print duplicate lines
-D, --all-repeated[=delimit-method] print all duplicate lines
delimit-method={none(default),prepend,separate}
Delimiting is done with blank lines.
-f, --skip-fields=N avoid comparing the first N fields
-i, --ignore-case ignore differences in case when comparing
-s, --skip-chars=N avoid comparing the first N characters
-u, --unique only print unique lines
-w, --check-chars=N compare no more than N characters in lines
--help 显示此帮助信息并退出
--version 输出版本信息并退出
2、只显示唯一的行
-bash-3.2# cat 1.txt
1:a:tail
2:b:one
3:c:less
4:d:more
4:d:more
4:d:cat
-bash-3.2# uniq -u 1.txt
1:a:tail
2:b:one
3:c:less
4:d:cat
3、统计次数
-bash-3.2# cat 1.txt |uniq -c
1 1:a:tail
1 2:b:one
1 3:c:less
2 4:d:more
1 4:d:cat
4、查找出重复的行
-bash-3.2# cat 1.txt |uniq -d
4:d:more
5、只显示第一行
-bash-3.2# cat 1.txt
1:a:tail
2:b:one
3:c:less
4:d:more
4:d:more
4:d:cat
-bash-3.2# uniq -f 1 1.txt
1:a:tail
6、指定跳过前N个字符和指定比较的最大字符
-bash-3.2# cat 1.txt
1:a:tail
2:b:one
3:c:less
4:d:more
4:d:more
4:d:cat
-bash-3.2# sort 1.txt |uniq -s 1 -w 2
1:a:tail
2:b:one
3:c:less
4:d:cat