Uniq

Last updated
uniq
Original author(s) Ken Thompson
(AT&T Bell Laboratories)
Developer(s) Various open-source and commercial developers
Initial releaseFebruary 1973;50 years ago (1973-02)
Written in C
Operating system Unix, Unix-like, Plan 9, Inferno, MSX-DOS, IBM i
Platform Cross-platform
Type Command
License coreutils: GPLv3+
Plan 9: MIT License
Website man7.org/linux/man-pages/man1/uniq.1.html

uniq is a utility command on Unix, Plan 9, Inferno, and Unix-like operating systems which, when fed a text file or standard input, outputs the text with adjacent identical lines collapsed to one, unique line of text.

Contents

Overview

The command is a kind of filter program. Typically it is used after sort. It can also output only the duplicate lines (with the -d option), or add the number of occurrences of each line (with the -c option). For example, the following command lists the unique lines in a file, sorted by the number of times each occurs:

$ sort file | uniq -c | sort -n 

Using uniq like this is common when building pipelines in shell scripts.

History

First appearing in Version 3 Unix, [1] uniq is now available for a number of different Unix and Unix-like operating systems. It is part of the X/Open Portability Guide since issue 2 of 1987. It was inherited into the first version of POSIX and the Single Unix Specification. [2]

The version bundled in GNU coreutils was written by Richard Stallman and David MacKenzie. [3]

A uniq command is also part of ASCII's MSX-DOS2 Tools for MSX-DOS version 2. [4]

The command is available as a separate package for Microsoft Windows as part of the GnuWin32 project [5] and the UnxUtils collection of native Win32 ports of common GNU Unix-like utilities. [6]

The uniq command has also been ported to the IBM i operating system. [7]

See also

Related Research Articles

grep is a command-line utility for searching plain-text data sets for lines that match a regular expression. Its name comes from the ed command g/re/p, which has the same effect. grep was originally developed for the Unix operating system, but later available for all Unix-like systems and some others such as OS-9.

In Unix and Unix-like operating systems, chmod is the command and system call used to change the access permissions and the special mode flags of file system objects. Collectively these were originally called its modes, and the name chmod was chosen as an abbreviation of change mode.

ls Command to list files and directories in Unix and Unix-like operating systems

In computing, ls is a command to list computer files and directories in Unix and Unix-like operating systems. It is specified by POSIX and the Single UNIX Specification.

basename is a standard computer program on Unix and Unix-like operating systems. When basename is given a pathname, it will delete any prefix up to the last slash ('/') character and return the result. basename is described in the Single UNIX Specification and is primarily used in shell scripts.

uname Standard UNIX utility that prints name and other details about the machine

uname is a computer program in Unix and Unix-like computer operating systems that prints the name, version and other details about the current machine and the operating system running on it.

chown Command used on Unix-like systems to change the owner of a file

The command chown, an abbreviation of change owner, is used on Unix and Unix-like operating systems to change the owner of file system files, directories. Unprivileged (regular) users who wish to change the group membership of a file that they own may use chgrp.

In computing, touch is a command used to update the access date and/or modification date of a computer file or directory. It is included in Unix and Unix-like operating systems, TSC's FLEX, Digital Research/Novell DR DOS, the AROS shell, the Microware OS-9 shell, and ReactOS. The command is also available for FreeDOS and Microsoft Windows.

patch (Unix)

The computer tool patch is a Unix program that updates text files according to instructions contained in a separate file, called a patch file. The patch file is a text file that consists of a list of differences and is produced by running the related diff program with the original and updated file as arguments. Updating files with patch is often referred to as applying the patch or simply patching the files.

join is a command in Unix and Unix-like operating systems that merges the lines of two sorted text files based on the presence of a common field. It is similar to the join operator used in relational databases but operating on text files.

tr (Unix) Unix text formatting utility

tr is a command in Unix, Plan 9, Inferno, and Unix-like operating systems. It is an abbreviation of translate or transliterate, indicating its operation of replacing or removing specific characters in its input data set.

wc (Unix) Unix command utility

wc is a command in Unix, Plan 9, Inferno, and Unix-like operating systems. The program reads either standard input or a list of computer files and generates one or more of the following statistics: newline count, word count, and byte count. If a list of files is provided, both individual file and total statistics follow.

rm (Unix) Unix command utility

rm is a basic command on Unix and Unix-like operating systems used to remove objects such as computer files, directories and symbolic links from file systems and also special files such as device nodes, pipes and sockets, similar to the del command in MS-DOS, OS/2, and Microsoft Windows. The command is also available in the EFI shell.

tail is a program available on Unix, Unix-like systems, FreeDOS and MSX-DOS used to display the tail end of a text file or piped data.

In computing, tee is a command in command-line interpreters (shells) using standard streams which reads standard input and writes it to both standard output and one or more files, effectively duplicating its input. It is primarily used in conjunction with pipes and filters. The command is named after the T-splitter used in plumbing.

In computing, sleep is a command in Unix, Unix-like and other operating systems that suspends program execution for a specified time.

sort (Unix) Standard UNIX utility

In computing, sort is a standard command line program of Unix and Unix-like operating systems, that prints the lines of its input or concatenation of all files listed in its argument list in sorted order. Sorting is done based on one or more sort keys extracted from each line of input. By default, the entire input is taken as sort key. Blank space is the default field separator. The command supports a number of command-line options that can vary by implementation. For instance the "-r" flag will reverse the sort order.

In computing, which is a command for various operating systems used to identify the location of executables. The command is available in Unix and Unix-like systems, the AROS shell, for FreeDOS and for Microsoft Windows. The functionality of the which command is similar to some implementations of the type command. POSIX specifies a command named command that also covers this functionality.

dirname is a standard computer program on Unix and Unix-like operating systems. When dirname is given a pathname, it will delete any suffix beginning with the last slash ('/') character and return the result. dirname is described in the Single UNIX Specification and is primarily used in shell scripts.

od is a command on various operating systems for displaying ("dumping") data in various human-readable output formats. The name is an acronym for "octal dump" since it defaults to printing in the octal data format.

pr is a command on various operating systems that is used to paginate or columnate computer files for printing. It can also be used to compare two files side by side, as an alternative to diff.

References

  1. McIlroy, M. D. (1987). A Research Unix reader: annotated excerpts from the Programmer's Manual, 1971–1986 (PDF) (Technical report). CSTR. Bell Labs. 139.
  2. uniq   Shell and Utilities Reference, The Single UNIX Specification , Version 4 from The Open Group
  3. uniq(1)    Linux General Commands Manual
  4. MSX-DOS2 Tools User's Manual by ASCII Corporation
  5. CoreUtils for Windows
  6. Native Win32 ports of some GNU utilities
  7. IBM. "IBM System i Version 7.2 Programming Qshell" (PDF). Retrieved 2020-09-05.