Understanding Pipes and Redirection For the Linux Command Line

Two powerful features of the Linux command line shell are redirection and pipes which allow the output (or even input) of a program to be sent to a file or another program. You may have already used this features without being aware of it. Whenever you have used the “>” sign in a command or “|” then you have used redirection or a pipe, respectively.

On all Unix-like operating systems, like Linux and FreeBSD, the output from a command line program automatically goes to a place known as standard output (stdout). By default, standard out is the screen (the console) but that can be changed using pipes and redirection. Likewise the keyboard is considered the standard input (stdin) and as with standard out, it can be changed.

Pipes allow you to funnel the output from one command into another where it will be used as the input. In other words, the standard output from one program becomes the standard input for another.

The “more” command takes the standard input and paginates it on the standard output (the screen). This means that if a command displays more information than can be shown on one screen, the “more” program will pause after the first screen full (page) and wait for the user to press SPACE to see the next page or RETURN to see the next line.

Here is an example which will list all the files, with details (-la) in the /dev directory and pipe the output to more. The /dev directory should have dozens of files and hence ensure that more needs to paginate.

ls -la /dev | more

ls pipe more

Notice the --More-- prompt at the bottom of the screen. Press SPACE to see the next page and keep pressing SPACE until the output is finished.

Here is another pipe example, this time using the “wc” (word count) tool.

ls -1 /dev | wc

wc counts the numbers of lines, words and characters in the standard input. If you use the -l parameter it will display only the number of lines, which is good way to see how many files are in a directory!

The In Depth Look at Linux’s Archiving and Compression Commands tutorial has an example using tar and 7-Zip together:

tar cvf - * | 7zr a -si somefiles.tar.7z

In this case the hyphen after the f option tells tar to send its output to the standard out and not to a file. The output from tar will be fed down the pipe into 7zr which is waiting for input from standard in due to the -si option.

Redirection is similar to pipes except using files rather than another program. The standard output for a program is the screen. Using the > (greater than) symbol the output of a program can be sent to a file. Here is a directory listing of /dev again but this time redirected to a file called listing.txt

ls -la > listing.txt

There won’t be anything displayed on the terminal as everything was sent to the file. You can take a look at the file using the cat command (which can be piped into more) or for convenience you can just use the more command on its own:

more listing.txt

If listing.txt had already existed, it will be overwritten. But you can append to an existing file using >> like this:

ls -la /home > listing.txt
ls -la /dev >> listing.txt

The first redirection will overwrite the file listing.txt while the second will append to it.

The cat command can be used to create a file using redirection, for example:

 cat > atextfile.txt

Now whatever text you type will be sent to the file atextfile.txt until you press Control-D, at which point the file will be closed and you will be returned to the command prompt. If you want to add more text to the file use the same command but with two greater than signs (>>).

Many of Linux command line programs are designed to work with redirection and pipes, try experimenting with them and see how they interact. For example the output of the ps command, which lists the current processes, can be piped into grep. See if you can work out how to list the processes owned by root.