How to Use Sed in Linux for Basic Shell Tasks

Sed can make your life easier. Learn how to use it.

Sed Basics 00 Featured Image

Sed is a simple UNIX program. It does not create and edit files. It only modifies the data that passes through its input and presents the modified data on its output. Despite that, sed is considered to be one of the most powerful UNIX utilities that you will encounter in Linux, especially when it is combined with other programs in your system.

This article aims to teach the basics of how to use this tiny program. Further, it will also give a brief overview of the UNIX philosophy and how that idea is important in understanding how your computer works.

What Is the UNIX Philosophy?

The UNIX philosophy is the idea that a program should only do a single task and do it well. It encourages you to write simple programs that you can reuse for different functions instead of creating complex programs that you cannot reuse for other tasks.

A good example of this is the program cat. This is a simple utility that only takes inputs and prints it as a stream of text.

Sed Basics 03 Cat Sample 2

The UNIX philosophy allows you to use the program in conjunction with other utilities in the system. In turn, this approach ultimately creates a single interface for your system that you can learn once and reuse for different contexts.

For example, you can feed a cat stream to a program such as grep. This is a simple program that only does a single thing. It looks for a string of text in an input and it prints the lines where that text appears.

Sed Basics 04 Grep Pipe Sample

On the other hand, you can instead feed the same cat stream to a screen pager such as less. This, in turn, will allow you to scroll through the stream as if it is a single text file.

Sed Basics 05 Less Pipe Sample

How Does Sed Work?

With that being said, sed is a program that read a stream of text and modify it. However, sed only modifies the text stream, which means that it does not directly change the files in your computer. This can be confusing for a beginner since any change that you make in sed will not appear in the disk.

Despite that, sed is still a powerful utility. For example, you can run the following line to read the output of the printf command:

Sed Basics 08 Sed Read Command

The Basics of Using Sed

Knowing that, it is incredibly easy to use sed for basic tasks. Consider the following example:

Sed Basics 09 Read File

In this, I used the n command to print the contents of the file “hello”.

1. Selecting and Trimming Text Streams

Further, you can also tell sed to modify this stream of text. For example, you can use the 2q command to tell it to only print the first two lines of the “hello” file:

Sed Basics 10 Print First Lines 1

This can be incredibly helpful if you only want to get the first few lines of a text file. However, you can also use the p command to tell sed to only print a range of lines in a file.

Sed Basics 11 Print Middle Lines

In the example above, I used sed to print the third and fourth line of the “hello” file. You can also use the same p command to print non-adjacent lines in your text file. Consider the following example:

Sed Basics 12 Print Non Adjacent Lines

With that, I am telling sed to only print the first and fourth line of the “hello” file. This can come in handy if you want to combine multiple lines from different streams together to a single file.

For example, you can use sed with the > operator to write the modified stream to a file.

Sed Basics 13 Write Non Adjacent Lines To File

2. Removing Text

Another great function of sed is with deleting text in a stream. Unlike printing, this allows you to select the lines of text that you want sed to omit from the stream.

For example, I used the d command to delete the second and fifth line of the hello file.

Sed Basics 14 Delete Lines

One advantage of using this instead of the p command is that this allows you to only remove the line that you do not want. This is useful if you want to either clean up a large file or trim an output to only show the information that you need.

3. Adding New Text

With that, you can also use sed with itself to include new data into your text stream. For example, you can use the a command to append a line of text to the current stream:

Sed Basics 15 Add Lines

Further, it is also possible to include entire files into a text stream. In order to do this, you can use the r command which will allow you to read a single file into your current stream. To do that, you can run the following command to include the welcome file after the second line of hello.

Sed Basics 16 Add File To Line

4. Text Substitution and Stream Manipulation

One brilliant use of sed is with modifying existing streams. This is when you take a text input stream and use sed’s built-in functions to change the content of the stream.

This is arguably the most powerful function of sed. As this gives you the ability to dynamically edit data as it passes through pipes. For example, you can easily edit the output of a program as if it is a text file:

Sed Basics 17 Modify Command Output

In this example, I used the s command to substitute the word “world” with “website”. Unlike the previous commands, however, this command uses a specific syntax to work properly.

Knowing that, the general form of the s command looks something like this:

  • The first column indicates that this is a substitute command. In most cases, you will only need to use the s command to modify a text stream.
  • The second column indicates the word that you want to replace using structural regular expressions. This is a way of describing words and grammar using general rules. For example, you can describe the words “prints”, “prince” and “primer” by writing pri[n,m]...
  • The third column describes the word that you want the old string to be. Unlike the previous column, this column does not use regular expressions. Instead, you can use plain text to describe your new word.
  • The last column tells sed to apply any additional options that you need for a substitution. In that, the most common flag that you might want to use is g. This indicates that the command will replace every instance of the regex with the new string.

5. Using Text Substitution with Files

You can apply the s command to manipulate any kind of text stream that you want. This is useful not only because it allows you to modify data on the fly but also because you can use this with almost anything in the Linux command line.

For example, you can use the s command with a text file by adding the file name at the end of the command:

Sed Basics 18 Modify File Single Word

In this, I am telling sed to read the “hello” file and replace the first instance of the word “hello” with “welcome”. I can then do this for every instance of the word “hello” in the file by adding the g option:

Sed Basics 19 Modify File All Words

Frequently Asked Questions

Is it possible to write modifications to a file using sed?

Yes! Aside from using file operators, it is possible to use sed to directly write any modified text stream to a file. To do this, you can either use sed with the -i flag or use the w command to write the stream to a file.

One important thing to note, however, is that these commands do not comply with the POSIX standard. This means that these will not work if you are using a non-GNU version of sed.

I am getting an "unterminated" error whenever I use sed. Is my installation broken?

Not at all! This is an issue that happens when you do not write a sed command properly. This is common with substitution commands where you are trying to either match or replace a string with spaces.

For example, running the following command will result in an “unterminated” error: sed s/hello/welcome back/g hello.txt.

You can fix this by adding the “\” escape character before your spaces. This is a special character that tells sed to include the space when either searching or replacing text.

With that, the correct version of the command above looks something like this: sed s/hello/welcome\ back/g hello.txt.

Aside from "g", are there other flags that I can use with the "s" command?

Yes! Aside from g, you can also use the p flag to tell sed to display any modified lines.

Another flag that you can use with s is e. This allows you to run shell commands every time sed replaces a word. For example, you can run the following command to run touch every time it finds the word “hello” in the file: sed s/hello\ /touch\ /e hello.txt.

Lastly, you can also use the i flag to tell sed to be case insensitive when looking for matches. This is useful if you want to match the same word but with different case styles. For example, the following command will replace the word “hello” with “welcome” regardless of its case: sed s/HELLO/welcome/i hello.txt.

If all this talk made you curious with the Linux command line. You can read our earlier article where we talk about some of the most interesting Bash prompts that you can use today.

Image credit: Unsplash

Subscribe to our newsletter!

Our latest tutorials delivered straight to your inbox