Change one line in multiple files with this Linux command

With this simple command you can change the text of one line occurring in multiple text files, all in one simple operation.

Some time ago we had some issues with high traffic on one of our old web servers. The Apache HTTP server had hundreds of virtual hosts in it, and if we could figure out what virtual host that had all the traffic, we should be able to solve the issue quite quickly. The only problem was that each vhost config file had this line in it:

CustomLog /dev/null combined

For some crazy reason, we thought it was a good idea not to do any logging from our virtual hosts. If we could only comment out this line in all the vhost config files, instead of editing one by one.

Well, it turned out it was possible with this simple command (after cd'ing to the dir with all the config files in it):

sed -i 's/^CustomLog \/dev\/null combined/#CustomLog \/dev\/null combined/' *

This command uses sed to search for the line "CustomLog /dev/null combined" at the beginning of each line in all files in the current directory and replaces it with a commented out version of the line by adding a "#" at the beginning of the line.

However, in our case, it didn't work the first time because the line was indented with a tab, but that was easy to fix as well:

sed -i 's/^\tCustomLog \/dev\/null combined/\t#CustomLog \/dev\/null combined/' *

Warning: This command will change all the files in the current directory, without the ability to undo. To make sure you can undo any mistakes, make a backup of that directory before running the sed command. You can also start by just changing one file first, then check if the change worked as intended before proceeding to the rest of the files:

$ ls -l
file1.conf
file2.conf
file3.conf
$ sed -i 's/^\tCustomLog \/dev\/null combined/\t#CustomLog \/dev\/null combined/' 1.conf
$ cat 1.conf

That way, if you make a mistake, you will only need to fix one file.

Other uses of the sed command

sed is a powerful command-line tool for manipulating text. Here are some examples of how you can use it:

Replace a word in a file

sed -i 's/old_word/new_word/g' filename

This command replaces all occurrences of old_word with new_word in filename. The g flag is used to replace all occurrences in the file, not just the first one.

Replace a word in all files in a folder

sed -i 's/old_word/new_word/g' folder/*

This command replaces all occurrences of old_word with new_word in all files in the folder directory.

Delete lines containing a word in a file

sed -i '/word/d' filename

This command deletes all lines containing word from filename.

Add a line after a specific line in a file

sed -i '/specific_line/a new_line' filename

This command adds new_line after the line that matches specific_line in filename.

Delete the first line of a file

sed -i '1d' filename

This command deletes the first line of filename.

Delete the last line of a file

sed -i '$d' filename

This command deletes the last line of filename.

Append text to the end of a line that matches a pattern

sed -i '/pattern/s/$/ new_text/' filename

This command finds the first line in filename that matches pattern, and appends new_text to the end of that line.

Replace a word in a specific line of a file

sed -i '3s/old_word/new_word/' filename

This command replaces the first occurrence of old_word with new_word in the third line of filename.

Delete the lines between two patterns

sed -i '/start_pattern/,/end_pattern/d' filename

This command deletes all lines between the first line that matches start_pattern and the first line that matches end_pattern in filename.

Replace a word only if it appears at the beginning of a line

sed -i 's/^old_word/new_word/' filename

This command replaces old_word with new_word only if it appears at the beginning of a line in filename.

Remove a line from a file without leaving a gap

sed -i '/pattern/d' filename

This command deletes all lines that match pattern from filename, and removes them without leaving a gap. If the deleted line is in the middle of the file, the lines below it will be moved up to fill the gap.

Note that if you want to remove multiple lines that match pattern, you can use the g flag to delete all occurrences of the pattern:

sed -i '/pattern/dg' filename

This command deletes all lines that match pattern from filename, and removes them without leaving a gap. The g flag tells sed to repeat the d command on every line that matches pattern, so all occurrences of the pattern will be removed.

Escape / when using sed

If the text string you want to change contains /, you can use a different delimiter in the sed command instead of /. This is useful because if you use / as the delimiter, it can conflict with the / in the text string you want to change.

Here's an example of how to use a different delimiter (in this case, |) to change a text string that contains /:

sed -i 's|http://example.com/old_path|http://example.com/new_path|g' filename

In this example, sed uses | as the delimiter instead of /. The text string to be changed is http://example.com/old_path, which contains /. By using | as the delimiter, sed can differentiate between the / in the text string and the / that separates the pattern and replacement in the s command.

Note that you can use any character as the delimiter, as long as it does not appear in the pattern or replacement strings.

Common situations where the sed command is useful

The sed command is a powerful tool for text manipulation, and there are many situations where it can be useful. Here are some common scenarios where sed is often used:

Search and replace

You can use sed to search for a pattern in a file and replace it with another string. This is useful for making quick changes to large files or scripts.

Editing configuration files

You can use sed to modify configuration files in place. For example, you can change the value of a parameter in a configuration file by using sed to replace the old value with a new one.

Extracting data

You can use sed to extract specific lines or fields from a text file. For example, you can extract only the lines that contain a specific keyword or extract a specific field from a CSV file.

Removing unwanted characters

You can use sed to remove unwanted characters from a file. For example, you can remove all tabs or spaces from a text file or remove any non-alphanumeric characters from a string.

Processing log files

You can use sed to process log files and extract useful information, such as the number of requests, errors, or warnings in a log file.

Batch editing

You can use sed to edit multiple files at once. For example, you can replace a string in all files in a directory or remove all lines that contain a specific keyword from a set of files.

Overall, sed is a versatile tool that can be used for many different text manipulation tasks. It is particularly useful for automating repetitive tasks or making quick edits to large files.

Similar commands and alternatives to sed

awk

awk is a command-line tool for processing and manipulating text files. It can be used to perform similar tasks as sed, such as searching for patterns, extracting specific fields, and replacing text.

grep

grep is a command-line tool for searching for patterns in text files. It can be used to search for specific lines or fields that match a pattern, and can also be combined with other commands such as sed or awk to perform more complex text manipulation tasks.

perl

perl is a programming language that can be used for text manipulation. It has many built-in functions and libraries for working with text files, and can be used for tasks such as searching and replacing text, extracting specific fields, and processing log files.

tr

tr is a command-line tool for translating or deleting characters in a text file. It can be used to remove unwanted characters or replace specific characters with other characters.

cut

cut is a command-line tool for extracting specific columns or fields from a text file. It can be used to extract specific fields from a CSV file or extract specific columns from a log file.