Change one line in multiple files with this Linux command
With this simple command you can change the text of one line occurring in multiple text files, all in one simple operation.
Some time ago we had some issues with high traffic on one of our old web servers. The Apache HTTP server had hundreds of virtual hosts in it, and if we could figure out what virtual host that had all the traffic, we should be able to solve the issue quite quickly. The only problem was that each vhost config file had this line in it:
CustomLog /dev/null combined
For some crazy reason, we thought it was a good idea not to do any logging from our virtual hosts. If we could only comment out this line in all the vhost config files, instead of editing one by one.
Well, it turned out it was possible with this simple command (after cd'ing to the dir with all the config files in it):
sed -i 's/^CustomLog \/dev\/null combined/#CustomLog \/dev\/null combined/' *
This command uses sed to search for the line "CustomLog /dev/null combined" at the beginning of each line in all files in the current directory and replaces it with a commented out version of the line by adding a "#" at the beginning of the line.
However, in our case, it didn't work the first time because the line was indented with a tab, but that was easy to fix as well:
sed -i 's/^\tCustomLog \/dev\/null combined/\t#CustomLog \/dev\/null combined/' *
Warning: This command will change all the files in the current directory, without the ability to undo. To make sure you can undo any mistakes, make a backup of that directory before running the sed command. You can also start by just changing one file first, then check if the change worked as intended before proceeding to the rest of the files:
$ ls -l
$ sed -i 's/^\tCustomLog \/dev\/null combined/\t#CustomLog \/dev\/null combined/' 1.conf
$ cat 1.conf
That way, if you make a mistake, you will only need to fix one file.
Other uses of the sed command
sed is a powerful command-line tool for manipulating text. Here are some examples of how you can use it:
Replace a word in a file
sed -i 's/old_word/new_word/g' filename
This command replaces all occurrences of old_word with new_word in filename. The g flag is used to replace all occurrences in the file, not just the first one.
Replace a word in all files in a folder
sed -i 's/old_word/new_word/g' folder/*
This command replaces all occurrences of old_word with new_word in all files in the folder directory.
Delete lines containing a word in a file
sed -i '/word/d' filename
This command deletes all lines containing word from filename.
Add a line after a specific line in a file
sed -i '/specific_line/a new_line' filename
This command adds new_line after the line that matches specific_line in filename.
Delete the first line of a file
sed -i '1d' filename
This command deletes the first line of filename.
Delete the last line of a file
sed -i '$d' filename
This command deletes the last line of filename.
Append text to the end of a line that matches a pattern
sed -i '/pattern/s/$/ new_text/' filename
This command finds the first line in filename that matches pattern, and appends new_text to the end of that line.
Replace a word in a specific line of a file
sed -i '3s/old_word/new_word/' filename
This command replaces the first occurrence of old_word with new_word in the third line of filename.
Delete the lines between two patterns
sed -i '/start_pattern/,/end_pattern/d' filename
This command deletes all lines between the first line that matches start_pattern and the first line that matches end_pattern in filename.
Replace a word only if it appears at the beginning of a line
sed -i 's/^old_word/new_word/' filename
This command replaces old_word with new_word only if it appears at the beginning of a line in filename.
Remove a line from a file without leaving a gap
sed -i '/pattern/d' filename
This command deletes all lines that match pattern from filename, and removes them without leaving a gap. If the deleted line is in the middle of the file, the lines below it will be moved up to fill the gap.
Note that if you want to remove multiple lines that match pattern, you can use the g flag to delete all occurrences of the pattern:
sed -i '/pattern/dg' filename
This command deletes all lines that match pattern from filename, and removes them without leaving a gap. The g flag tells sed to repeat the d command on every line that matches pattern, so all occurrences of the pattern will be removed.
Escape / when using sed
If the text string you want to change contains /, you can use a different delimiter in the sed command instead of /. This is useful because if you use / as the delimiter, it can conflict with the / in the text string you want to change.
Here's an example of how to use a different delimiter (in this case, |) to change a text string that contains /:
sed -i 's|http://example.com/old_path|http://example.com/new_path|g' filename
In this example, sed uses | as the delimiter instead of /. The text string to be changed is http://example.com/old_path, which contains /. By using | as the delimiter, sed can differentiate between the / in the text string and the / that separates the pattern and replacement in the s command.
Note that you can use any character as the delimiter, as long as it does not appear in the pattern or replacement strings.
Common situations where the sed command is useful
The sed command is a powerful tool for text manipulation, and there are many situations where it can be useful. Here are some common scenarios where sed is often used:
Search and replace
You can use sed to search for a pattern in a file and replace it with another string. This is useful for making quick changes to large files or scripts.
Editing configuration files
You can use sed to modify configuration files in place. For example, you can change the value of a parameter in a configuration file by using sed to replace the old value with a new one.
You can use sed to extract specific lines or fields from a text file. For example, you can extract only the lines that contain a specific keyword or extract a specific field from a CSV file.
Removing unwanted characters
You can use sed to remove unwanted characters from a file. For example, you can remove all tabs or spaces from a text file or remove any non-alphanumeric characters from a string.
Processing log files
You can use sed to process log files and extract useful information, such as the number of requests, errors, or warnings in a log file.
You can use sed to edit multiple files at once. For example, you can replace a string in all files in a directory or remove all lines that contain a specific keyword from a set of files.
Overall, sed is a versatile tool that can be used for many different text manipulation tasks. It is particularly useful for automating repetitive tasks or making quick edits to large files.
Similar commands and alternatives to sed
awk is a command-line tool for processing and manipulating text files. It can be used to perform similar tasks as sed, such as searching for patterns, extracting specific fields, and replacing text.
grep is a command-line tool for searching for patterns in text files. It can be used to search for specific lines or fields that match a pattern, and can also be combined with other commands such as sed or awk to perform more complex text manipulation tasks.
perl is a programming language that can be used for text manipulation. It has many built-in functions and libraries for working with text files, and can be used for tasks such as searching and replacing text, extracting specific fields, and processing log files.
tr is a command-line tool for translating or deleting characters in a text file. It can be used to remove unwanted characters or replace specific characters with other characters.
cut is a command-line tool for extracting specific columns or fields from a text file. It can be used to extract specific fields from a CSV file or extract specific columns from a log file.