Transform files with sed

sed is the tool you use when you do not want to open an editor. You have 50 configuration files to update, a password to mask, a hostname to switch from prod to staging across every file of a directory: sed does it in one line, in a scriptable and repeatable way.

It directly serves the RHCSA objective "Create and edit text files", and extends "Use grep and regular expressions to analyze text". The current LFCS curriculum does not name it, but its performance-based tasks constantly ask for this kind of configuration change.

What you will learn

• Substitute text with the s/before/after/ command and its variants
• Edit a file directly on disk with sed -i without losing the original
• Target specific lines with numeric addresses or regex patterns
• Delete lines, comments and empty lines
• Chain several transformations in a single command with -e
• Reorder fields with \1, \2 capture groups
• Apply a complete config file migration workflow

In which context?

sed is the default tool as soon as you must transform text in an automatable, non-interactive way:

• change the database URL in db.conf during a prod → staging migration
• mask a password or an API key in config files before committing them
• remove comments and empty lines from a configuration file to feed it to a parser
• normalise the spaces around = in dozens of .ini or .conf files
• extract only the error lines of a log to feed a report

In a deployment script, sed -i is often the first solution: simple, dependency-free, available on every Linux system. In an Ansible playbook, prefer the lineinfile and replace modules, which only modify the file when needed and report it: a sed -i run by Ansible rewrites the file on every run, without saying whether it changed anything.

The three datasets of this guide

Every output on this page was measured with GNU sed 4.9 on Debian 13, from the three files below. Create them in a working directory to reproduce each command. hosts.old mimics a production /etc/hosts file, db.conf a configuration with irregular spacing, and access.log an excerpt of a web log.

cat > hosts.old <<'EOF'
192.168.1.10  web01.prod.example.com   web01
192.168.1.11  web02.prod.example.com   web02
192.168.1.20  db01.prod.example.com    db01
10.0.0.5      bastion.prod.example.com bastion
EOF

cat > db.conf <<'EOF'
# Database configuration
host = db.internal.example.com
port = 5432
user=admin
password=  secret123
database=production
max_connections = 100
# timeout in seconds
connect_timeout=   30
EOF

cat > access.log <<'EOF'
192.168.1.10 - alice [01/Apr/2026:08:15:02 +0000] "GET /index.html HTTP/1.1" 200 1043
10.0.0.5 - bob [01/Apr/2026:08:16:44 +0000] "POST /login HTTP/1.1" 401 219
192.168.1.10 - alice [01/Apr/2026:08:17:30 +0000] "PUT /api/orders/7 HTTP/1.1" 500 0
EOF

The s command: substitute text

The s command is the heart of sed. Its syntax:

s/PATTERN/REPLACEMENT/FLAGS

• PATTERN: regular expression (BRE by default, ERE with -E)
• REPLACEMENT: the replacement text
• FLAGS: g (global), i (case-insensitive), 2 (second occurrence only)

Basic substitution, first occurrence per line

The minimal form replaces prod with staging in hosts.old and sends the result to standard output, without touching the file. Each line contains only one prod: the substitution therefore changes all of them.

sed 's/prod/staging/' hosts.old

192.168.1.10  web01.staging.example.com   web01
192.168.1.11  web02.staging.example.com   web02
192.168.1.20  db01.staging.example.com    db01
10.0.0.5      bastion.staging.example.com bastion

Without a flag, sed replaces only the first occurrence on each line, then moves to the next line. In a file where the word appears twice per line, the second occurrence stays untouched.

The /g flag, every occurrence

The g (global) flag applies the substitution to every occurrence of each line. The shortest demonstration uses echo, with the same word three times on a single line.

echo "prod prod prod" | sed 's/prod/staging/g'

staging staging staging

When to use /g?

In configuration files where the searched word appears only once per line (hostname, URL, parameter), /g is not required. But in logs or free text, forget /g and you will miss occurrences, without any message.

Edit in place with sed -i

Without -i, sed sends the result to standard output, and the original file stays intact: it is a filter, like grep. With -i (in-place), the change is written directly into the file. The right reflex: run the command without -i, read the output, then run it again with -i.

cp hosts.old hosts.staging
sed -i 's/prod/staging/g' hosts.staging

Create an automatic backup with -i.bak

The extension passed right after -i becomes the suffix of the backup file. On GNU sed, it sticks to the option (-i.bak), without a space:

sed -i.bak 's/prod/staging/g' hosts.staging

ls -1
# hosts.staging       ← modified file
# hosts.staging.bak   ← copy of the original

sed -i rewrites the file without a safety net

On GNU sed (Linux), -i without an extension edits the file directly, without a backup. A malformed expression does no harm: sed stops before writing (measured with GNU sed 4.9: unterminated s command error, exit code 1, file intact). The real risk is a valid but wrong substitution, which rewrites the file with no way back. Always use -i.bak in production, or work on a copy first.

Addresses: target specific lines

By default, sed applies the command to every line. An address limits the action to a selection.

Numeric addresses

A numeric address designates a line by its number, a range by two numbers separated by a comma, and the last line by $. This $ has nothing to do with the end-of-line anchor of regex: its position, before the command, gives it this meaning.

# Edit only line 2
sed '2s/example/EXAMPLE/' hosts.old

# Edit lines 1 to 2
sed '1,2s/example/EXAMPLE/' hosts.old

# Edit the last line
sed '$s/example/EXAMPLE/' hosts.old

The last command changes only the bastion line:

192.168.1.10  web01.prod.example.com   web01
192.168.1.11  web02.prod.example.com   web02
192.168.1.20  db01.prod.example.com    db01
10.0.0.5      bastion.prod.EXAMPLE.com bastion

Regex addresses

A regex address, written between slashes, selects the lines that contain the pattern. It is the most useful form in administration: you target a group of servers by name rather than by a line number that changes at the slightest addition.

# Edit only the lines that contain "web"
sed '/web/s/prod/staging/' hosts.old

192.168.1.10  web01.staging.example.com   web01
192.168.1.11  web02.staging.example.com   web02
192.168.1.20  db01.prod.example.com    db01
10.0.0.5      bastion.prod.example.com bastion

The db01 and bastion lines do not match the web pattern: they come out unchanged, with their prod. Without an address, the same substitution would have changed everything.

Delete lines with d

The d (delete) command removes from the output the lines selected by the address. The ^ anchor matters: without it, /#/d would also delete a valid line followed by an end-of-line comment.

# Delete the comments and empty lines of a config file
sed '/^#/d; /^$/d' db.conf

host = db.internal.example.com
port = 5432
user=admin
password=  secret123
database=production
max_connections = 100
connect_timeout=   30

The comments (# ...) are gone, and any empty line would have met the same fate. The original file is intact: add -i to write the result.

Chain several transformations with -e

Rather than chaining sed | sed | sed, pass several expressions with -e: they apply in order, in a single pass over the file. A semicolon inside one expression ('/^#/d; /^$/d') remains the equivalent for short cases.

sed -E \
  -e '/^#/d' \
  -e '/^$/d' \
  -e 's/[[:space:]]*=[[:space:]]*/=/' \
  db.conf

host=db.internal.example.com
port=5432
user=admin
password=secret123
database=production
max_connections=100
connect_timeout=30

Three transformations applied in one pass:

1. Removal of the comments
2. Removal of the empty lines
3. Normalisation of the spaces around =: [[:space:]]* means zero or more spaces, the * quantifier being detailed in the note that follows

Quantifiers: *, +, ? and {n,m}, and what -E changes

A quantifier goes right after a character, a class such as [0-9] or a group, and says how many times that element may repeat. They give the expressions that follow their meaning:

Quantifier	Effect	Example	Matches
*	zero or more times	ab*c	ac, abc, abbc
+	one or more times	[0-9]+	7, 192: at least one digit
?	zero or one time, the element is optional	https?	http and https
{n,m}	between n and m times	[0-9]{1,3}	one to three digits

Swipe to see more

They are greedy: they take the longest possible run. The -E option turns on extended regular expressions (ERE), where +, ?, {n,m} and | are written as is. In BRE, the default mode, they are written \+, \? and \{n,m\}, and a bare + is a literal character. The full difference is in the Basic regular expressions guide.

Capture groups and backreferences

With -E, parentheses (...) create capture groups, numbered from left to right. The backreferences \1, \2 inject the captured content back into the replacement, which lets you reorder fields without going through awk.

# Swap IP and hostname in hosts.old
sed -E 's/^([0-9.]+)([[:space:]]+)([a-z0-9.]+)/\3\2\1/' hosts.old

web01.prod.example.com  192.168.1.10   web01
web02.prod.example.com  192.168.1.11   web02
db01.prod.example.com  192.168.1.20    db01
bastion.prod.example.com      10.0.0.5 bastion

• ([0-9.]+) → group 1: one or more digits or dots, hence the IP
• ([[:space:]]+) → group 2: one or more spaces or tabs, hence the separator
• ([a-z0-9.]+) → group 3: one or more lowercase letters, digits or dots, hence the hostname

Without the +, each class takes a single character: ^[0-9.] stops at the 1 of 192.168.1.10, the next character is not a space, the pattern no longer matches and sed leaves the line unchanged, without reporting any error.

In the replacement, \3\2\1 injects back the hostname, the separator and the IP, in the new order. The captured separator keeps its original width, hence the irregular alignment of the output.

Complete workflow: patch a config file for a migration

Here is the complete workflow of a migration from production to staging: a copy of the original, a single sed pass with a backup, then two checks. Each step protects against a different kind of error.

# 1. Make a safety copy
cp db.conf db.conf.orig

# 2. Apply every transformation in a single pass
sed -E -i.bak \
  -e 's/host[[:space:]]*=[[:space:]]*.*/host=db.staging.example.com/' \
  -e 's/database[[:space:]]*=[[:space:]]*.*/database=staging/' \
  -e 's/[[:space:]]*=[[:space:]]*/=/' \
  -e '/^#/d' \
  -e '/^$/d' \
  db.conf

# 3. Check the result
cat db.conf

# 4. Check the difference with the original
diff db.conf.orig db.conf

Result of cat db.conf:

host=db.staging.example.com
port=5432
user=admin
password=secret123
database=staging
max_connections=100
connect_timeout=30

Result of the diff:

1,3c1,2
< # Database configuration
< host = db.internal.example.com
< port = 5432
---
> host=db.staging.example.com
> port=5432
5,9c4,7
< password=  secret123
< database=production
< max_connections = 100
< # timeout in seconds
< connect_timeout=   30
---
> password=secret123
> database=staging
> max_connections=100
> connect_timeout=30

The diff after sed -i is your mandatory check before shipping the file: it shows exactly what changed, and immediately reveals a too broad substitution or a pattern that matched nothing.

Extract lines with -n and p

By default, sed prints every line, even unmodified ones. The -n option suppresses this automatic printing, and the p command explicitly prints the selected lines. Combined, they reproduce a grep. Forget -n, and the matching lines come out twice: on this 3-line excerpt, the command prints 4.

# Print only the lines with status 500 in access.log
sed -n '/" 500 /p' access.log

192.168.1.10 - alice [01/Apr/2026:08:17:30 +0000] "PUT /api/orders/7 HTTP/1.1" 500 0

Frequent traps

These traps have one thing in common: sed almost never reports an error. A substitution that finds nothing, a regex written in ERE run in BRE, a dot that matches too broadly: the command succeeds, and only the result gives the problem away. Hence the importance of the final diff.

Trap	Symptom	Solution
Forgetting /g	Only the first occurrence is replaced	Add the /g flag
sed -i without a backup	Original file lost in case of mistake	Use -i.bak systematically
Mixing up BRE and ERE	+, ? not recognised without a backslash	Add -E for ERE
Slash / in the pattern	unknown option to `s'	Use another delimiter: s|/old/path|/new/path|g
Unescaped dot .	Matches too broadly	Escape it: \. for a literal dot
Editing the file shipped by the package	Lost at the next package update	Write the change in an override file (/etc/<service>/conf.d/, or systemctl edit for a systemd unit)

Swipe to see more

Change the delimiter

If your pattern contains / (paths, URLs), change the delimiter rather than escaping every slash:

# Change a path in a configuration file
sed -i 's|/var/log/app|/data/logs/app|g' supervisord.conf

The character after s becomes the delimiter: |, #, @ all work.

Key points

• s/before/after/g: global substitution on each line
• -i.bak: in-place edit with a backup, always in production
• Numeric address 2s/... or regex address /pattern/s/... to target lines
• /pattern/d: delete the matching lines
• -e: chain several commands in one pass
• Quantifiers: * zero or more times, + one or more, ? optional, {n,m} between n and m times
• -E: turn on ERE for +, ?, {n,m}, (...) capture groups
• \1, \2: inject a captured group back into the replacement
• -n ... p: print only the selected lines (like grep)
• Check with diff after every sed -i on a config file

Knowledge check

Check that the essentials of this guide stuck. The questions cover only what has just been explained here.

Knowledge check

Check what you have learned with this interactive quiz

6 questions

6 min

70% to pass

How it works

• The timer starts when you click Start
• Multiple choice, true/false and short answer questions
• You can move between questions
• Detailed results are shown at the end

Start the quiz

Starts the quiz and the timer

×

See in the glossary →