Pass parameters to the kernel at boot

Some settings only apply at the very start of the boot, before any service runs: they go through the kernel command line. It is the case of a hardening such as audit=1, of a hardware workaround, or of switching to rescue mode. This guide shows how to add such a parameter persistently, on RHEL and derivatives as on Debian and Ubuntu, and above all why the method you read everywhere, editing /etc/default/grub then regenerating, fails silently on the Red Hat family. All outputs come from a test on AlmaLinux 10.2, real reboot included.

What you will learn

• Read the current kernel command line with /proc/cmdline
• Add a persistent parameter with grubby (RHEL) or update-grub (Debian)
• Understand what BLS is and why it changes the method
• Prove the result after a real reboot
• Remove a parameter cleanly, without leaving a trace

Prerequisites

• root (or sudo) access on a disposable VM, never a production machine for a first try
• Knowing how to reboot a machine cleanly
• A fallback console access (cloud console, hypervisor): a faulty parameter can prevent the boot

RHCSA and LFCS skill

The RHCSA objectives (EX200, RHEL 10) ask to "Modify the system bootloader", and state that any configuration must persist after reboot without intervention: a parameter typed in the GRUB menu therefore does not meet the requirement. The LFCS asks to "Configure kernel parameters, persistent and non-persistent", in its Operations Deployment domain.

Read the current command line

Before changing anything, look at what the kernel has really received. This information lives in a virtual file, rebuilt at each boot:

cat /proc/cmdline

BOOT_IMAGE=(hd0,gpt3)/vmlinuz-6.12.0-211.34.1.el10_2.x86_64
root=UUID=1f5fce98-2902-4ac3-b784-b4f10857f44e ro console=tty0
console=ttyS0,115200n8 no_timer_check biosdevname=0 net.ifnames=0

It is the only source of truth. Everything you write in configuration files only matters if the parameter ends up appearing here, after a reboot. Keep this reflex: at the end of any operation, you reread /proc/cmdline.

The trap: four locations, only one counts

On RHEL 8 and later, AlmaLinux and Rocky, the kernel command line can appear in four different files, and only one is read at boot. This is what makes the topic confusing.

The cause is BLS, the Boot Loader Specification, enabled by default, as the GRUB_ENABLE_BLSCFG variable shows:

grep '^GRUB_ENABLE_BLSCFG' /etc/default/grub

GRUB_ENABLE_BLSCFG=true

Under BLS, each installed kernel has its own entry file under /boot/loader/entries/, and it is the options line of this file that is actually read:

ls /boot/loader/entries/

553eb47a42cb4c30822993b5a845ff06-6.12.0-211.34.1.el10_2.x86_64.conf
8516cfdc80ee46979f9d40b76524701f-6.12.0-211.7.3.el10_2.x86_64.conf

Two installed kernels, hence two files, each with its own parameter line. Any method that does not write to these files will change nothing at the next boot.

`/etc/default/grub` + `grub2-mkconfig` does not touch the installed kernels

It is the method found in almost every tutorial, and it fails silently on this family of distributions. Measured on AlmaLinux 10.2:

sudo sed -i 's/^GRUB_CMDLINE_LINUX="\(.*\)"$/GRUB_CMDLINE_LINUX="\1 audit=1"/' /etc/default/grub
sudo grub2-mkconfig -o /boot/grub2/grub.cfg

The command exits with rc=0, everything looks normal. Here is, however, where the parameter ended up:

Location	audit=1 present?	Read at boot?
/etc/default/grub	yes, you just wrote it	no, it is only a template
/boot/grub2/grub.cfg (kernelopts fallback)	yes	no, ignored when BLS is active
/boot/loader/entries/*.conf	no	yes, it is the only one that counts
/etc/kernel/cmdline	no	no, template for future kernels

Swipe to see more

grubby --info=ALL | grep -c 'audit=1' returns 0: no boot entry carries the parameter. At reboot, /proc/cmdline is unchanged, and nothing explains why.

Add a parameter, the right method

The command differs according to the distribution family. Choose the right tab, the two are not interchangeable.

RHEL / AlmaLinux / Rocky / Fedora

The tool that acts on already installed kernels is grubby, present out of the box.

1. Add the parameter to every installed kernel.

   sudo grubby --update-kernel=ALL --args="audit=1"

   The command says nothing when it succeeds: it is normal, it returns 0 silently.

2. Check what was written, before any reboot.

   sudo grubby --info=ALL | grep '^args'

   args="ro console=tty0 console=ttyS0,115200n8 no_timer_check biosdevname=0
   net.ifnames=0 $tuned_params audit=1"

   The parameter does appear at the end of the line, on each entry.

3. Reboot, then prove the result.

   sudo systemctl reboot

   grep -o 'audit=1' /proc/cmdline

   audit=1

`ALL` does more than its name says

grubby --update-kernel=ALL writes to three locations at once, which is precisely what you want: the BLS entries (for the current kernels), /etc/default/grub and /etc/kernel/cmdline (for future ones). A single move therefore covers today's kernels and tomorrow's.

The other targets are narrower, and it is sometimes useful:

Target	Scope
--update-kernel=ALL	every installed kernel and the templates for future ones
--update-kernel=DEFAULT	only the entry of the default kernel
--update-kernel=/boot/vmlinuz-<version>	a single kernel, named by its path

Swipe to see more

Checked: --update-kernel=DEFAULT touches only the default entry, and leaves /etc/default/grub intact. Handy to test a parameter on a single kernel while keeping the others as a safety net.

Debian / Ubuntu

No BLS here, so the classic method is the right one: the /etc/default/grub file directly feeds the generated entries.

1. Edit the GRUB_CMDLINE_LINUX variable.

   sudo nano /etc/default/grub

   GRUB_CMDLINE_LINUX="audit=1"

   Two variables coexist: GRUB_CMDLINE_LINUX applies to every entry, GRUB_CMDLINE_LINUX_DEFAULT only to normal entries, not to rescue mode. For a hardening, use the first.

2. Regenerate the configuration.

   sudo update-grub

   Generating grub configuration file ...
   Found linux image: /boot/vmlinuz-6.8.0-136-generic
   done

3. Reboot, then check in /proc/cmdline.

   sudo systemctl reboot
   grep -o 'audit=1' /proc/cmdline

grubby does not exist on Debian and Ubuntu (checked: no package in their repositories): do not look for the command there.

Understand /etc/kernel/cmdline

This file often goes unnoticed, although it explains why editing /etc/default/grub ends up counting anyway on RHEL.

cat /etc/kernel/cmdline

root=UUID=1f5fce98-2902-4ac3-b784-b4f10857f44e ro console=tty0
console=ttyS0,115200n8 no_timer_check biosdevname=0 net.ifnames=0

It is the template applied to future kernels. When a new kernel is installed, the /usr/lib/kernel/install.d/20-grub.install script (grub2-common package on AlmaLinux 10) reads it to compose the options line of the new BLS entry, resynchronising it from /etc/default/grub if the latter is more recent.

The consequence is useful to know: a parameter written only in /etc/default/grub will have no effect today, but will appear at the next installed kernel. It is this delay that gives the impression that "it ended up working", weeks later, after an update.

What you want	What to write
The parameter now, on the installed kernels	grubby --update-kernel=ALL
The parameter on future kernels	/etc/default/grub, which feeds /etc/kernel/cmdline
Both, the most common case	grubby --update-kernel=ALL, which does everything

Swipe to see more

Check that a parameter is taken into account

A parameter can reach the kernel without the kernel understanding it, for example because of a typo or an option removed from one version to the next. The kernel reports it at boot:

journalctl -k -b | grep -i 'unknown kernel command line'

kernel: Unknown kernel command line parameters "BOOT_IMAGE=(hd0,gpt3)/vmlinuz-…
biosdevname=0", will be passed to user space.

Read this line carefully before worrying: it is normal on most machines. BOOT_IMAGE is set by GRUB, and biosdevname is handled by user space, not by the kernel. The mention will be passed to user space indicates that these parameters are not lost, simply passed on.

What should alert you is your own parameter in this list: it means the kernel does not know it, and that it will do nothing.

Two installed kernels, a single one running

The topic holds one more confusion. A machine hosts several kernels, and the one running is not necessarily the one grubby designates by default:

sudo grubby --default-kernel
uname -r

Measured on the test machine before reboot: the default kernel was 6.12.0-211.34.1, whereas uname -r answered 6.12.0-211.7.3. A more recent kernel was installed, but never booted for lack of a reboot.

It is one more reason to prefer --update-kernel=ALL: with DEFAULT, you risk configuring an entry that is not the one you observe.

Remove a parameter

Removal follows the same logic as adding, and it must be checked everywhere.

RHEL / AlmaLinux / Rocky

sudo grubby --update-kernel=ALL --remove-args="audit=1"

Check the three locations grubby had touched:

sudo grubby --info=ALL | grep -c 'audit=1'
grep -c 'audit=1' /etc/default/grub /etc/kernel/cmdline

The three counters must fall back to 0. If one resists, fix it by hand: grubby only cleans what it wrote itself, and a parameter added earlier with the editor may remain.

Debian / Ubuntu

Remove the parameter from the variable in /etc/default/grub, then regenerate:

sudo nano /etc/default/grub
sudo update-grub

In both cases, the removal only takes effect after a reboot: the command line of a running kernel cannot be changed.

Test without breaking anything

On a machine you care about, there is a method without persistence: editing the entry in the GRUB menu. At boot, select the entry, press e, add the parameter at the end of the linux line, then boot with Ctrl+X. The change only applies to that boot.

It is the right reflex to validate that a parameter has the expected effect before making it permanent. Note however two limits: this editing requires console access, and it is blocked if a GRUB password has been set, which is precisely what is recommended in production.

An intermediate variant consists in changing only one kernel, keeping the other intact as a safety net:

sudo grubby --update-kernel=/boot/vmlinuz-$(uname -r) --args="test=1"

If the machine no longer boots, the GRUB menu lets you choose the other kernel, left healthy.

Troubleshooting

These symptoms cover most of what goes wrong, sorted by what you observe. Start from the left column and check the cause before applying the fix.

Symptom	Likely cause	Solution
The parameter does not appear in /proc/cmdline after reboot	On RHEL, /etc/default/grub + grub2-mkconfig does not write to the BLS entries	sudo grubby --update-kernel=ALL --args="…"
grubby: command not found	You are on Debian or Ubuntu, where the tool does not exist	Edit /etc/default/grub then sudo update-grub
The parameter appears on one kernel but not the other	--update-kernel=DEFAULT only touches one entry	Use ALL
Unknown kernel command line parameters "<yours>" at boot	Typo, or option missing from this kernel version	Check the spelling and the documentation of the installed kernel
The parameter appears weeks later, without intervention	It was only in /etc/default/grub, and a new kernel was installed	Normal behaviour of /etc/kernel/cmdline
The machine no longer boots after adding it	Parameter incompatible with the hardware or the storage	In the GRUB menu, choose another kernel, or press e and remove the parameter for this boot
Cannot edit the entry in the GRUB menu	A GRUB password protects editing	Authenticate with the declared superuser, see the GRUB hardening guide

Swipe to see more

Key points

• /proc/cmdline is the only proof: everything else is just configuration.
• On RHEL, AlmaLinux and Rocky, BLS is active: it is the options line of /boot/loader/entries/*.conf that is read at boot.
• /etc/default/grub + grub2-mkconfig does not touch the installed kernels on these distributions. The command returns 0, and nothing changes.
• The right tool there is grubby --update-kernel=ALL --args="…", which writes at once to the BLS entries, /etc/default/grub and /etc/kernel/cmdline.
• On Debian and Ubuntu, no BLS: GRUB_CMDLINE_LINUX then update-grub is the right method, and grubby does not exist.
• /etc/kernel/cmdline is the template of future kernels: it is what makes a forgotten setting end up applying after an update.
• A parameter unknown to the kernel is reported by Unknown kernel command line parameters in journalctl -k -b.
• To test without risk, edit the entry in the GRUB menu (not persistent), or change only one kernel and keep the other as a fallback.

Knowledge check

Check that the essentials of this guide stuck. The questions cover only what has just been explained here.

Knowledge check

Check what you have learned with this interactive quiz

6 questions

6 min

70% to pass

How it works

• The timer starts when you click Start
• Multiple choice, true/false and short answer questions
• You can move between questions
• Detailed results are shown at the end

Start the quiz

Starts the quiz and the timer

×

See in the glossary →