systemctl: list, start and diagnose Linux services

systemctl is the central command to manage services on Linux. It lists, starts, stops, reloads and diagnoses everything systemd drives on a modern server. This page covers the life cycle of a service, the two ways to list services (the one most readers miss), how to write a unit and the basic diagnosis of a service that refuses to run. Audience: a beginner discovering systemctl, and an administrator tired of being fooled by an over-optimistic start.

All the outputs shown here come from a Debian 13 server with systemd 257 and nginx installed. The commands are identical on Ubuntu, Debian and RHEL; only a few enable outputs differ, and this is pointed out.

What you will learn

• List services with list-units and list-unit-files, and understand why the two lists differ
• Drive the life cycle: start, stop, restart, reload, status
• Read a status line by line and query a service with is-active, is-enabled, is-failed
• Write a custom service unit without falling into the Type=simple trap
• Diagnose a failure with journalctl and the 203/EXEC code

LFCS / RHCSA

Managing services with systemctl is a major objective of both certifications. You must be able to list, start, stop and enable a service, read its status and diagnose a failure without documentation.

The five everyday commands

Most of the daily work on services fits in five verbs. Each one acts immediately, on the system as it runs right now, and none survives a reboot of the machine: enabling at boot is a separate notion. All these commands change the state of the system, so they need root privileges through sudo, except status, which only reads.

sudo systemctl start nginx      # start now
sudo systemctl stop nginx       # stop now
sudo systemctl restart nginx    # stop then start again
sudo systemctl reload nginx     # reread the configuration without cutting
systemctl status nginx          # where do we stand?

The good habit is to chain the action and its check on the same command line. The shell only runs the second part if the first one succeeded, and is-active answers in one word:

systemctl start nginx && systemctl is-active nginx

active

The .service suffix is optional: without a suffix, systemctl appends .service by default, so systemctl start nginx and systemctl start nginx.service are strictly equivalent. The suffix becomes necessary to target another unit with the same name, a timer or a socket: systemctl status logrotate shows the service, and only systemctl status logrotate.timer shows the schedule.

Reload without cutting traffic

reload asks the service to reread its configuration while it keeps serving the requests in progress. restart kills the process then starts a new one: the open connections drop. On a production web server, the difference shows in the client logs.

systemctl reload nginx && systemctl is-active nginx

active

reload does not exist everywhere

A service supports reload only if its unit declares an ExecReload= directive. The units shipped for nginx and Apache declare it. If the service does not provide it, systemctl reload returns an error and you must fall back on restart. The command systemctl cat nginx displays the full unit and lets you check for ExecReload= before going ahead.

Read a status line by line

systemctl status is the identity card of a service: where its unit file comes from, whether it runs, since when, with which PID, and the last lines of its journal. It is the first reflex in front of a suspicious service.

systemctl status nginx

● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; preset: enabled)
     Active: active (running) since Sun 2026-07-12 16:54:11 UTC; 31s ago
 Invocation: 78240b0f696a4bc29a331a580ba0cdd5
       Docs: man:nginx(8)
    Process: 76 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
    Process: 80 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
   Main PID: 81 (nginx)
      Tasks: 17 (limit: 8616)
     Memory: 10.9M (peak: 11.7M)
        CPU: 29ms
     CGroup: /system.slice/nginx.service
             ├─81 "nginx: master process /usr/sbin/nginx -g daemon on; master_process on;"
             ├─82 "nginx: worker process"
             ├─83 "nginx: worker process"
             (... one worker process per core, 17 tasks in total ...)

Jul 12 16:54:11 srv01 systemd[1]: Starting nginx.service - A high performance web server and a reverse proxy server...
Jul 12 16:54:11 srv01 systemd[1]: Started nginx.service - A high performance web server and a reverse proxy server.

Each line answers a precise question, and the first three are usually enough to settle a diagnosis:

Line	What it tells you
Loaded	Path of the unit file in use, and whether the service is enabled at boot (enabled / disabled / masked)
Active	Current state and duration: active (running), inactive (dead), failed
Process	Commands already finished and their exit code (status=0/SUCCESS or an error)
Main PID	PID of the main process; if it is missing, nothing runs
CGroup	The real tree of processes attached to the service

Swipe to see more

The Loaded line is the one people forget to read. It gives the exact path of the loaded unit file, which saves ten minutes of editing a file that systemd does not use.

One-word answers

In a script, a full status is unusable: you need a short answer and a reliable exit code. Three commands provide it, and they work just as well by hand as in an Ansible automation or a deployment script.

systemctl is-active nginx     # active | inactive | failed | activating
systemctl is-enabled nginx    # enabled | disabled | static | masked
systemctl is-failed demo      # failed if the unit is in failure

After resetting a failed service with reset-failed, is-failed answers with the current state:

systemctl reset-failed demo && systemctl is-failed demo

inactive

List services: list-units or list-unit-files?

This is the most asked question about systemctl, and the answer is a trap: there are two lists of services, and they do not contain the same things. The command found everywhere (systemctl list-units) is precisely the one that hides part of the services of the machine.

The distinction fits in one sentence: list-units lists the units loaded in memory by systemd; list-unit-files lists the unit files installed on disk. A service installed by a package but never started does exist on disk, but systemd has not loaded it, so it does not appear in list-units. On the demonstration server, the gap is striking: 14 loaded services against 119 installed unit files.

Services loaded in memory

systemctl list-units --type=service answers the question "what is systemd handling right now?". It is the runtime view, the one you want when you inspect a live server.

systemctl list-units --type=service

  UNIT                                     LOAD   ACTIVE SUB     DESCRIPTION
  ldconfig.service                         loaded active exited  Rebuild Dynamic Linker Cache
  nginx.service                            loaded active running A high performance web server and a reverse proxy server
  systemd-journal-catalog-update.service   loaded active exited  Rebuild Journal Catalog
  systemd-journal-flush.service            loaded active exited  Flush Journal to Persistent Storage
  systemd-journald.service                 loaded active running Journal Service
● systemd-modules-load.service             loaded failed failed  Load Kernel Modules
  systemd-remount-fs.service               loaded active exited  Remount Root and Kernel File Systems
  systemd-sysctl.service                   loaded active exited  Apply Kernel Variables
  systemd-sysusers.service                 loaded active exited  Create System Users
  systemd-tmpfiles-setup-dev-early.service loaded active exited  Create Static Device Nodes in /dev gracefully
  systemd-tmpfiles-setup-dev.service       loaded active exited  Create Static Device Nodes in /dev
  systemd-tmpfiles-setup.service           loaded active exited  Create System Files and Directories
  systemd-update-done.service              loaded active exited  Update is Completed
  systemd-user-sessions.service            loaded active exited  Permit User Sessions

14 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.

Three columns carry the useful information. LOAD says whether the unit file was read correctly, ACTIVE gives the general state and SUB the detail: a service in active (exited) finished its task successfully and left no process behind, which is normal for start-up tasks. The black dot at the start of a line flags the units that need your attention, here systemd-modules-load.service in failed.

The last two lines of the output are a user manual that systemd gives you for free: --all to also see the units that are loaded but inactive, list-unit-files to see everything installed.

Services installed on disk

systemctl list-unit-files --type=service answers the question "what is installed on this machine, and will it start at boot?". It is the disk view, the one you want for an audit or to find a service you never started.

systemctl list-unit-files --type=service

UNIT FILE                                    STATE           PRESET
apt-daily-upgrade.service                    static          -
apt-daily.service                            static          -
autovt@.service                              alias           -
console-getty.service                        masked          disabled
debug-shell.service                          disabled        disabled
getty@.service                               enabled         enabled
nginx.service                                enabled         enabled
systemd-networkd.service                     disabled        enabled
systemd-pstore.service                       enabled         enabled
systemd-remount-fs.service                   enabled-runtime enabled
systemd-udevd.service                        masked          enabled
x11-common.service                           masked          enabled

119 unit files listed.

The full output counts 119 lines; the excerpt above shows one example of each state met. The STATE column is the heart of the matter: it says what systemd will do with this service at the next boot. The PRESET column shows what the distribution recommended by default, which lets you spot at a glance the services whose state was changed by hand.

STATE	What it means
enabled	Will start at boot: a symbolic link exists in a .wants directory
disabled	Will not start at boot, but can still be started by hand
static	No [Install] section: cannot be enabled, it is pulled in by a dependency
masked	Linked to /dev/null: impossible to start, even manually
alias	Another name for an existing service (on Debian, kmod.service is a link to systemd-modules-load.service)
enabled-runtime	Enabled until the next reboot only, the enabling lives in /run and not in /etc
indirect	Can be enabled, but the real enabling goes through another service of the same group
generated	Built on the fly by a generator, typically from an old SysV script

Swipe to see more

The static state is the one that surprises the most. A systemctl enable systemd-tmpfiles-clean.service does nothing, not because of a mistake on your side, but because this service is not meant to be enabled: other units trigger it. The command confirms it in one word.

systemctl is-enabled systemd-tmpfiles-clean.service

static

The listings that really help

Listing the 119 units of a server has no operational value. What you want is to filter: which services run, which ones fell over, which ones will start at the next boot. The --state= filter applies to both commands, with different values since they do not talk about the same thing.

The services that really run, with a live process to prove it:

systemctl list-units --type=service --state=running

  UNIT                     LOAD   ACTIVE SUB     DESCRIPTION
  nginx.service            loaded active running A high performance web server and a reverse proxy server
  systemd-journald.service loaded active running Journal Service

2 loaded units listed.

The services that will start at the next boot, the command to know for a server audit:

systemctl list-unit-files --type=service --state=enabled

UNIT FILE              STATE   PRESET
getty@.service         enabled enabled
nginx.service          enabled enabled
systemd-pstore.service enabled enabled

3 unit files listed.

The failed services, the first reflex when you take over a machine you know nothing about:

systemctl list-units --state=failed

  UNIT                         LOAD   ACTIVE SUB    DESCRIPTION
● demo.service                 loaded failed failed Demo qui echoue
● systemd-modules-load.service loaded failed failed Load Kernel Modules

2 loaded units listed.

Two options complete the kit. --all adds the units that are loaded but inactive, which takes the list from 14 to 65 entries on the demonstration machine: this is where the stopped services reappear. --no-legend removes the header and the legend, which makes the output usable in a script.

systemctl list-units --type=service --no-legend | wc -l

14

The table that prevents the mistake

Remember the question each command really asks. It is the only way to stop hesitating in front of an unknown server.

You want to know	Command	Watch out
What runs right now	systemctl list-units --type=service --state=running	Does not show services installed but never started
Everything systemd has loaded	systemctl list-units --type=service	Add --all to include inactive units
What is installed on the machine	systemctl list-unit-files --type=service	The longest list, it is the disk view
What will start at the next boot	systemctl list-unit-files --type=service --state=enabled	The right command for an audit
What fell over	systemctl list-units --state=failed	The first diagnostic reflex
Find a service by name	systemctl list-unit-files | grep docker	Search the unit files, not the loaded units

Swipe to see more

The states of a service

A service goes through several states during its life, and systemctl status shows them on the Active line. Knowing them avoids false diagnoses: a service in active (exited) is not broken, it simply finished its work.

State	Meaning
active (running)	The service runs, a process is alive
active (exited)	The task finished successfully, no process remains (Type=oneshot)
inactive (dead)	The service is stopped, without error
failed	The service failed, the journal holds the reason
activating	Start-up in progress
deactivating	Shutdown in progress
masked	The service is blocked, a start will refuse to do anything

Swipe to see more

enabled is not active

This is the starting confusion of every beginner, and it produces servers that run perfectly until the first reboot. The two notions are independent: one talks about the present, the other about the next boot.

Concept	Question asked	Command
active	Is the service running right now?	systemctl is-active nginx
enabled	Will it start on its own at the next boot?	systemctl is-enabled nginx

Swipe to see more

A service can be enabled and inactive: it is stopped now, it will come back at reboot. It can be active and disabled: it runs, but it will not survive the restart. This last case is the classic scenario of the service installed on a Friday evening and gone on Monday morning.

The diagram separates the two axes: on the left the state right now, changed by start and stop; on the right the state at the next boot, changed by enable, disable and mask. No arrow links the two columns, and that is the whole point.

Source · SVG

Esc to close, + and − to zoom, scroll to move around. 100 % − Fit + Close

Enable a service at boot

Two commands are enough day to day. enable creates the symbolic link that will start the service at boot, enable --now does both at once, and disable removes the link.

sudo systemctl enable nginx          # at the next boot
sudo systemctl enable --now nginx    # at the next boot AND now
sudo systemctl disable nginx         # will no longer start at boot
systemctl is-enabled nginx           # check

The output of enable depends on the distribution, and it is confusing the first time. On RHEL, systemd prints the symbolic link it created. On Debian and Ubuntu, a package that still ships a SysV script (nginx does) first triggers a synchronisation, and the message talks about SysV without mentioning any link:

Synchronizing state of nginx.service with SysV service script with /usr/lib/systemd/systemd-sysv-install.
Executing: /usr/lib/systemd/systemd-sysv-install enable nginx

The disable, on the other hand, does show the removal of the link, which confirms the underlying mechanism:

Removed '/etc/systemd/system/multi-user.target.wants/nginx.service'.

In every case, the only check that counts is systemctl is-enabled nginx, which must answer enabled. Do not trust the message printed, trust the state.

Enabling at boot in detail

The targets, the WantedBy= directives, the distribution presets, the static and indirect states and the persistence test after a real reboot are covered in the dedicated guide Enable a service at boot.

Mask a service for good

disable prevents automatic start-up, but leaves the door open to a manual systemctl start, including one triggered by a dependency of another service. mask closes the door: systemd creates a link to /dev/null, and nothing can start the service any more.

sudo systemctl mask nginx

Created symlink '/etc/systemd/system/nginx.service' → '/dev/null'.

Any start attempt then fails with an unambiguous message, and that is exactly the behaviour you want for a service that must never run on a server, such as Bluetooth, Avahi or CUPS:

Failed to start nginx.service: Unit nginx.service is masked.

Going back is done with sudo systemctl unmask nginx, which removes the link to /dev/null and makes the service startable again. A systemctl is-enabled nginx confirms it left the masked state.

mask has a limit that surprises: it only applies to a unit shipped by a package, stored in /usr/lib/systemd/system/. For a unit you wrote yourself in /etc/systemd/system/, the link to /dev/null would take the place of your file, and systemctl refuses with Failed to mask unit: File '/etc/systemd/system/mon-app.service' already exists and an exit code of 1 (measured on Debian 13 and AlmaLinux 10). To neutralise such a unit, disable --now stops it and removes it from boot; only deleting the file, followed by a daemon-reload, prevents any start.

Create a service unit

Running a business application as a systemd service brings three things a nohup will never give: automatic restart after a crash, start-up at boot and centralised logs in the journal. The unit file lives in /etc/systemd/system/ (the directory reserved for the administrator, which takes precedence over the units shipped by packages).

A unit file reads as three blocks, and each one answers a different question: [Unit] describes the service and its dependencies, [Service] says how to start the process, [Install] says when to enable it at boot. Remember the trap that follows: without an [Install] section, the service is static and systemctl enable will have no effect.

1. Create the unit file with the built-in editor, which handles the path and the permissions for you:

   sudo systemctl edit --full --force mon-app.service

2. Describe the service, explicitly naming the unprivileged user that will run it:

   [Unit]
   Description=Internal billing API
   After=network-online.target postgresql.service
   Wants=network-online.target
   
   [Service]
   Type=exec
   User=appuser
   Group=appuser
   WorkingDirectory=/opt/mon-app
   ExecStart=/opt/mon-app/bin/api --config /etc/mon-app/api.toml
   ExecReload=/bin/kill -HUP $MAINPID
   Restart=on-failure
   RestartSec=5s
   NoNewPrivileges=true
   
   [Install]
   WantedBy=multi-user.target

3. Enable and start the service. systemctl edit already reloaded the systemd configuration when the editor closed; daemon-reload remains mandatory as soon as the file is written another way, by a cp or a deployment tool:

   sudo systemctl daemon-reload      # useless after systemctl edit, mandatory after a cp
   sudo systemctl enable --now mon-app.service

4. Really check that the service stays up, letting a few seconds pass:

   sleep 3 && systemctl is-active mon-app.service

   The output must show active. If it shows failed, journalctl -xeu mon-app.service gives the reason.

Forgetting daemon-reload after changing a unit file without going through systemctl edit is the most frequent mistake: systemd keeps using the version in memory, and your changes seem to have no effect.

The three sections

A unit file always reads the same way, which makes any unknown unit readable in thirty seconds once you know the role of each section.

Section	Role
[Unit]	Description, dependencies (After=, Wants=, Requires=), start-up order
[Service]	The type, the command, the user, the restart policy
[Install]	What happens on enable: the target that will pull this service in at boot

Swipe to see more

A unit without an [Install] section is precisely a static unit: it cannot be enabled, only pulled in by a dependency.

systemctl enable without an [Install] section returns 0, not an error

This is a costly trap in automation. Measured on Debian 13 and AlmaLinux 10 (systemd 257) as well as on AlmaLinux 8 (systemd 239), on a unit without an [Install] section: the command prints thirteen lines of explanation, creates no symbolic link, and exits with exit code 0.

The unit files have no installation config (WantedBy=, RequiredBy=, UpheldBy=,
Also=, or Alias= settings in the [Install] section, and DefaultInstance= for
template units). This means they are not meant to be enabled or disabled using systemctl.

A deployment script that tests $? therefore concludes that enabling succeeded. At the next reboot, the service does not start, and nothing in the deployment logs warned you.

The reliable check is not the exit code, it is the state:

systemctl is-enabled mon-service.service

It answers static when the [Install] section is missing, and enabled when the enabling really happened.

Choose the right Type

The Type= directive tells systemd how to know that the service has started. It is the most misunderstood setting of all of systemd, and the wrong choice produces false diagnoses.

Type	Behaviour	When to use it
simple	The process started by ExecStart is the service; systemd considers it started as soon as it forks	The default when ExecStart= is set, to avoid in favour of exec
exec	Like simple, but systemd waits for exec() to succeed	The right default for a foreground binary (systemd 240 and later)
forking	The process detaches, systemd follows the child process	Traditional daemons with a PIDFile=
oneshot	Runs a command then ends	Maintenance scripts, one-off tasks
notify	The service tells systemd it is ready through sd_notify()	Compatible applications, ideal for start-up order
idle	Waits for the other jobs to finish	Rare cases, console output

Swipe to see more

The trap: Type=simple does not check that your binary exists

Here is the behaviour almost nobody explains, and which costs hours of debugging. With Type=simple, systemd considers the service started as soon as it has forked, so before it even tried to execute the binary. The official documentation says it in black and white: systemctl start reports success even if the service binary cannot be invoked. Result: the command returns with an exit code of 0, and the service is already dead. Run right after, is-active can still answer active: systemd switches the unit to failed only after reaping the death of the child process, a window of a few milliseconds.

Two identical units, a single detail that changes, and two opposite behaviours. The first declares Type=simple, the second Type=exec, and both point to a binary that does not exist:

[Service]
Type=simple
ExecStart=/opt/demo/absent

systemctl start t-simple; echo "exit code of start = $?"; systemctl is-active t-simple

exit code of start = 0
active

The start is a success for the shell, and the service declares itself active. With Type=exec, systemd waits for the result of exec() and refuses to lie:

systemctl start t-exec; echo "exit code of start = $?"; systemctl is-active t-exec

Job for t-exec.service failed because the control process exited with error code.
See "systemctl status t-exec.service" and "journalctl -xeu t-exec.service" for details.
exit code of start = 1
failed

The consequence is very concrete. A deployment script that tests $? after a systemctl start believes it succeeded while the application is dead, and the pipeline turns green. A systemctl status run a second too early shows a perfectly misleading active (running), with a Main PID in double parentheses that betrays a process that never managed its exec():

● demo.service - Demo qui echoue
     Loaded: loaded (/etc/systemd/system/demo.service; disabled; preset: enabled)
     Active: active (running) since Sun 2026-07-12 16:54:43 UTC; 11ms ago
   Main PID: 551 ((app.py))

The fix

Declare Type=exec as soon as your ExecStart= starts a foreground binary. systemd does no prior check of the path: it waits for the result of exec() and makes the start fail if the binary is missing or not executable. Available since version 240, and backported to the systemd 239 of RHEL 8 (accepted by systemd-analyze verify on AlmaLinux 8), so present on every maintained distribution. If you must stay on Type=simple, never trust the exit code of start: check afterwards with sleep 3 && systemctl is-active mon-app.service.

Restart automatically after a crash

A service that dies at three in the morning must get back up on its own. The Restart= directive handles this case, and the StartLimit* directives prevent the infinite restart loop that would hide a real problem.

Directive	Value	Effect
Restart	on-failure	Restarts if the exit code is non-zero, or after a signal, a timeout or the watchdog
Restart	always	Always restarts, except after an explicit systemctl stop
Restart	no	Never restarts (the default)
RestartSec	5s	Waits 5 seconds between two attempts
StartLimitBurst	3	[Unit] section: beyond 3 attempts in the window, systemd gives up
StartLimitIntervalSec	60s	[Unit] section: the window in which attempts are counted

Swipe to see more

Restart=on-failure combined with RestartSec=5s is the setting that the systemd.service(5) manual recommends for a long-running service. The two StartLimit* directives live in [Unit], not in [Service]: put in the wrong place, StartLimitIntervalSec= is ignored with a mere warning (Unknown key 'StartLimitIntervalSec' in section [Service], ignoring, measured on systemd 257 and 239), and your counting window does not exist. Restart=always has a nasty side effect: it also restarts a service whose configuration is broken, which fills the journal and delays the diagnosis.

Diagnose a failing service

When a service is in failed, the cause is almost always written in black and white in the journal. You still have to read it in the right place. This four-step workflow solves the majority of cases.

1. Read the status, which already shows the last lines of the journal:

   systemctl status mon-app.service

2. Open the full journal of the unit, with the explanations of systemd:

   journalctl -xeu mon-app.service

   The -x option adds contextual explanations, -e jumps to the end, -u filters on the unit.

3. Spot the failure code. On a service whose ExecStart= points to a missing file, the journal is explicit:

   demo.service: Main process exited, code=exited, status=203/EXEC
   demo.service: Failed with result 'exit-code'.

   The 203/EXEC code is by far the most frequent: the executable is missing, not executable, or its path has a typo. An ls -l on the path of ExecStart= settles it in one second.

4. Fix, reload, restart, in this order:

   sudo systemctl daemon-reload
   sudo systemctl restart mon-app.service
   sleep 3 && systemctl is-active mon-app.service

The most common failure codes form a vocabulary to know by heart: 203/EXEC for an invalid path, 217/USER for a User= that does not exist, 200/CHDIR for a missing WorkingDirectory=, 226/NAMESPACE for a hardening directive that is too strict. systemd-analyze exit-status lists them all.

When the service stays in failure

The restart loop, the StartLimitBurst limit being reached, systemctl reset-failed and the full catalogue of exit codes are covered in the guide A service does not start: diagnosis method.

Harden an exposed service

A service reachable from the network should not be able to read /home, load a kernel module or gain new privileges. systemd provides sandboxing directives that go into the [Service] section without changing a single line of the application. These four already cover most of the risk and rarely break anything:

[Service]
# forbids any privilege escalation (suid)
NoNewPrivileges=true
# the whole file system read-only, except the data directory
ProtectSystem=strict
ReadWritePaths=/var/lib/mon-app
# /home, /root and /run/user invisible
ProtectHome=true
# an isolated /tmp, invisible to other services
PrivateTmp=true

The comments sit on their own line, and this is not a matter of style: systemd accepts no end-of-line comment. Written NoNewPrivileges=true # forbids…, the directive is rejected (Failed to parse NoNewPrivileges=…, ignoring) then ignored, and the service runs without the protection you think you set. Measured on the four directives above, on systemd 257 and 239: systemd-analyze verify shows the error, the start of the service says nothing.

The command systemd-analyze security mon-app.service rates the service from 0 to 10 (the lower the score, the better) and lists the missing protections one by one. It is the fastest way to measure the effect of a directive: rate before, add, rate after.

Harden one directive at a time

A sandboxing option can prevent a perfectly healthy service from starting: file access denied, network cut. Add the directives one by one, test in between, and if the service falls into 226/NAMESPACE, the last one added is the culprit. The full set of directives and the detailed diagnosis are in the guide Harden a systemd service with sandboxing (in French).

Troubleshooting

The symptoms below cover almost all the support calls about a systemd service. The right-hand column gives the action, not a lead.

Symptom	Probable cause	Solution
systemctl start succeeds but the service is dead	Type=simple does not check exec()	Switch to Type=exec, or check with sleep 3 && systemctl is-active <service>
Service failed with 203/EXEC	Missing binary, wrong path or not executable	ls -l on the path of ExecStart=, then chmod +x if needed
The service does not appear in list-units	It is installed but never loaded	systemctl list-unit-files --type=service | grep <name>
A change to the unit file has no effect	systemd uses its cached version	sudo systemctl daemon-reload then restart
start answers Unit is masked	Service masked to /dev/null	sudo systemctl unmask <service>
enable prints "The unit files have no installation config" and returns 0	No [Install] section in the unit	Normal if the service is pulled in by a dependency; otherwise add [Install] and WantedBy=
A hardening directive has no effect	End-of-line comment, the directive is ignored	systemd-analyze verify <file>, comment on its own line
mask answers already exists	The unit was written in /etc/systemd/system/	disable --now, or delete the file: mask only targets package units
The service runs but disappears after reboot	Never enabled at boot	sudo systemctl enable <service> then check is-enabled
Permission denied in the journal	User= without rights on the files	Check the owner of the files and the hardening directives

Swipe to see more

Practice

Writing a unit while reading a guide and making it stand on a real system are two different exercises. In a disposable VM, you write a complete .service file, load it with daemon-reload, enable it with enable --now and check its real state with is-active and the journal.

Guided lab Practice

Learn how to do it

Create and enable a systemd service unit

$ dsoxlab run l3-service-create-unit

Environment: VM·Duration: ~30 min

How it is checked: the state of the system is checked after a reboot, not the command you typed.

First time? What DSOXLab is and how to install it

DSOXLab is the command line that prepares a disposable environment (an isolated shell or a VM), runs the lab scenario, then checks the real state of the system and computes a score. Nothing is installed or broken on your workstation.

It needs Python 3.11 or later, uv or pipx, and git. The installation guide details these prerequisites; on Windows, or on a Mac without Python, use the appliance instead.

This lab starts virtual machines: it also needs ansible-playbook, a hypervisor (KVM or Incus), Terraform and hardware virtualization (/dev/kvm), which inside a VirtualBox VM means nested virtualization. The first time, dsoxlab start l3-service-create-unit checks all of it before creating the machines. Prepare your machine for vm labs.

Install the CLI and add the catalogue, once:

uv tool install dsoxlab   # or: pipx install dsoxlab
dsoxlab catalog add https://github.com/stephrobert/linux-dsoxlab-training

Before you run the lab, check:

• DSOXLab installed
• catalogue available
• compatible environment (isolated shell or VM)

To check, nothing is probed from this page: run dsoxlab doctor. Read the diagnosis

Run this lab, then validate your work:

dsoxlab run l3-service-create-unit
dsoxlab check l3-service-create-unit

See every lab: dsoxlab list-labs. Track your progress: dsoxlab progress. The full dsoxlab guide covers scoring and the pitfalls.

Key points

• list-units lists what is loaded in memory, list-unit-files what is installed on disk. A service installed but never started does not appear in list-units without --all: 14 loaded units against 119 installed files on the demonstration server.
• For an audit, the right command is systemctl list-unit-files --type=service --state=enabled: it says what will start at the next boot.
• enabled and active are independent: one talks about the next boot, the other about now.
• Type=simple lies. On a missing binary, systemctl start returns 0 while the service is already dead in 203/EXEC (and is-active can answer active for a short moment). Declare Type=exec, or check the state a few seconds after the start.
• 203/EXEC is the most frequent failure code: invalid ExecStart= path or non-executable file.
• After changing a unit file outside systemctl edit, daemon-reload is mandatory, otherwise systemd keeps its cached version.
• reload rereads the configuration without cutting connections, restart kills the process: prefer reload when the unit declares an ExecReload=.
• mask is stricter than disable: it makes a package service impossible to start, even through a dependency.
• No end-of-line comment in a unit file: the directive is ignored. systemd-analyze verify reports it.

Knowledge check

Check that the essentials of this guide stuck. The questions cover only what has just been explained here.

Knowledge check

Check what you have learned with this interactive quiz

6 questions

6 min

70% to pass

How it works

• The timer starts when you click Start
• Multiple choice, true/false and short answer questions
• You can move between questions
• Detailed results are shown at the end

Start the quiz

Starts the quiz and the timer

Frequently asked questions

What is systemd and why did it replace the old init systems?

Which systemctl commands should you know?

How do you create a custom systemd service?

What is the difference between enable, start, restart and reload?

How do you diagnose a service that does not start?

How do you read the logs of a service with journalctl?

What is a systemd target and how does it work?

How do you mask a service to prevent it from starting?

How do you limit the resources of a service (CPU, memory)?

How do you secure a systemd service?

How do you analyse the boot time of the system?

How do you restart a crashing service automatically?

How do you schedule a task with a systemd timer?

What are the systemd service types (simple, exec, forking, oneshot)?

How do you manage dependencies between services?

×

See in the glossary →