Run Your Own Daemon with systemd — Writing Service Unit Files

Every long-running thing I put on a Linux box ends up as a systemd unit: game servers, little Node APIs, Discord bots, a Python script somebody swore was temporary. No screen sessions, no nohup ... &, no @reboot cron lines. A unit file is twenty lines, survives reboots, restarts on crash, and puts all output in the journal where I can actually find it.

TL;DR: Drop a file in /etc/systemd/system/, reload, enable, done.

sudo nano /etc/systemd/system/myapp.service    # contents below
sudo systemctl daemon-reload
sudo systemctl enable --now myapp
systemctl status myapp
journalctl -u myapp -f

Here's the file and what each line is actually doing.

The unit I start from

# /etc/systemd/system/myapp.service
[Unit]
Description=My App
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User=myapp
Group=myapp
WorkingDirectory=/opt/myapp
EnvironmentFile=-/etc/myapp/myapp.env
ExecStart=/opt/myapp/bin/myapp --port 8080
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

That's 90% of every unit I write. Line by line:

• [Unit] — metadata and ordering. After= only orders start-up, it doesn't pull anything in. Wants= pulls it in. You usually want both for network-online.target, otherwise your app starts before the network is actually up and fails to bind or resolve.
• Type=simple — the process in ExecStart is the service and stays in the foreground. That's what you want for basically every modern app. More on the other types below.
• User= / Group= — don't run things as root that don't need root. Create a system user for it (below).
• WorkingDirectory= — relative paths in your app resolve from here. Lots of game servers and scripts expect to be started from their own directory and crash with "file not found" otherwise.
• EnvironmentFile= — KEY=value lines, one per line. The leading - means "don't fail if the file doesn't exist". Secrets go here, not in the unit, because unit files are world-readable.
• ExecStart= — absolute path to the binary. Always absolute.
• Restart=on-failure — restart when it exits non-zero or gets killed by a signal, but not when it exits cleanly. Restart=always restarts even after a clean exit; I use that for game servers that shut down on an in-game stop command.
• [Install] / WantedBy=multi-user.target — this is what enable hooks into. Without an [Install] section, systemctl enable has nothing to do and the service won't come back after reboot.

The service user:

sudo useradd --system --home /opt/myapp --shell /usr/sbin/nologin myapp
sudo chown -R myapp:myapp /opt/myapp

If you want a real login user instead (to su in and poke around), see user management with adduser, sudo and SSH keys.

Gotcha: ExecStart is not a shell

This bites everyone once. ExecStart is executed directly, not through bash. So this does not do what you think:

ExecStart=/opt/myapp/run.sh > /var/log/myapp.log 2>&1

The > and 2>&1 get passed to your script as literal arguments. No pipes, no redirects, no &&, no globbing, no ~. systemd does expand $VAR and ${VAR} from Environment=/EnvironmentFile=, but that's it.

You don't need the redirect anyway — stdout and stderr already go to the journal. If you really need shell features, say so explicitly:

ExecStart=/bin/bash -c 'exec /opt/myapp/bin/myapp --token "$(cat /etc/myapp/token)"'

The exec matters: it replaces bash with your app, so systemd tracks and signals the real process instead of a shell wrapper.

Picking the right Type=

• simple — default. Foreground process. systemd considers it "started" the moment it forks it.
• exec — like simple, but systemctl start only reports success once the binary was actually executed. Catches typos in the path at start time instead of a second later. I use it more and more.
• forking — old-school daemons that fork into the background. Needs PIDFile= so systemd knows which process to watch. If your app has a "foreground" or --no-daemon flag, use that with simple instead.
• oneshot — runs, finishes, exits. Backups, cleanup scripts, anything triggered by a timer. Add RemainAfterExit=yes if it should count as "active" afterwards (e.g. a script that sets up firewall rules).
• notify — the app tells systemd itself when it's ready via sd_notify. Only if your app supports it.

For scheduled jobs, a oneshot service plus a .timer replaces cron — that's in systemd timers instead of cron.

Gotcha: you edited the file and nothing changed

systemd caches unit files. After every edit:

sudo systemctl daemon-reload
sudo systemctl restart myapp

systemctl status will even warn you: Warning: The unit file, source configuration file or drop-ins of myapp.service changed on disk. Run 'systemctl daemon-reload' to reload units. Read it.

Before reloading, I check the file for typos:

sudo systemd-analyze verify /etc/systemd/system/myapp.service

Keys in the wrong section are ignored — Restart= in [Unit] does nothing, and the only hint is a one-line warning buried in the journal.

Gotcha: Java exits 143 and systemd calls it a failure

When systemd stops a service it sends SIGTERM. The JVM handles that, runs its shutdown hooks and exits with code 143 (128 + 15). systemd sees non-zero and marks the unit failed after a perfectly normal stop. For Java things — Minecraft, proxies, anything on the JVM — add:

SuccessExitStatus=143

While you're there, give slow shutdowns time. The default stop timeout is 90 seconds, then systemd sends SIGKILL. A big game world saving to disk can need more:

TimeoutStopSec=180

If your app wants a different signal to shut down cleanly (some want SIGINT, like Ctrl+C), set KillSignal=SIGINT.

Gotcha: crash loops and the start limit

Restart= isn't infinite. By default, if a unit starts more than 5 times within 10 seconds, systemd gives up and leaves it failed with start request repeated too quickly. With RestartSec=5 you'll rarely hit that, but with a tiny RestartSec and a broken config you will. The knobs live in [Unit], not [Service]:

[Unit]
StartLimitIntervalSec=300
StartLimitBurst=5

After fixing the cause, clear the failed state:

sudo systemctl reset-failed myapp

If you want to know when something crash-loops instead of finding out from users, OnFailure= can trigger another unit — I do that with Discord webhooks in Discord alerts from server logs.

Change units without touching them: drop-ins

For units shipped by a package (nginx, docker, whatever), never edit the file in /usr/lib/systemd/system/ — the next package update overwrites it. Use an override:

sudo systemctl edit someservice

That opens an editor and saves to /etc/systemd/system/someservice.service.d/override.conf, then reloads for you. Only put the lines you want to change in there.

One trap: list-type settings like ExecStart= add in a drop-in instead of replacing. To replace, clear it first with an empty assignment:

[Service]
ExecStart=
ExecStart=/usr/bin/someservice --config /etc/someservice/custom.conf

systemctl cat someservice shows the original plus all drop-ins in order, which is the fastest way to see what's actually in effect.

Cheap sandboxing

systemd can lock a service down without containers. These are the lines I add to most of my own apps:

[Service]
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/opt/myapp/data /var/log/myapp

ProtectSystem=strict mounts the whole filesystem read-only for the service, so you must whitelist whatever it writes to with ReadWritePaths=. If the app suddenly fails with Read-only file system, that's the cause — add the path. ProtectHome=true hides /home, /root and /run/user entirely.

Then see how exposed it still is:

systemd-analyze security myapp

It scores every unit from 0 (locked down) to 10 (exposed) and lists what's missing. You don't need to chase a perfect score, but going from 9.6 to around 5 with five lines is free.

Debugging a unit that won't start

systemctl status myapp                 # last log lines + exit code
journalctl -u myapp -b --no-pager      # everything since boot
journalctl -u myapp -n 100 -e          # last 100 lines, jump to end

status=203/EXEC means systemd couldn't execute ExecStart at all: wrong path, missing execute bit, or a script without a shebang. status=200/CHDIR is a WorkingDirectory that doesn't exist. status=217/USER is a User= that doesn't exist. Those three cover most of the "it won't even start" cases.

User services, briefly

No root on the box? ~/.config/systemd/user/myapp.service, same format, managed with systemctl --user. Use WantedBy=default.target in that case. User services stop when you log out unless lingering is enabled:

sudo loginctl enable-linger myuser

Fine for a personal bot. For anything that matters, a system unit with a dedicated user is cleaner.

If you're running a game server, the Minecraft Java server guide has a complete real-world unit built on exactly this pattern.

Related posts

• UFW Firewall Rules That Don't Lock You Out
• Linux Log Rotation with logrotate — Before Your Disk Fills Up
• Linux User Management — adduser, Groups, sudo and SSH Keys