rsync Backups That Actually Work — No Excuses
rsync backups that hold up: trailing-slash rules, dry runs, hard-link snapshots with --link-dest, rotation, pull over SSH with rrsync, a timer.
rsync is on every Linux box already, it's fast, and it doesn't need a server, a repository format or a password manager entry. That's exactly why it's my first backup layer: if setting up a backup takes ten minutes, it actually gets done. What rsync doesn't give you out of the box is history — a plain mirror happily syncs your mistakes too. This post is the pattern I use to fix that: dated snapshots with hard links, rotation, and pulling over SSH.
TL;DR: A nightly snapshot that only stores what changed, everything else hard-linked to yesterday:
rsync -a --delete --link-dest=/backup/myapp/latest /srv/myapp/ /backup/myapp/$(date +%F)/
ln -sfn /backup/myapp/$(date +%F) /backup/myapp/latest
Every dated folder looks like a full copy, but unchanged files take no extra space. The full script with rotation is below.
The flags I actually use
rsync -a --delete /srv/myapp/ /backup/myapp/current/
-a— archive mode: recursive, keeps permissions, owners, groups, timestamps, symlinks, device files. It's-rlptgoDin one letter.--delete— remove files from the target that no longer exist in the source. Without it the backup grows forever and a restore brings back files you deleted on purpose.
For full system-level copies (configs, /etc, things with ACLs) I add:
rsync -aHAX --numeric-ids --delete /etc/ /backup/etc/
-H keeps hard links, -A ACLs, -X extended attributes, and --numeric-ids stores UID/GID numbers instead of mapping names — important when you restore onto a box where www-data has a different ID. Owners only survive if rsync runs as root on the receiving side.
Over the network add -z for compression (pointless on already-compressed data like videos or .tar.gz) and, for big initial copies, --info=progress2 for one overall progress line.
Gotcha: the trailing slash
This is the one everybody gets wrong at least once:
rsync -a /srv/myapp/ /backup/myapp/ # copies the CONTENTS of myapp into /backup/myapp/
rsync -a /srv/myapp /backup/myapp/ # creates /backup/myapp/myapp/
Trailing slash on the source means "what's inside". No slash means "the directory itself". The destination slash doesn't matter. Combine the wrong one with --delete and you can wipe a target directory full of other things. Which brings me to the rule I never skip:
rsync -a --delete --dry-run --itemize-changes /srv/myapp/ /backup/myapp/
--dry-run (-n) shows what would happen without touching anything. --itemize-changes (-i) prints one line per file with what changes; lines starting with *deleting are the ones to read twice. Any new rsync command with --delete gets a dry run first.
Snapshots with --link-dest
A mirror is not a backup. If ransomware or a bad rm hits the source at 02:00 and the sync runs at 03:00, your mirror is now also empty. You need history.
--link-dest=DIR tells rsync: for every file that's unchanged compared to DIR, don't copy it — create a hard link to the file in DIR instead. Result: each dated directory is a complete, browsable copy, but only changed files use new disk space. Restore is cp or another rsync. No special tool.
Here's the script I drop on boxes:
#!/usr/bin/env bash
# /usr/local/bin/snap-backup.sh
set -euo pipefail
SRC="/srv/myapp/"
DEST="/backup/myapp"
KEEP=14
TODAY="$(date +%F)"
mkdir -p "$DEST"
rc=0
rsync -a --delete \
--link-dest="$DEST/latest" \
"$SRC" "$DEST/$TODAY/" || rc=$?
# 24 = some source files vanished during the run, normal on live data
if [ "$rc" -ne 0 ] && [ "$rc" -ne 24 ]; then
echo "rsync failed with exit code $rc" >&2
exit "$rc"
fi
ln -sfn "$DEST/$TODAY" "$DEST/latest"
# rotation: keep the newest $KEEP snapshots, delete the rest
ls -1d "$DEST"/20??-??-?? | head -n -"$KEEP" | xargs -r rm -rf
sudo chmod +x /usr/local/bin/snap-backup.sh
sudo /usr/local/bin/snap-backup.sh
du -sh /backup/myapp/* # first one is big, the rest are small
Notes on the details:
- On the very first run
latestdoesn't exist yet. rsync prints a warning that the--link-destdir doesn't exist and just makes a full copy. That's expected. ln -sfn— the-nmatters. Without it,lnfollows the existinglatestsymlink and creates the new link inside the old snapshot.- Running it twice on the same day re-syncs into the same dated folder. Fine.
du -shacross snapshots counts hard-linked files only once per invocation, so the numbers show real incremental usage.
Gotcha: don't rotate by mtime
The obvious rotation is find "$DEST" -maxdepth 1 -mtime +14 -exec rm -rf {} +. Don't. rsync -a sets the snapshot directory's mtime to the source directory's mtime, which can be months old. find -mtime then deletes snapshots you made yesterday. Rotating by name — which is the date — is the only thing you can trust. That's why the script uses ls | head -n -14: date-named folders sort chronologically.
Gotcha: live databases and game saves
rsync copies files. It has no idea whether those files are consistent. Copying /var/lib/mysql or a Postgres data dir while the database runs gives you a backup that might not start. Dump first, then rsync the dump:
mysqldump --single-transaction --all-databases | gzip > /srv/dumps/mysql-$(date +%F).sql.gz
pg_dumpall -U postgres | gzip > /srv/dumps/pg-$(date +%F).sql.gz
Game worlds are the same problem: the server writes region and chunk files constantly. For anything that matters I do stop → rsync → start. A minute of downtime at 04:00 beats a corrupt world. The Minecraft Java server guide and the various game server posts use exactly this pattern.
Off the box: pull, don't push
A backup on the same disk is a copy, not a backup. It needs to land on different hardware. Two ways:
- Push — the server rsyncs to the backup host. Easy, but the server holds credentials to the backup host. If the server gets owned, the attacker can delete the backups too.
- Pull — the backup host connects to the server and fetches. The server has no access to the backups at all. That's what I prefer.
On the backup host, generate a dedicated key:
ssh-keygen -t ed25519 -f ~/.ssh/backup_ed25519 -N '' -C 'backup-pull'
On the server, put the public key in the backup user's ~/.ssh/authorized_keys and lock it to read-only rsync of one path with rrsync, which ships with the rsync package on current Debian and Ubuntu (/usr/bin/rrsync):
command="/usr/bin/rrsync -ro /srv/myapp/",restrict ssh-ed25519 AAAAC3...yourkey... backup-pull
restrict disables port forwarding, PTY, agent forwarding and friends. -ro means this key can only read, only from /srv/myapp/. Even if the key leaks, it can't write anything or open a shell.
Then the pull on the backup host — paths are now relative to the directory rrsync allows, so ./ means "everything in /srv/myapp/":
rsync -a --delete -e "ssh -i ~/.ssh/backup_ed25519" \
--link-dest=/backup/myapp/latest \
backup@server.example.com:./ /backup/myapp/$(date +%F)/
Same snapshot script, just with the remote source. Key-based SSH setup in general is in user management and SSH keys, and the server side should follow the SSH hardening checklist anyway.
One limitation: the backup user can only read what it has permission to read. For app data owned by a service user, add the backup user to that group, or back up dumps that you write world-readable to a staging directory. I avoid giving the pull user root.
Schedule it
A systemd timer instead of cron, so failures show up in the journal and missed runs catch up after downtime:
# /etc/systemd/system/snap-backup.service
[Unit]
Description=rsync snapshot backup
[Service]
Type=oneshot
ExecStart=/usr/local/bin/snap-backup.sh
# /etc/systemd/system/snap-backup.timer
[Unit]
Description=Nightly rsync snapshot
[Timer]
OnCalendar=*-*-* 03:30:00
Persistent=true
[Install]
WantedBy=timers.target
sudo systemctl daemon-reload
sudo systemctl enable --now snap-backup.timer
systemctl list-timers snap-backup.timer
Why timers over cron is its own post: systemd timers instead of cron.
Test the restore or it doesn't count
A backup you've never restored is a hope, not a backup. Every now and then:
mkdir -p /tmp/restore-test
rsync -a /backup/myapp/latest/ /tmp/restore-test/
diff -r /srv/myapp/ /tmp/restore-test/ | head
And watch the disk on the backup target — snapshots of fast-changing data grow faster than you'd expect. If it fills up, disk-full forensics helps you find where the space went.
rsync snapshots are my fast, local, "oops I deleted that" layer. For encrypted off-site copies I put restic on top — different tool, different failure modes, and that's the point.
