Skip to main content
restic is an open-source backup program that deduplicates and encrypts your data before it leaves the server, and it speaks the S3 API natively. That makes it a good fit for Object storage: your server writes straight to a bucket, and Latitude.sh does not charge for the traffic in either direction (see Egress). Everything that matters happens on the server. restic splits your files into chunks, drops the chunks it has already stored, encrypts what is left, and packs it into objects. The bucket only ever sees encrypted data, which is why the repository password matters as much as the access key.

What you’ll do

The guide covers a full backup lifecycle on a single server, in four phases. Do them in order: the repository has to exist before the first backup, and a restore is only worth automating once you have tested it manually.
The steps below set up one server. To cover several, go through them once per host, with a single change when you reach Step 4: append the hostname to the repository URL before running restic init.
That gives each host its own repository inside the same bucket, reachable with the same access key. Pointing every host at one shared repository instead would hand each of them a key that can read and delete the whole fleet’s backups.

Prerequisites

  • A bare metal server you can reach over SSH as a user with sudo. The commands below assume you run them as root
  • A verified team, which is required to create access keys
  • The Latitude.sh CLI installed and logged in on your workstation. Steps 1 and 2 use the lsh s3 command group; if lsh s3 --help does not recognize it, run lsh update first
  • A place to store the repository password that is not the server you are backing up (a password manager, for example)
restic encrypts the repository with a password. If you lose that password, the backups cannot be decrypted by anyone, including Latitude.sh. Store it somewhere other than the machine being backed up.

Step 1: Create a bucket

Create a dedicated bucket for backups so its lifecycle and credentials stay separate from application data. Run this step from your workstation, where lsh is logged in. Bucket administration goes through the Latitude.sh API, so it uses your lsh session and needs no S3 credentials. To do it from the dashboard instead, see Creating a bucket.
1

Set the values the commands will use

The commands in this step and the next read three values from your shell. Export them once, substituting the placeholders, so every command below can be pasted as is:
  • LSH_PROJECT — the project that will own the bucket, by ID or slug. lsh projects list prints both for every project; the dashboard shows the ID under project settings. The CLI reads this variable itself, so the --project flag below is there only to make the choice explicit.
  • SITE — the site where the bucket will live. lsh regions list prints the slugs (DAL, SAO, NYC), and Regions & Locations explains how sites relate to regions. Pick a site that is not the same location as the server you are backing up: if the server and its backups sit in one facility, a single site-level incident takes out both.
  • BUCKET — the display name of the bucket. Names are globally unique, so a generic backups is likely taken; <team>-<hostname>-backups avoids collisions and still reads clearly a year from now. See Bucket naming rules for the full constraints.
These variables live in this shell only. If you open a new terminal before finishing Step 2, export them again.
2

Create the bucket

What each flag decides:
  • --storage-class standard — backups are sequential and write-mostly, so the low latency that high_performance is built for does not shorten a nightly run. The two classes are also priced differently: see the storage pricing page. Standard is the default; the flag is there to make the choice visible.
  • --versioning — preserves every version of an object, so an accidental overwrite or delete inside the repository can be rolled back.
  • --no-access-key — skips the prompt mb shows after creating a bucket, which offers to create an access key and store it in your lsh profile. A key stored there is for running lsh s3 commands from your workstation. restic needs its own key on the server, which you create in Step 2.
3

Read back the bucket

Check that Versioning says yes and Locking matches the tab you used, then keep two lines from the output. You will export them on the server in Step 4:
  • Bucket name (backend) — the name the bucket has on the S3 endpoint. It can differ from the display name you chose, and it is the one restic needs.
  • Endpoint — the S3 API endpoint for this bucket. Buckets of different storage classes are reached at different endpoints.
You can run this command again at any time to read both values back.

Step 2: Create an access key

Access keys are project-scoped S3 credentials, separate from the API token lsh logged in with. Make a dedicated one for the backup job, scoped to this bucket only, so revoking it later touches nothing else. To do it from the dashboard instead, see Access keys. Still on your workstation, with the variables from Step 1 exported:
--bucket "$BUCKET=rw" creates a limited key that reaches only this bucket, with read-write permission. restic needs to write new data and to delete data when you prune, so readonly will not work for the full lifecycle. Storage class, site and project are inferred from the bucket. --name is capped at 25 characters by the API, which is why it does not carry the hostname. If you back up several servers into this bucket, one key serves all of them. The output includes an Access Key ID and a Secret Access Key line. Keep both for Step 4.
The secret is displayed only once, right after the key is created. Neither the API nor the CLI can show it again. If you lose it, create a new key.
Do not add --save. It stores the key in your workstation’s lsh profile and hides the secret, which is the opposite of what you need here: this key belongs on the server, in the environment file you will write in Step 4.

Step 3: Install restic on the server

Log in to the server over SSH. Every remaining step in this guide runs there, not on your workstation.
Everything from here runs as root. If you logged in as another user, switch once with sudo -i rather than prefixing each command: several commands below are chained with &&, where a single sudo would apply only to the first half.
Most distributions package restic, though often a few versions behind:
On the RHEL family restic ships in EPEL, not in the base repositories, which is why those commands enable EPEL before installing.
Use this to run a current release, or if your distribution has no restic package. Check the releases page for the latest version, without the v, and export it once:
Then run the tab for your architecture. uname -m tells you which: x86_64 for the first, aarch64 for the second.
The binary lands in /usr/local/bin/restic, which matters again in Step 9.
Confirm the install before going further:
It prints the restic version, the Go version it was built with, and the platform. If the shell reports command not found, the install did not land on your PATH.
restic self-update exists only in the official binary. Distribution packages are built without it and answer unknown command "self-update" for "restic", so upgrade those through the package manager:

Step 4: Configure credentials

restic reads its configuration from environment variables. Keeping them in one file means the secret never has to appear in a restic command line, or in the systemd unit you will write later.
1

Generate the repository password

Print it and copy it into your password manager now:
The file stays on the server so restic can read it unattended, but that copy dies with the server. The one in your password manager is what makes the backups recoverable.
2

Write the environment file

Four values from your workstation go in this file. Export them on the server first, substituting the placeholders. The variable names are the ones the CLI reads for its own S3 commands, so they carry over if you ever use lsh s3 against this bucket:
  • BUCKET_BACKEND_NAME and LSH_S3_ENDPOINT_URL — the Bucket name (backend) and Endpoint lines that lsh s3 stat printed in Step 1. The backend name is not the display name you chose.
  • LSH_S3_ACCESS_KEY_ID and LSH_S3_SECRET_ACCESS_KEY — the Access Key ID and Secret Access Key from Step 2.
Then write the file. The heredoc expands those variables, so it can be pasted as is:
The repository URL is the s3: prefix, then the endpoint exactly as the CLI prints it, then the backend bucket name as the last path segment.
The export line with the secret lands in your shell history. If you would rather it never did, create the file with an editor instead, or clear the history afterwards.
3

Load it into your shell

Check that it took, before restic has a chance to fail for a reason you would have to go looking for:
Both should print the values you just wrote. An empty line means the file did not load, usually a typo in the path. Neither of these is the secret, so it is safe to print them.Every restic command in the rest of this guide assumes these variables are loaded. They live in this shell only: open a new SSH session and you have to load them again.

Step 5: Initialize the repository

This writes the repository structure and the encrypted key material into the bucket. Run it once per repository, not once per backup. Now confirm restic can actually reach it:
Expect repository <id> opened, then the repository’s own metadata. That line is the signal, not the metadata: a wrong endpoint, bucket name, key, or password fails the command outright. See Troubleshooting if it does.

Step 6: Run the first backup

Decide what to back up. On a server, the useful set is usually configuration, application data, and databases — not the whole root filesystem. Reinstalling packages is cheap; recovering /etc and your data is not.
1

Write an exclude file

These are paths that either cannot be restored or are not worth storing: kernel and runtime filesystems, scratch space, caches, and sockets.
Adjust the list to the server. /var/lib/docker, for example, only matters if you run Docker, and its images are rebuildable anyway.
2

Dump any databases

Skip this if the server does not run one. A file-level copy of a running database can be inconsistent, so dump it to disk first and let restic pick up the dump. The example is PostgreSQL; other databases have their own dump tools, and the same rules apply: run the dump as the database’s system user, into a directory only that user and root can read.
The dump lands under /var/backups, which is already in the backup command of the next step.
This step is for a database you run on the server yourself. A Managed Database lives outside the server and has its own scheduled backups, so it does not go in this list.
3

Back up

The first three exist on any server, and /var/backups is where the dump from the previous step lands. Add the ones that hold your data: /var/www for a web root, whatever else applies.
If you dumped a database, /var/backups has to stay in this list. The dump is only protected if the path that holds it is backed up, and nothing warns you when it is not: the backup succeeds without it.
A path that is not there is not fatal. restic prints <path> does not exist, skipping and backs up the rest, so a list that is slightly wrong still produces a snapshot. Read that line when it appears, though: it is also what you would see if the data you meant to protect is somewhere else.--one-file-system keeps restic from walking into other mounts, such as an attached block volume or a network filesystem. Drop it if you do want those included.
The first run uploads everything. Later runs only upload changed data, because restic splits files into content-addressed chunks and skips those already in the repository.

Step 7: Verify the backup

Check the repository after the first run.
Lists one row per snapshot under ID, Time, Host, Tags, and Paths. After a first backup you should see exactly one row. An empty list means nothing was stored.
Validates the repository structure and metadata without downloading every pack file. It should end with no errors were found. Any other ending names what is damaged.
The same check, plus downloading and hashing a sample of the pack files, so it also catches corruption in the stored data rather than just the index. Ends with the same line, takes considerably longer, and is worth running periodically rather than after every backup.

Step 8: Restore

Practice this before you need it. If you are back in a new SSH session, load the environment file from Step 4 first, or every command below fails with Fatal: Please specify repository location:
1

Restore everything to a scratch directory

restic creates the target directory, reports progress, and ends with a summary of how many files and how much data it restored. Run it twice against the same target and the second summary reports most files as skipped: restic leaves files that already match the snapshot alone.The real check is the result on disk. Under the target you get the paths you backed up, rebuilt as a tree:
You get one top-level directory per path you backed up in Step 6, so etc, home, srv, var, and whichever others you kept. Spot-check a file you recognize, for example cat /tmp/restore/etc/hostname.If the snapshot holds the database dump from Step 6, load it back as root, through stdin:
restic creates the target directory so that only root can enter it, which is why psql -f run as the postgres user fails with Permission denied. Errors about roles or databases that already exist are expected when a --clean dump is loaded on the server it came from.Restoring to /tmp/restore rather than / is what makes this safe to practice: nothing live is overwritten.
2

Restore a single path

Same command, narrowed, and pointed at a fresh target so the result is unambiguous. Only the included subtree is written, so etc/ssh is all you get:
/etc/ssh is used here because it is guaranteed to be in the snapshot: you reached this server over SSH.
If --include matches nothing, restic reports Restored 0 files/dirs (0 B), does not create the target directory at all, and still exits successfully. An empty result means the pattern is wrong, not the repository.
restic mount exposes the repository as a read-only filesystem. It is not part of the backup lifecycle, and the two restores above already cover recovery. It earns its place when you need one file and do not know which snapshot holds it, because you can look instead of guessing.Mounting needs the fusermount binary from FUSE. Whether it is already there depends on the image, not on restic: the Ubuntu and AlmaLinux images ship it, the Oracle Linux one does not. Check before mounting:
If that prints nothing, restic mount would fail with fusermount: executable file not found in $PATH. On the RHEL family, install it with:
It has to be fuse, not fuse3: only the first one provides fusermount.Create the mountpoint first, because restic does not:
The command prints Now serving the repository at /mnt/restic and then blocks, holding the terminal. Leave it running and open a second SSH session to browse. That session starts as your login user, so switch to root again with sudo -i before continuing:
Each snapshot is a directory, alongside a latest shortcut to the newest one. Inside, the tree looks like the server did, and ordinary commands work on it, so you can copy a single file straight out:
Back in the first session, Ctrl+C unmounts.

Step 9: Schedule it with systemd

Everything so far has been manual, which means the backup is only as current as the last time someone remembered to run it. systemd is the service manager already running on your server, and scheduling with it takes two files: a service unit that defines what to run, and a timer unit that defines when to run it. The timer starts the service on a schedule the way cron would, with two differences that matter here: every run is recorded in the system journal, so you can read what happened afterwards, and a run missed while the server was off is caught up on the next boot. You will write both files, tell systemd to pick them up, then trigger one run manually to confirm it works before leaving it to the schedule.
1

Create the service unit

systemd runs this unit with no shell profile and no PATH of its own, so every binary has to be named by absolute path, and a path that does not exist fails the whole unit. Two things in the unit below depend on your server:
  • The restic path. A packaged install lands in /usr/bin/restic, the official binary in /usr/local/bin/restic.
  • The backup paths. The same list you settled on in Step 6, including /var/backups if you dump a database. The environment file carries your credentials, not your path choices, so they have to be spelled out again here.
Ask the server which restic it has, and keep the answer in a variable:
It prints /usr/bin/restic for a packaged install, or /usr/local/bin/restic for the official binary. Then create the unit, substituting your own backup paths. The heredoc fills in the restic path from the variable:
What the directives do:
  • EnvironmentFile loads the credentials from Step 4, which is how the timer authenticates without a shell.
  • Environment=HOME=/root is there because systemd starts the unit without a HOME. Without it, restic logs unable to open cache and re-downloads metadata it could have reused.
  • Nice and IOSchedulingClass keep the backup from competing with your workload for CPU and disk.
If you dump a database, add a line above ExecStart so the scheduled run refreshes the dump instead of storing the one you made manually, getting staler every night. It is the same command as in Step 6, with the absolute paths that which runuser and which pg_dumpall print:
Check that /var/backups is in the ExecStart line above before you move on. With the dump line in place and the path missing, every scheduled run refreshes the dump and then backs up everything except it, and the unit still reports SUCCESS.
2

Create the timer

Persistent=true runs a missed backup after a reboot. RandomizedDelaySec spreads the load when you run this across a fleet.
3

Load and arm the timer

Makes systemd read the two new files. Silent on success.
Prints a Created symlink line, which is systemd confirming the timer is enabled at boot and started now.
4

Run it once manually

Do not wait for the schedule to find out the unit is broken.
This runs a real backup and returns only when it finishes, printing nothing. No output is the good outcome; an error here means the unit failed to start at all.
Look for status=0/SUCCESS on the Main PID: line. Active: inactive (dead) above it is expected and not a failure: a Type=oneshot unit is meant to run and exit, so it sits inactive between runs.
Shows what restic actually did on that run, including the new snapshot and the files it processed. If systemctl start failed instead, these same two commands say why: see Troubleshooting.
5

Confirm the schedule

One row, with NEXT and LEFT showing when the next run fires. LAST and PASSED stay - until the timer itself triggers the service: starting it manually does not count as a timer activation. An empty table means the timer is not enabled, so revisit the enable command above.

Step 10: Apply a retention policy

The timer now adds a snapshot every night and never removes one, so the repository grows forever. A retention policy decides which snapshots are worth keeping: recent ones in full detail, older ones thinned out to one per week, then one per month. Two separate things do that work, and the distinction explains the commands below:
  • forget marks snapshots as no longer wanted. It is fast, because it only drops the snapshot’s entry, not the data behind it.
  • prune deletes the data that no remaining snapshot references. It is what actually frees space in the bucket, and it is slow, because it rewrites pack files.
Deduplication is why they are separate: a chunk shared by five snapshots can only be deleted once nothing points at it, so the data cannot go until you have decided which snapshots are staying.
That keeps the last 7 daily snapshots, 5 weekly, and 12 monthly, and drops the rest. --prune runs the second phase right after the first. Add --dry-run first to see which snapshots the policy would remove, without removing anything. The real run prints the policy it applied, then a keep table of every snapshot that survived it. The Reasons column there is worth reading: it names the rule that saved each one, so a snapshot kept as daily snapshot, weekly snapshot, and monthly snapshot at once is filling all three roles. Snapshots on their way out appear in a second table below, and --prune then reports what it repacked and deleted.
Right after setting this up, nothing is removed: with one or two snapshots, every one still matches the policy.
Confirm the state it left behind:
restic snapshots should list exactly the snapshots the policy said to keep, and nothing else. restic check should still end with no errors were found, which is what tells you prune removed data without leaving the repository inconsistent. Run this on a schedule too, or the repository grows anyway. Give it a timer of its own rather than adding it to the nightly backup unit, built the same way as Step 9, for two reasons: prune is the heavy half and does not need to run daily, and a prune that fails would otherwise mark your backup as failed even though the backup itself succeeded.

Wrap-up

You now have:
  • A versioned bucket in a different site from the server, reached with an access key scoped to that bucket alone
  • An encrypted restic repository in it, with the password stored off the server
  • A nightly systemd timer that backs up your chosen paths and logs every run to the journal
  • A retention policy that keeps the repository from growing forever, and a restore you have already tested once
Keep the repository password and the access key in your password manager, and repeat the restore from Step 8 from time to time. A backup counts only once it has been restored.

Troubleshooting

Run systemctl status restic-backup.service and journalctl -u restic-backup.service -n 50. The usual causes:
  • status=203/EXEC — systemd could not execute the binary named in ExecStart or ExecStartPre. The path is wrong or the file is not there. Compare it against which restic.
  • restic ran and exited non-zero — the journal carries restic’s own message. A path in the backup list that does not exist on the server is the common one.
restic did not find RESTIC_REPOSITORY in the environment. The variables from Step 4 live in the shell that loaded them: a new SSH session, or a fresh sudo -i, starts without them. Load the file again:
Hundreds of these lines during a restore, ending in Fatal: There were N errors, mean the host runs SELinux. The data was restored; what failed is restic trying to change the SELinux label the kernel put on each new file. Run the restore again with --exclude-xattr security.selinux, as in Step 8.
pg_dumpall was run as root. PostgreSQL authenticates local connections by system user, and there is no database role called root. Run the dump as the postgres user with runuser, as in Step 6, and make sure the output directory belongs to that user.
restic could not find a repository at RESTIC_REPOSITORY. Check that restic init has been run, that the last path segment of the URL is the Bucket name (backend) that lsh s3 stat "s3://$BUCKET" prints, not the display name, and that the endpoint is the Endpoint from the same output. Buckets of different storage classes are reached at different endpoints, so one copied from another bucket will not resolve.
Usually the secret key. Because it is shown only once, a truncated paste is common. It can also mean the access key is scoped elsewhere — keys apply to a specific storage class and region, and a limited key only reaches the buckets it was created with. Run lsh s3 access-keys get restic-backup on your workstation and confirm the Buckets line lists this bucket with rw.
Some S3 clients need an explicit region for a non-AWS endpoint. Set AWS_DEFAULT_REGION in /root/.restic-env to the Signing region that lsh s3 stat "s3://$BUCKET" prints, or pass it per command with -o s3.region=<region>.
A previous run left a stale lock, usually after a crash or a killed process. Confirm no backup is actually running, then clear it:
restic groups chunks into pack files of roughly 16 MB rather than storing one object per file, so a backup of many small files does not create a matching number of objects. To see what the bucket is costing, run lsh s3 metrics "s3://$BUCKET" on your workstation: it prints the storage consumed in the current billing period and the estimated cost. The storage pricing page lists what each class of operation costs.

Next steps