Backup solution with Raspberry Pi – part 1

What I want to show you in this series is how to ...

  • (part 1) set up your Raspi to provide a mountable drive for your clients
  • (part 1) do an incremental and versioned back-up of your Linux clients to that drive via rsync
  • (part 2) mirror your backup to your friend's Raspi to have an (encrypted) copy in case of desaster
  • (part 2) include your phone or tablet into this scenario (well, kind of ...)
the hardware setup
(Icons made by srip from www.flaticon.com, licensed by CC 3.0 BY)

For this to work we only need the hardware and some basic Linux commands. This is what you'll need:

  • knowledge of basic Linux commands (sudo, mount, ...) and how to edit a file owned by root on the command line
  • ssh and root access (via sudo) on your Raspberry Pi
  • root access on the computers which are going to be backed-up

And these are the directories, device names and so on I will use in this article:

  • /dev/sda1: partition for the primary backup (see graphic above)
  • /media/Backup1: mount point for /dev/sda1
  • bean, luci: devices I want to backup (two Linux Mint laptops - my wife's and mine)
  • raspi: my Raspberry Pi
  • pi: the user I use to connect to my Raspiberry Pi (ssh pi@raspi)

Preparing your network

First, we must make sure, that the Raspberry Pi can address all laptops and desktops by their names or static IP addresses and vice versa.

static IP addresses and host names

To accomplish that I told my router (a Fritz-Box) to always give the same IP address to my Laptops and the Raspi. Then I edited the /etc/hosts file on all of them and added the names and corresponding IP addresses of the devices:

... other stuff here ...

192.168.150.20  bean
192.168.150.27  luci
192.168.150.60  raspi

... more other stuff here ...

I actually left out the corresponding line on the corresponding machine. Meaning: on bean I left out the "192.168.150.20 bean", on luci I left out the "192.168.150.27 luci" and so on.

Now I can address all of them by name or IP address. I prefer "by name". Like "ping luci" while being on raspi for example.

Another solution to accomplish this is to make your Router or your Raspberry Pi a DHCP server. But I wanted to keep it simple for now, concentrate on the backup task and not get caught up in other nasty fun stuff.

Setting up your Raspberry Pi

Ok, it's time to plug your designated backup drive into the Raspi right now. I formatted mine with an ext4 file system beforehand.

add the backup drive to /etc/fstab

Then create a mount point for it (here: mkdir /media/Backup1) and add your device to /etc/fstab. But first find out the UUID of your drive by issuing this command:

sudo blkid

My /etc/fstab entry for /dev/sda1 looks like this now:

UUID=a4ef22cd-b8ad-48f6-bd53-648805e9e58b /media/Backup1 ext4 defaults 0 1

I used the UUID instead of /dev/sda1 for device identification. This is more fail proof, because the partition /dev/sda1 of your backup drive may get a new name (e.g. /dev/sdb1) if you attach another drive to your Raspi.

NFS server

Now let's give the other devices access to the backup drive.

I did that by exporting the whole drive via NFS. NFS is perfect for laptops, because the mounts survive sleep mode flawlessly. You can also try Samba (SMB/CIFS); I won't, because I don't have any Windows installations here.

Some might say "Why do you want to mount that drive remotely? You can use rsync's remote copy abilities (root@raspi:/backup as target)!". Well, you could. But a) you would need to enter the Pi's root password every time you do a backup and b) I also have some media directories on that disk I want to provide to the client machines anyway and they will also be synced to the remote backup disk. (If you think not-mounting-the-drive-remotely is better for your purpose, just skip this part.)

Still on the Raspi install the NFS packages, ...

sudo apt install nfs-kernel-server

... edit /etc/exports and add something like this (all in one line) plus a helpful comment:

/media/Backup1 bean(fsid=0,rw,async,no_subtree_check,no_root_squash) luci(fsid=0,rw,async,no_subtree_check,no_root_squash)

# after changing anything here, do:
#   exportfs -ra && service nfs-kernel-server restart 

I added my two laptops (bean, luci) here by name. This means, no-one else should be able to mount the backup drive remotely.

You could only export a single directory of that drive, too. In my case I want to export the whole drive. It also holds GBs of media files which I want to be able to access from my laptops.

After editing /etc/exports run these commands to re-read the NFS configuration and restart the NFS server:

sudo exportfs -ra && sudo service nfs-kernel-server restart

NFS won't start at boot time

There's one problem with NFS left: if you'd reboot your Raspberry Pi now, you would see that the NFS server hadn't started at boot time. This is because NFS is started before rpcbind or so. But the NFS server needs rpcbind to run. We can tell the Raspi that there is a dependency between these two by adding this to a new file called /etc/systemd/system/nfs-kernel-server.service.d/10-dep.conf:

# https://raspberrypi.stackexchange.com/questions/69839/nfs-kernel-server-does-not-start-after-reboot
[Unit]
Requires=rpcbind.service
After=rpcbind.service

On the next boot the NFS server will start automagically now.

bonus: securing sudo on the Raspberry Pi

You might have noticed, that sudo on the Raspi doesn't ask for a password. This is odd. And insecure. It actually is insane.

But if you don't experience this or don't know what you're doing, please skip this part of this How-To. I also don't know on which versions of Raspian-OS this happens. Mine is a Raspberry Pi 3 B+ of November 2018.

To make sudo ask for a password I had to add the user pi to the sudo group. I even had to create the sudo group first:

sudo su -
groupadd -g 27 sudo
usermod -a -G sudo pi
# leave this shell open! If something goes wrong, you might lock yourself out!

Then uncomment the line starting with "pi" in /etc/sudoers.d/010_pi-nopasswd.

Now connect to your Raspi from another terminal session, to check if you can still do things like "sudo ls". If not, don't close the other ssh session and repair that!

Mount your backup drive

On the Linux devices you want to backup, install the NFS client first and create a mount point for the remote drive:

sudo apt install nfs-common
sudo mkdir /media/Backup1 # this is the local mount point for the drive

Add the drive to /etc/fstab now. My entries look like this:

raspi:/media/Backup1 /media/Backup1 nfs defaults 0 0

To check if mounting works, do this:

sudo mount /media/Backup1
sudo touch /media/Backup1/x # checks write access

In your nautilus or nemo or whatever file browser you use, you should see the drive now and be able to access it - depending on the rights you set on the files and directories on it.

Back it up now!

There's some more to do and think about: Write a backup script, choose where to put it, what to include in / exclude from the backup and finally how to start the backup...

If you don't want to write your own scripts, you can also try other backup solutions like Back In Time, rsnapshot or rsbackup. But I wanted to write my own:

a backup script to start with

The following is my backup script which I run on each of my clients bean & luci:

#!/bin/bash

# set terminal window title
wmctrl -r :ACTIVE: -N "Backing Up $HOSTNAME"

VOLUME="/media/Backup1"
TARGET="$VOLUME/backup-$HOSTNAME"

if [ ! -d "$TARGET" ]; then
  dialog --msgbox "$TARGET does not exist - aborting ..."
  exit 1
fi

# pre backup

# gather some infos
mkdir -p /root/backup/
dpkg -l > /root/backup/dpkg.get-selections

# dump databases
# mysqldump -u root -p******** --all-databases > /root/backup/mysql.dump

# END OF pre backup

# temp files
PROG=$( basename "$0" )
EXCLUDE="/tmp/$PROG-exclude"
ERRORS="/tmp/$PROG-errors"

# excluded paths
echo "# excludes
/bin/
/cdrom/
/dev/
/home/*/.cache/
/home/*/.config/google-chrome/Default/Service Worker/CacheStorage
/home/*/.dropbox/
/home/*/Dropbox/
/home/*/.local/share/Trash/
/home/*/MEGA/
/home/*/.mldonkey/temp/
/home/*/**/*.nobackup
/lib/
/lib32/
/lib64/
/lost+found/
/media/
/mnt/
/opt/
/proc/
/run/
/sbin/
/srv/
/sys/
/swapfile
/tmp/
/usr/
/var/
" > $EXCLUDE

# do backup to external drive
VERSIONING_TARGETDIR=$( date "+%Y-%m-%d_%H:%M:%S" )
rsync -a \
  --progress \
  --backup --backup-dir="$TARGET/$VERSIONING_TARGETDIR/" \
  --delete --delete-excluded \
  --exclude-from="$EXCLUDE" \
  / "$TARGET/latest/" \
  2> "$ERRORS"

CLOSEMSG="* you can close this window now *"
if [ -f "$ERRORS" -a $( cat "$ERRORS" | wc -l ) -ne 0 ]; then
  MSG="$HOSTNAME backup completed with errors:"
  ERRORS=$( cat "$ERRORS" )
  #spd-say "$HOSTNAME backup complete with errors"
  dialog --title "Backup completed" --msgbox "$MSG\n\n$ERRORS\n\n$CLOSEMSG" 1000 1000
else
  MSG="$HOSTNAME backup completed without errors"
  #spd-say "$HOSTNAME backup complete without errors"
  dialog --title "Backup completed" --msgbox "$MSG\n\n$CLOSEMSG" 9 50
fi

I changed it's access rights like this, so everyone can start it but only root can change it:

sudo chown root:root raspi-backup
sudo chmod 755 raspi-backup

It also uses the dialog command, so install that:

sudo apt install dialog

Some comments about the script:

  • $TARGET: this is the indiviual backup target
  • $HOSTNAME: Be aware: this is used to define the directory name into which the files are backed up. A malicious user could use this to overwrite another one's backup. Keep your host's names distinguishable!
  • # pre backup: here I do some things like making a list of the installed packages and dump my mysql database to a file
  • $EXCLUDE: this is the list of files and directories I don't want to backup. As I do a backup starting at the root directory / I exclude almost everything but /etc, /home and /root
  • $VERSIONING_TARGETDIR: this is the directory name for the --backup-dir option. It is a timestamp, for example "2019-01-02_15:23:04"
  • rsync options:
    • --delete and --delete-excluded: will delete files on the backup drive that are not existing on the client anymore
    • --backup: this tells rsync to not delete or overwrite anything - this is what makes the backup a versioned backuprsync's parameters:
    • --backup-dir: rsync before overwriting or deleting anything, rsync moves the old versions of these files to this directory
    • $TARGET/latest/: this is the directory where the backup will be put

where to put the script

I put the script on my backup drive under the name raspi-backup. It is suitable for both the laptops that I back-up with it.

If you need individual scripts for individual machines, you might just want to put them in the root directory of the laptops / desktop computers.

start the backup script

On your client machine the above script can be started like this now:

sudo /media/Backup1/raspi-backup

If you want to allow your user(s) to start it without entering a password (because of sudo), you can put something like this into your sudoers file by using the "sudo visudo" command:

krst ALL=(ALL) NOPASSWD: /media/Backup1/raspi-backup

krst is the username and raspi-backup is the script he is allowed to run without issuing a password. Nice and safe, eh?!

But wouldn't it even be nicer to just click on an icon on your desktop to start the backup? Let's do that!

On my Linux Mint 19 laptops I wrote this litle file named Backup.desktop and shoved it to my ~/Desktop folder. I think it will work on any Ubuntu based distribution:

[Desktop Entry]
Name=Backup
Exec=sudo /media/Backup1/raspi-backup
Terminal=true
Type=Application
Icon=/usr/share/pixmaps/faces/7_fox.png

My wife and me, we have different user names on our individual laptops. But their UIDs are the same (UID 1000). That's why we can peek into the other one's backed up files and they appear as if they are written with our own user. If you don't want this, you can ...

  1. change the UID of the users
  2. use /etc/export to only export the indiviual backup target (backup-$HOSTNAME) directory for the individual client

Final thoughts

This backup solution is basic, but it works without any backup file cataloges, strange or even proprietary backup file formats or databases. You can just restore your files from the backup media without any extra software.

If you have huge files like Virtual Box VMs in .vdi files, this can become quite nasty because it takes long time to back these files up. Especially over Wifi. Maybe I can find a solution for this.

possible improvements

  • Why not try letting the Raspberry Pi start the backup? It can ping the indiviual machines now and then and when they are up => grab the data. When done => sync to mirror backup media.

If you have any thoughts on this - feel free to leave a comment!

The second part is not released yet - I even haven't done the hardware setup right now... But it will come in the first quarter of 2019 - promised!

Have fun!

Leave a Reply

Your email address will not be published.