A DIY Backup Solution
At Imaginate we take backing up our work and keeping client's data safe and secure very seriously. Everyone knows they should back up but all too often it takes a disaster to make people actually do anything about it.
For business critical and client data we use secure external backup services, which I'll talk about in another post but for less sensitive data we have another, cheaper and more fun solution.
We keep most of our source code in BitBucket and while this is a highly stable and trusted platform, it would be most inconvenient if it were not available for any reason and we had a Raspberry Pi knocking about the office just begging to be useful.
What follows is a guide to exactly how we made an effective backup solution from a £15 Rasperry Pi and an external SATA hard drive.
What you'll need;
- 1 Rasperry Pi
- 1 External SATA HDD (ours is 2TB)
- 1 USB to SATA adapter
- 1 Micro SD card, at least 4GB (the faster the better)
- A screen and keyboard (for initial setup only)
1 - Grab an image of Raspbian from; https://www.raspberrypi.org/downloads/raspbian/
2- Burn it to the SDCard with Win32DiskImager; https://sourceforge.net/projects/win32diskimager/ (if you're on a PC)
3 - Connect your Pi to your network with a cable and plug in the screen and keyboard.
4 - Login as user: `pi` with the deault password of: `raspberry`
In the following, commands you type are general after the $. The pi@imagipi is the username and hostname (I set ours to be imagiupi, by default it'll be rasperry)
First things first, enable SSH so you can have your screen and keyboard back!
pi@imagipi:~ $ sudo raspi-config Select Interfacing Options Navigate to and select SSH Choose Yes Select Ok Choose Finish
Before we unplug and reboot, ensure we'll be able to connect it by setting a static IP address.
Get the details of your local network and DNS settings.
pi@imagipi:~ $ sudo ifconfig
Note the IP address, the gateway and the DNS servers and edit your dhcpcd.conf accordingly. Here's ours;
pi@imagipi:~ $ sudo nano /etc/dhcpcd.conf
interface eth0 static ip_address=192.168.4.156 static routers=192.168.4.16 static domain_name_servers=22.214.171.124 126.96.36.199
interface wlan0 static ip_address=192.168.4.156 static routers=192.168.4.16 static domain_name_servers=188.8.131.52 184.108.40.206
Save and reboot
pi@imagipi:~ $ sudo reboot
At this point you should be safe to disconnect your screen and keyboard and access the Pi over SSH.
Now, power down, make sure your external HDD is connected power everything up and login over SSH.
I use a Windows PC with Git BASH but you can use any SSH client such as Putty.
$ ssh email@example.com
Mount your external HDD.
First we scan to make sure it's detected;
pi@imagipi:~ $ sudo blkid
/dev/mmcblk0: PTUUID="63392ccc" PTTYPE="dos" /dev/mmcblk0p1: LABEL="boot" UUID="70CD-BC89" TYPE="vfat" PARTUUID="63392ccc-01" /dev/mmcblk0p2: UUID="8a9074c8-46fe-4807-8dc9-8ab1cb959010" TYPE="ext4" PARTUUID="63392ccc-02" /dev/sda: UUID="c4dc089a-7d09-4925-b30a-c53484fe66b8" TYPE="ext2
In my case, it's showing as/dev/sda.
pi@imagipi:~ $ sudo mkfs /dev/sda
When that's done (it might take some time, depending on the size of your drive), check the partitions.
pi@imagipi:~ $ sudo fdisk -l
Device Boot Start End Sectors Size Id Type /dev/mmcblk0p1 8192 137215 129024 63M c W95 FAT32 (LBA) /dev/mmcblk0p2 137216 60751871 60614656 28.9G 83 Linux Disk /dev/sda: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes
As we see, we have a 1.8TB partition on /dev/sda.
Mount that into a folder we can use.
I'll mount it straight into /mnt. You could create a folder inside /mnt if you wanted to mount more multiple devices there.
pi@imagipi:~ $ sudo mount /dev/sda/mnt
Get the permissions so the owner and group can write to it and everyone can read.
pi@imagipi:~ $ sudo chmod 775 /mnt
Now we want to make sure it mounts every time we reboot the Pi so edit your fstab to look something like this (add the line beginning /dev/sda);
pi@imagipi:~ $ sudo nano /etc/fstab proc /proc proc defaults 0 0 /dev/mmcblk0p1 /boot vfat defaults 0 2 /dev/mmcblk0p2 / ext4 defaults,noatime 0 1 /dev/sda /mnt auto defaults 0 0
Reboot to make sure it does.
pi@imagipi:~ $ sudo reboot
Now we should be able to browse the contents of the (currently empty HDD).
pi@imagipi:~ $ ls -la /mnt
Now, I wanted to backup our BitBucket repositories. We don't want to insert a password in the text file and it;s no good if we have to enter it for the backups to work so we'll need to create an SSH key;
pi@imagipi:~ $ sshkey-gen
Just accept the defaults and this will create 2 files (id_rsa and id_rsa.pub) in your home folder under .ssh.
Copy the contents of id_rsa.pub to your clipboard;
pi@imagipi:~ $ cat ~/.ssh/id_rsa.pub
And add that as new key in your BitBucket account.
Now, create a basic script to clone whichever repositories you want to backup. This one is very simple and just removes and reclones a respository.
pi@imagipi:~ $ nano repo_backup.sh cd ~ rm -Rf /mnt/repos/ mkdir /mnt/repos/ cd /mnt/repos/ git clone firstname.lastname@example.org:imaginatecreative/6081_hwr.git cd ~
Make it executable;
pi@imagipi:~ $ chmod +x repo_backup.sh
We want this to execute automatically every night, so lets add it as a cron job;
pi@imagipi:~ $ crontab -e 10 0 * ~/repo_backup.sh
This will kick off that script at 10 past midnight every day.
That's about all you need to do to actually make your Pi backup system but it's nice to be abel to see what's been backe dup and access it over your network from your desktop without having to SSH to the Pi, so I setup Samba file sharing.
pi@imagipi:~ $ sudo apt-get install samba samba-common-bin
Edit the Samaba config to look something like mine below, where ExternalHDD is the mounted SATA drive. If your network workgourp is anything other than WROKGROUP, change that too.
pi@imagipi:~ $ sudo nano /etc/samba/smb.conf
workgroup = WORKGROUP wins support = yes
[ExternalHDD] comment=External HDD path=/mnt browseable=Yes writeable=Yes only guest=no create mask=0777 directory mask=0777 public=yes
Reboot and you should be able to see the share "ExternalHDD" on your network and browse it like any other share.
Going forward, you'd want to add more repositories to backup to the script. I also installed Apache, PHP and MySQL on it and we use it as central server for sharing databases during internal local development.