Some of you have probably used rsync to backup files and directories from one machine to another. If one of those machines is in in an open network, you probably are doing it inside a ssh tunnel. If not, you should. And, it is really not that hard to do.
Let's say you wanted to copy directory called pickles inside the user bob's home directory at flyingmonkey.example.com, which is a Linux/Unix box out in the blue yonder. If you have rsync
installed (most Linux distros do come with it or offer it as a package), you could do something like:
rsync -az -e ssh bob@flyingmonkey.example.com:pickles /path/to/backup/dir/
The -e ssh is what tells rsync to do all of its monkeying about inside a ssh tunnel. And, when you run the above statement, it will then ask for bob's password and then proceed to copy the directory
~bob/pickles inside the directory /path/to/backup/dir. Which is great but I think we can do better.
Look Ma! No passwords!
First thing I want to get rid of is needing to enter a password. Yeah, it was great while we are testing it, but if we have a flyingmonkey loose in the internet, I would like to make it a bit harder for someone to break into it; I think I owe that to the Wicked Witch of the West.
The other reason is that then we can do the rsync dance automagically, using a script that is run whenever it feels like. In other words, backup. For this discussion we will just cover backup as in copying new stuff over old stuff; incremental backup is doable using rsync but will be the subject for another episode.
So, how are we going to do that? you may ask. Well, ssh allows you to authenticate using public/private key pairs. Before we continue, let's make sure sshd in flyingmonkey is configured to accept them:
bob@flyingmonkey:~$ grep -E 'PubkeyAuthentication|RSAAuthentication' /etc/ssh/sshd_config
#RSAAuthentication yes
#PubkeyAuthentication yes
#RhostsRSAAuthentication no
bob@flyingmonkey:~$
Since
PubkeyAuthentication and
RSAAuthentication are set to
yes, we are good to go. Now if flyingmonkey runs OSX, you would want to use
/etc/ssh/sshd_config instead.
A quick note on ssh keys: they are very nice way to authenticate because they make life of whoever is trying to break into your machine rather hard. Now, just guessing the password does not do you much good; you need to have the key. And, to add insult to injury, you can have a passphrase in the key itself.
Enough digressing. The next step is to create the key pair. The tool I would use in Linux/Solaris/OSX is ssh-keygen because I like to do command line thingies. So, we go back to the host that will be rsnc'ing to flyingmonkey and create it by doing
ssh-keygen -b 4096 -t rsa -C backup-key -f ~/.ssh/flyingmonkey
which will create a 4096 bit (a lot of places still use 1024 and some now are announcing they have new state-of-the-art ultra secure settings of 2048 bits. So unless your server can't handle it, use 4096 or better) RSA key pair called
flyingmonkey and
flyingmonkey.pub in your
.ssh directory:
raub@backup:~$ ls -lh .ssh/flyingmonkey*
-rw------- 1 raub raub 3.2K Dec 31 11:30 .ssh/flyingmonkey
-rw-r--r-- 1 raub raub 732 Dec 31 11:30 .ssh/flyingmonkey.pub
raub@backup:~$
During the creation process, it will ask for a passphrase. Since we are going to have a script using this keypair, it might not make sense to have a passphrase associated to it. Or it might, and there are ways to provide said passphrase to script in some secure way. But this post is getting long so I will stick to the easy basic stuff. If you remember, we said this is a public/private key authentication; that means it uses to keys: public and private. The public is taken to the machine you want to ssh into while the private stays, well, private. Let's look at the public key (it is a single line):
raub@backup:~$ cat .ssh/flyingmonkey.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACACsgpy/ihq31kv+Zji6Eknr46nbyx38uPE54X3STbaNC8oCheulVk
/+bTmrFy8Ne8RcTeWYd93wwabgBVDJYzjnsuwUgBO/JPXE4GiQrcnIz5fPsqJqslYxR5WnuUfkYPsAYgJL33XWZWi
dPu+A38OSxXf7UAfpKe5WXa93knXIERUA7NOzCKO96YzpW96i7LxAs20bsmNAw6bZbrZG8Dn3EssIK8CtEUvw4nWb
uFKZQS+b5AM8q20+IrGGVG193H6Rm3/iw0jip0VQOFozUB6yjToyZ5MTzShjb+f56o3+VUG2Bel7OMDfYXYYEKIoj
+cmTMLP5yu4v1t5dTkN3osneK/+2KHwXFTQY48TqxyqH+ZqEFy2X+kXoKff/89aD8lwj+uYKl6HNKhveKSZMNq/yc
7jCc05hLQCQkyC9/1lY9LI2UMHq2kqsgbdmR3uu3Oua2y1HhyeR9hqP9Om+kLu2K7cIXu5NBO9ro0vWBJII7T+z98
awbGH4jSryOxvAlpTT7d+POev13oOWonIwyTmkT72Q+/qJhPU/Vdtd7n5gSUomRT8dJQH+2hyA8c3+YPSW2VckBY/
Ax5aGX+AFoy1Y6WKpUWMIbwHJqizpdEd3WQWzivR1psfsjFzqrdG5SOSZFH2SHvzdNQOTz0FbYvgBV2Egq7WXv98C
se9ZDx backup-key
raub@backup:~$
You probably notices the
backup-key string on the end of the file; we put it there using the
-C (comment) option. Usually it writes the username@host, which is OK most of the times but I wanted something to remind me of what this key is supposed to do. You can change it later if you want.
So we go back to flyingmonkey and place the contents of the public key, flyingmonkey.pub in ~bob/.ssh/authorized_keys by whatever means you want. cut-n-paste works fine.
cat flyingmonkey.pub >> authorized_keys
also does a great job. Or you can even use
ssh-copy-id if you feel frisky. Just remember the contents of
flyingmonkey.pub is a single line. Of course, if
flyingmonkey is a windows machine, you will do something else probably involving clicking on a few windows, but the principle is the same: get the bloody key into the account in the target host you want to connect to.
Once that is done, connect using ssh by providing the key
ssh -i .ssh/flyingmonkey bob@flyingmonkey.example.com
Can you login fine? Great; now try rsync
rsync -az -e 'ssh -i .ssh/flyingmonkey' bob@flyingmonkey.example.com:pickles /path/to/backup/dir/
Do not continue until the above works. Note in a real script the private key will probably be somewhere only the user which runs the backup script can access.
Limiting
So far so good. We eliminated the need to use a password so we can write a script to use the above. But, we can still ssh using that key to do other things besides just rsync. Time to finally get to the topic of this post.
If the IP/hostname of the host you are backing up flyingmonkey from does not change, you can begin by adding that to the front of the ~bob/.ssh/authorized_keys entry for the flyingmonkey public key. Now, if the backup server is in a private/NATed lan, you want to use the IP for its gateway. In this example, let's say we all all inside a private lan and the IP for backup server is 192.168.42.24:
from="192.168.42.24" ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACACsgpy/ihq31kv+Zji6Eknr46nbyx38uPE54X3STbaNC8oCheulVk
/+bTmrFy8Ne8RcTeWYd93wwabgBVDJYzjnsuwUgBO/JPXE4GiQrcnIz5fPsqJqslYxR5WnuUfkYPsAYgJL33XWZWi
dPu+A38OSxXf7UAfpKe5WXa93knXIERUA7NOzCKO96YzpW96i7LxAs20bsmNAw6bZbrZG8Dn3EssIK8CtEUvw4nWb
uFKZQS+b5AM8q20+IrGGVG193H6Rm3/iw0jip0VQOFozUB6yjToyZ5MTzShjb+f56o3+VUG2Bel7OMDfYXYYEKIoj
+cmTMLP5yu4v1t5dTkN3osneK/+2KHwXFTQY48TqxyqH+ZqEFy2X+kXoKff/89aD8lwj+uYKl6HNKhveKSZMNq/yc
7jCc05hLQCQkyC9/1lY9LI2UMHq2kqsgbdmR3uu3Oua2y1HhyeR9hqP9Om+kLu2K7cIXu5NBO9ro0vWBJII7T+z98
awbGH4jSryOxvAlpTT7d+POev13oOWonIwyTmkT72Q+/qJhPU/Vdtd7n5gSUomRT8dJQH+2hyA8c3+YPSW2VckBY/
Ax5aGX+AFoy1Y6WKpUWMIbwHJqizpdEd3WQWzivR1psfsjFzqrdG5SOSZFH2SHvzdNQOTz0FbYvgBV2Egq7WXv98C
se9ZDx backup-key
This is a small improvement: only host that can connect is the one with this IP, be it legit or faking that. Test it.
Next step is specify which commands that can be run when connected using this key. And that one again will require playing with ~bob/.ssh/authorized_keys. This time we will specify the command:
from="192.168.42.24",command="/home/bob/.ssh/validate-rsync" ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACACsgpy/ihq31kv+Zji6Eknr46nbyx38uPE54X3STbaNC8oCheulVk
[...]
se9ZDx backup-key
And define
validate-rsync as
cat > .ssh/validate-rsync << 'EOF'
#!/bin/sh
case "$SSH_ORIGINAL_COMMAND" in
rsync\ --server\ --sender\ -vlogDtprze.iLsf\ .\ pickles)
$SSH_ORIGINAL_COMMAND
;;
*)
echo "Rejected"
;;
esac
EOF
chmod +x .ssh/validate-rsync
And this is where it get really exciting. All that
validate-rsync is doing is seeing if the command being sent is not only an rsync command but a specific one. Once we figure out how to get the proper
SSH_ORIGINAL_COMMAND, we can change the line
rsync\ --server\ --sender\ -vlogDtprze.iLsf\ .\ pickles)
to what it needs to be to match our backup script and test. Note that if you change the rsync statement, you will need to change the case.
