Anyone who has ever used SSH key-pairs to access more than a couple of servers (or hundreds in my case), will tell you they are an invaluable convenience. It is a natural progression and very common usage that SSH key-pairs are coupled with other common tasks or tools, where having a pass phrase attached to the key would be counter-intuitive to the task automation. So, what do we do despite our better judgment? We create key-pairs with absolutely no pass phrase. The implications are abundantly obvious, if the private key ever gets lost or stolen, any accounts that have the key-pair associated to it can be instantly compromised.
In the case of my recently released project Incremental Rsync (IRSYNC), one of the implementation hurdles at work was to have servers backup using a secure medium. This is easily handled with rsync’s -e option to have data transferred over ssh using a key-pair but then the obvious issue comes up that what if a client server ever gets compromised? Then the backup account on the backup server can be compromised (please don’t use root!@#!@#) allowing for backups to be deleted or worse yet data to be stolen for every server that backups to said server/account.
A solution to this is to limit the commands that can be executed over SSH by a specific public key, though this is not a perfect way to mitigate the threat it does go a long way to help. For my backup server implementation I have setup the user ‘irsync’ on the backup server, this account has the usual ‘~irsync/.ssh/authorized_keys’ file where I place the public key. Where things differ is that you prefix a script path in front of the public key that is used to interpret commands sent over ssh, which looks something like this:
command="/data/irsync/validate-ssh.sh" ssh-dss AAAAB3NzaC1kc3MAAAC......87JVNLJ5nhaK1A== irsync@irsync
The ‘validate-ssh.sh’ script is basically a simple interpreter, it looks at the commands being passed over ssh and either allows them or denies them with some logging thrown in for auditing purposes. The script can be downloaded from: http://www.rfxn.com/downloads/validate-ssh.sh. Please take note to edit the scripts ‘log_file=’ value to an appropriate path, usually the base backup path or user homedir.
An example of validate-ssh.sh in play would be as follows, first the client side view then the logs from $log_file:
root@praxis [~]# ssh -i /usr/local/irsync/ssh/id_dsa irsync@buserver3 "rm -rf /some/path"
sshval(13156): ssh command rejected from 192.168.3.33: rm -rf /some/path
root@praxis [~]# ssh -i /usr/local/irsync/ssh/id_dsa irsync@buserver3
sshval(13403): interactive shell rejected from 192.168.3.33
May 04 11:36:15 buserver3 sshval(13156): ssh command rejected from 192.168.3.33: rm -rf /some/path
May 04 11:40:03 buserver3 sshval(13403): interactive shell rejected from 192.168.3.33
On the flip side when a command is authorized, it gets recorded into the $log_file as follows:
May 04 05:29:08 buserver3 sshval(29993): ssh command accepted from 10.10.6.6: rsync --server -lHogDtprx --timeout=600 --delete-excluded --ignore-errors --numeric-ids . /data/irsync/mysql02.mynetwork.com.full
Take note that if you do choose to use validate-ssh.sh with irsync, you will need to create your own script to manage the snapshots as internally irsync uses the find command, piping results to xargs and rm which will not be authorized by validate-ssh.sh (for good reason!). This is actually a very simple task, although all your snapshots will have to use the same rotation age (whatever).
#!/bin/sh
age=14
bkpath=/data/irsync
for i in `ls $bkpath | grep snaps`; do
wd=$bkpath/$i
find $wd -maxdepth 1 -mtime +14 -type d | xargs rm -rf
done
You can save this to /root/irsync_rotate.sh, chmod 750 it and run it as a daily cronjob by linking it into /etc/cron.daily/ (ln -s /root/irsync_rotate.sh /etc/cron.daily/) or you can add an entry into /etc/crontab as follows:
02 4 * * * root /root/irsync_rotate.sh >> /dev/null 2>&1
Although I detailed the use of validate-ssh.sh in the context of backups with irsync, this could easily be adapted to any usage when you want to restrict the commands executed over ssh with key pairs. You could even create your own script in perl or whatever floats your boat and use that instead — if you happen to go that route, please share with me what you created in the comments or by e-mail to ryan <at> rfxn.com.