Need to sync some files? Locally or remotely? How about re-thinking an old friend, rsync?

You may be like I was, and have discounted rsync for a long time due to the security risk imposed by running the “r” daemon on your servers. Guess what? You can not only use rsync to sync up local directories on the same server (this can be real handy for backups), but you can also sync from one server to another via SSH rather than the rsync daemon. This would be much like scp, only you can sync whole directory trees.

So. Let’s say you want to sync two local directories, how would you do that? Well, if we are syncing /export/datadir to /export/backupdir it would look something like this:

rsync -aruv /export/datadir/ /export/backupdir/

It’s just that easy. Now, those command line switches, what do they do? Check it out:

a = archive
r = recursive
u = skip files that are newer on the receiving end
v = verbose, tell me what's going on

There is another one that is good when syncing between two separate servers, and that is the “z” switch. This tells rsync to use compression during the file transfer thus saving bandwidth. Let’s see what the above would look like from one server to another, as if you are running the command from the server you are syncing to:

rsync -aruvhz --progress server1:/export/datadir/ /export/backupdir/

There are a couple other options there, did you notice? I have added the “h” which tells rsync to output information in human readable format (GB, MB, K, etc), and the –progress which tells rsync to report exactly that, the progress of each transfer. You can use these with local transfers too, mix and match as you see fit.

For any of you that use MailWasher, have you gotten tired of manually re-entering all of your email account data every time you re-install, re-build or switch computers? I know I have. If you don’t have to do that very often, or if you don’t have more than one or two email accounts, it’s not that bad. But still, there is all of the data in your Friends list and Black list that you might not want to re-create. If you don’t know what MailWasher is, it’s a great email management and anti-spam tool that we featured as a Cool Tool awhile back, check it out.

Read the rest of this entry »

I have posted before about Backups and their importance in the world.  The subject has inspired me to write a whole article on the subject of Backups, Disaster Recovery and Mitigating Downtime in the hopes that I can explain a few things, share some ideas and observations, and maybe help you understand the different aspects of data protection that go into the subject of backups.  I hope it will prove helpful!

Greetings to all you surfers out there. I wanted to visit a topic that I most likely will not only visit often as time marches onward, but write about a bunch too, and that is backing up your data. Most folk don’t like to think about the bad things that might happen to your precious computer(s). This applies to home users and big time data center folk alike. I have found backups and the true art of backing up data to be one of the most mis-understood, under utilized and under appreciated art forms out there.

Lots of people don’t think about backups for one thing. Another is they have that “it won’t happen to me” mentality. Well, I have seen more than my share of drives fail, and brothers and sisters, it dang sure can happen to you. In fact, I have seen two statistical anomalies/impossibilities in the last couple of years alone, and that was two separate servers, made by two separate vendors, that at two separate times, had two drives fail at the same time. This took out a RAID 5 array on one server, and a mirror on the other. Luckily though, my team knows and understands backups so it was really a non-event, we pulled the data off of tape and were back in business.  Aaah, but if there wasn’t a good backup!!??

Another thing, I see folk (usually management) that don’t “get” backups. They think it’s some automagic thing that just happens because the backup operator pressed a button marked “Backup Your Stuff Now By Pressing This Button”. Well kids, it ain’t that easy. There are lots of factors and hurdles to think about and overcome when it comes to backing up your data, again that’s true whether it’s your home PC or a whole server floor full of machines. I’ll get into all of this later, but for now I just wanted to get the topic out in the lime light, and try to remind everyone that they need to backup their stuff!

One quick tip that I have for desktop users, like folk at home or in small offices. Go grab you an external hard drive, one that connects to your PC with the USB interface. That way you can turn it on when you need it, and simply copy all of your important stuff onto it. You can get software even, that will do the copy jobs for you. Some have nice features that track changes, compress and sort your files. That’s all well and good, but it’s not required. As long as you have a copy of the stuff that you need somewhere, you will be doing much better, and be much more worry and stress free when (notice I said when, not if) your main hard drive decides to go belly up.