dslreports logo
site
 
    All Forums Hot Topics Gallery
spc

spacer




how-to block ads


Search Topic:
share rss forum feed


koitsu
Premium,MVM
join:2002-07-16
Mountain View, CA
kudos:23

1 edit
reply to chrisretusn

Re: Why ever use a hard link?

FWIW, when I ran my co-lo for 18+ years, I went through many backup methods/models and ended up (very happily) using rsnapshot. On FreeBSD dump and restore are really wonderful, but only when they don't deadlock or screw up or piss off the kernel/underlying filesystem (which is the case today with SU+J / soft updates with journalling. You can't use dump any more on that due to SU+J design bugs that lock the system up).

rsync is used for the data transmission part (i.e. getting filesystem data off one machine and storing it on another) and the writing-to-the-disk part. You can use it across SSH (the default) or run rsyncd and use the native rsync protocol instead (which is often a better choice if you have lots of data, because even using SSH with an alternate cipher still makes SSH a network I/O bottleneck). We did use SSH and with a very specific set of settings (I had to test all the ciphers to see which was fastest, etc.).

rsnapshot just basically acts as a very intelligent wrapper (with config file support) around rsync.

We stored (at first) 30 days of backups, but as more and more customers began using more and more disk, by August 2012 we had cut that down to 12 days. Our backups were stored on a ZFS raidz1 pool consisting of three (3) 1TB disks, so we had 2TB of space to work with.

The advantage to rsnapshot is mainly ease of use when it comes to restoring from backups. I cannot tell you how much my users and few paying customers loved this thing, particularly since our backups were done nightly, across a dedicated gigE network, and stored on a server that exported the backups as an NFS filesystem. The boxes had an NFS mount (read-only) to filer:/backups/machinename, mounted locally as /backups, and one could do cd /backups ; cd daily.3 ; ls -l and get a full filesystem listing and treat that just as one would a full level 0 backup -- meaning despite rsnapshot/rsync only sending over the differences, due to use of hardlinks it acts/behaves like a full filesystem dump no matter if you're in daily.3 (4th-oldest backup) or daily.0 (most recent backup). Customers never needed to call me if they needed to restore some data from a backup -- they could do it themselves.

Certain things were excluded from the list (such as excluding /backups from the backup list ;-) ), and things like /dev, as /var/run, and lots of other places.

Anyway sorry for rambling, but rsnapshot is somewhat hard to describe in text. It's a lot easier to sit down with an SA and show them it. Once they see it in action it clicks almost immediately (once you bring up the hard links part) and they go "holy crap, that is SLICK".

You just gotta make sure you don't run out of space, as well as inodes, on your storage filesystem for the backups. :-)
--
Making life hard for others since 1977.
I speak for myself and not my employer/affiliates of my employer.


Bill_MI
Bill In Michigan
Premium,MVM
join:2001-01-03
Royal Oak, MI
kudos:2
Oh yes, it IS quite slick! I do use an rsync script to snapshot selected directories - all semi-manually. I'll definitely be looking into rsnapshot as I only maintain a single snapshot with rsync now.