dslreports logo
 
    All Forums Hot Topics Gallery
spc
Search similar:


uniqs
1854

Black Box
join:2002-12-21

Black Box

Member

Backup solution ideas

I am not sure that this is the right forum to post in, but I think I can get the best responses here.

At home I am running an essentially Linux shop, with an OpenIndiana server and a few virtual machines. It is comparable with a small business setup. My current "backup" solution is a combination of data replication amongst machines and USB key/DVD+RW copies for the most important files. This leaves a lot of less important files (read I wouldn't like it if I lose them, but I wouldn't lose any sleep over it) not covered and vulnerable.

I am currently looking for a proper backup solution. The total amount of data for the short to medium time horizon is between 4-8 TB. I am willing to spend a reasonable amount, but I don't have enterprise level financing for this. Please pitch in ideas and examples.

Thanks in advance.

DoItFree
@army.mil

1 recommendation

DoItFree

Anon

Write a script using find, tar, dd, and compress.

guppy_fish
Premium Member
join:2003-12-09
Palm Harbor, FL

guppy_fish to Black Box

Premium Member

to Black Box
Run Raid 1 on the Linux/VM server, with 6TB disks, your looking at $250

Have a NAS, also with RAID 1 as your backup store, $750. I use a Synology NAS, great box, plus can do so much more.

With 6TB, you can forget cloud based backup

PToN
Premium Member
join:2001-10-04
Houston, TX

PToN to Black Box

Premium Member

to Black Box
Bacula, Zmanda, Bareos, Burp. are good tools to get your backup on.

Bareos and Burp are forks off Bacula. Burp aims to make configs simpler, Bareos to add features.

Black Box
join:2002-12-21

1 edit

Black Box

Member

Thanks guys, but I was looking more towards the backup architecture than specific software. When I was in charge of the enterprise backup at a previous employer I was using LTO tapes and a combination of tar/dump for the Unix servers launced by crond. I even had hand-rolled scripts for the databases to ensure consistency. For inter-server replication usually rsync works very well.

guppy_fish See Profile: I am using RAID 1 on my workstation and the server is RAIDZ2 (I know, slow, but I don't need lightning fast speed). The current solution is somewhat equivalent to what you are proposing. A NAS is virtually useless when facing something like Cryptowall. I am looking for offline storage. Don't they say that RAID is not backup? ;)

Cloud backup is out of the question because privacy issues.

astroroxy
Premium Member
join:2013-07-26
Newport Beach, CA

astroroxy

Premium Member

said by guppy_fish See ProfileA NAS is virtually useless when facing something like Cryptowall. I am looking for offline storage. Don't they say that RAID is not backup?

Not true, simply use versioning , and it would be just fine. You can lock down a nas super tight so it is practically local storage.

Black Box
join:2002-12-21

Black Box

Member

said by astroroxy:

said by guppy_fish See ProfileA NAS is virtually useless when facing something like Cryptowall. I am looking for offline storage. Don't they say that RAID is not backup? ;)

Not true, simply use versioning , and it would be just fine. You can lock down a nas super tight so it is practically local storage.

Would you depend on this for a business customer? If the virus propagates to the NAS then it is game over. Also, if the virused workstation has access to the backup it can destroy the "backed up" files too. To allow automatic backup ssh would be used with stored keys, so the files are as good as local.

Then, there is the capacity issue. If the protected data is 4-8 GB then, to have 3 versions one would need 12-24 GB online storage (assuming no deduplication). This is a full blown storage array (3-5 grand populated), not NAS.
applerule
Premium Member
join:2012-12-23
Northeast TN
(Software) pfSense
ARRIS SB6183
Asus RT-N66

applerule

Premium Member

Here's my backup solution (which should protect from cryptowall). Note my home environment (at least the part I care to backup) is full MS.

1. All machines have File History/Shadow Copies turned on with Documents/Desktop mapped to a Server 08 box
2. Server 08 box has a scheduled task that runs every night and copies that data to another location (locally on the server that the clients do not have access to).
3. Server runs a 7z job that encrypts the files and splits into 500mb pieces
4. 7z files are moved into a folder that gets uploaded to Onedrive.
5. This is a "full backup" every time. Runs 3 days a week. Delete backups older than 14 days.

You could just do the same thing but instead of uploading to a cloud service just do rotating external HDD's. You could have a "monthly" HDD in a safe deposit box and do weeklys (and incremental dailies) or something on two other drives that you could put in a fire safe locally.

This would allow for versioning locally up to 7 days, and if that gets corrupt you have restore from the weeklys. If your house burns down or something you have a monthly in the safe deposit box.

IMO if you are not going to cloud backup you have to have some form of off-site storage if you want to truly backup your data. Using just a storage array leaves you with a big gaping hole in your backup solution.

astroroxy
Premium Member
join:2013-07-26
Newport Beach, CA

astroroxy to Black Box

Premium Member

to Black Box
said by Black Box:

said by astroroxy:

said by guppy_fish See ProfileA NAS is virtually useless when facing something like Cryptowall. I am looking for offline storage. Don't they say that RAID is not backup? ;)

Not true, simply use versioning , and it would be just fine. You can lock down a nas super tight so it is practically local storage.

Would you depend on this for a business customer? If the virus propagates to the NAS then it is game over. Also, if the virused workstation has access to the backup it can destroy the "backed up" files too. To allow automatic backup ssh would be used with stored keys, so the files are as good as local.

Then, there is the capacity issue. If the protected data is 4-8 GB then, to have 3 versions one would need 12-24 GB online storage (assuming no deduplication). This is a full blown storage array (3-5 grand populated), not NAS.

I would depend on this solution for a business customer. I use it for mine. Its simple, first my nas has built in virus protection, also it would have to be a very specialized version of a virus for a very custom version of linux to mess with the nas os. Lets say against all odds, a virus destroys all the data. Simply select a previous snapshot/version and bam, its all back. As for snapshots, it does not require that much extra space. For example my backup size is only about 2 tb, yet it holds about 30TB of backups. To top it off it synced with an offsite location.
Shady Bimmer
Premium Member
join:2001-12-03

Shady Bimmer to Black Box

Premium Member

to Black Box
said by Black Box:

guppy_fish See Profile: I am using RAID 1 on my workstation and the server is RAIDZ2 (I know, slow, but I don't need lightning fast speed). The current solution is somewhat equivalent to what you are proposing. A NAS is virtually useless when facing something like Cryptowall. I am looking for offline storage. Don't they say that RAID is not backup?

RAIDZ2 is not specifically "slow". Any parity-based raid has a potential performance impact due to parity calculation but with any modern hardware this is not a particular concern.

RAID is not a "backup" solution itself, as you've noted. Using alternate storage, whether it is disk, tape, optical, or other is the underlying concept. This may be online or offline.

You can use your RAIDZ2 volumes as backup targets. How you backup is more important. Your assumption that NAS is useless against Cryptowall is not necessarily correct or relevant - again it depends how you perform your backups.

You also need to identify what you require when you say "backups". Do you need long-term archiving? Do you need disaster-recovery?

You don't mention any details on your server or clients. Samba or native CIFS service can provide direct access to windows clients. Netatalk, CIFS, or NFS can provide access to MacOS clients. ZFS snapshots can provide point-in-time copies.

For instance you can use native windows backup to save to a CIFS-shared volume for windows clients, and Netatalk can appear as a Time Machine target (Time Capsule).

You can layer additional encryption on top of "cloud" solutions. There are also client-side encrypted cloud solutions such as SpiderOak and Tresorit.

If you are using a Windows Server you may be able to use that for client backups to your ZFS storage (via iSCSI as one option).

One architecture involving Unix/Linux clients, Windows clients, and MacOS clients: Solaris-derivative server with ZFS RAIDZ2 arrays, shared via NFS, CIFS, and Netatalk. MacOS clients backup directly using Time Machine, Windows clients backup via Windows Server Essentials to an iSCSI volume, and Unix/Linux servers backup via NFS. SpiderOak used to backup critical items from that ZFS storage offsite to secure encrypted storage.
Shady Bimmer

Shady Bimmer to Black Box

Premium Member

to Black Box
said by Black Box:

This is a full blown storage array (3-5 grand populated), not NAS.

My home "NAS" solution provides 36TB online storage. Commercial and Enterprise NAS solutions can offer Petabytes. NAS simply means network-based file storage. It has no implications on size or scaling.

Modern storage solutions blur the lines between SAN and NAS, often offering both block and file services. In other words these can appear both as SAN and NAS, in the traditional definitions.

Black Box
join:2002-12-21

Black Box to Shady Bimmer

Member

to Shady Bimmer
Well, the OpenIndiana server with RAIDZ2 is the main storage that I want to protect. The clients back up data on the server. This is handled already. I want to be able to do disaster recovery and if I can recover an older mangled file that would be a bonus. I don't need long-time archiving, as I don't have data retention requirements. Being able to restore a file that was deleted a month ago is about as much as I need.

I can use ZFS snapshots for normal operations, but if the server is compromised I will assume that all the snapshots are compromised (or outright dropped) too. My disaster recovery scenario is all local machines are compromised, everything with online storage needs nuke-an-pave before being reconnected to the network. No chances taken.

The clients and virtual machines are mostly Linux. The virtual machines are mostly on the server (always on services), a few on the clients (sometimes on only). I have a Windows virtual machine without network access running sometimes on a client. The sharing is done via NFS, sshfs and sftp. The first two are the issue, as they provide direct access to the remote files. There is no SMB/CIFS as I don't need to support Windows.

A compromised server would possibly mean compromised SpiderOak account/keys too. Recently malware started to go after passwords and keys to things like KeePass and 1Password so SpiderOak cannot be far behind.

Out of curiosity, what is your solution for the 36TB NAS? I have to think about this in the long run, when I'll outgrow my current setup.

dennismurphy
Put me on hold? I'll put YOU on hold
Premium Member
join:2002-11-19
Parsippany, NJ

dennismurphy to Black Box

Premium Member

to Black Box
I also have a Synology NAS at home ... so I backup to it, and then replicate across a VPN link to another Synology NAS I put at a family member's house.

It rocks.

tubbynet
reminds me of the danse russe
MVM
join:2008-01-16
Gilbert, AZ

tubbynet

MVM

said by dennismurphy:

I also have a Synology NAS at home ... so I backup to it, and then replicate across a VPN link to another Synology NAS I put at a family member's house.

i would like to second the sheer kick-assedness of the synology.
i purchased one for my home storage needs at the recommendation of dennismurphy See Profile (the guy does know what he's talking about).
purchased the ds1815+ with (8) 4tb hgst 7200 nas drives in an shr2 (basically a fancy raid6). gives me a little under 22tb usable.
plus -- there are articles out there to synch the whole thing to crashplan or so. haven't done that yet -- as i've had other committments -- but the box kicks much ass.

q.

dennismurphy
Put me on hold? I'll put YOU on hold
Premium Member
join:2002-11-19
Parsippany, NJ

dennismurphy

Premium Member

said by tubbynet:

i would like to second the sheer kick-assedness of the synology.
i purchased one for my home storage needs at the recommendation of dennismurphy See Profile (the guy does know what he's talking about).
purchased the ds1815+ with (8) 4tb hgst 7200 nas drives in an shr2 (basically a fancy raid6). gives me a little under 22tb usable.
plus -- there are articles out there to synch the whole thing to crashplan or so. haven't done that yet -- as i've had other committments -- but the box kicks much ass.

Well thanks for the endorsement... I'm happy with the thing - really does work so well!
Shady Bimmer
Premium Member
join:2001-12-03

1 edit

Shady Bimmer to Black Box

Premium Member

to Black Box
said by Black Box:

I can use ZFS snapshots for normal operations, but if the server is compromised I will assume that all the snapshots are compromised (or outright dropped) too. My disaster recovery scenario is all local machines are compromised, everything with online storage needs nuke-an-pave before being reconnected to the network. No chances taken.

How would you know that your alternate copies are not also compromised? If you use any form of automated solution you run that same risk.

ZFS snapshots are immutable: you can only destroy the snapshot and you can't modify the snapshot in any way.

You could send ZFS streams from your snapshots to another location (and store it as such), if you trust your alternate storage solution. This can be done manually or it can be automated.

A compromised server would possibly mean compromised SpiderOak account/keys too. Recently malware started to go after passwords and keys to things like KeePass and 1Password so SpiderOak cannot be far behind.

This would be the case with any solution.

The method used by SpiderOak (or other client-side-encrypted solutions) is completely different than that used by password managers. Your credentials would not be compromised, however access to the server could provide access to the remote copies - again this would be the case with any solution unless you manually enter credentials every time, and only after ensuring absolute integrity of your server.

Out of curiosity, what is your solution for the 36TB NAS? I have to think about this in the long run, when I'll outgrow my current setup.

I only selectively transfer what I specifically want "offsite" - certainly not all 36TB. Encrypted containers and client-side-encrypted remote storage solutions provide what I need.
Shady Bimmer

Shady Bimmer to dennismurphy

Premium Member

to dennismurphy
said by dennismurphy:

I also have a Synology NAS at home ... so I backup to it, and then replicate across a VPN link to another Synology NAS I put at a family member's house.

Something like this sounds like a good option for the OP if there is a remote location with broadband available. The second server could also be kept onsite but "disconnected" until a backup is desired.

Instead of between a pair of Synology NAS units, this can also be done easily between a pair of servers with ZFS (since the OP is already using ZFS). Commercial ZFS-based storage solutions automate this, but it is not difficult to hand-automate (even if manually initiated) with OpenIndiana - there may already be scripts to do this in fact.

The VPN could be manually brought up to initiate a backup then torn back down when complete. This would address the concerns of the OP over compromise of everything online.

The alternate storage does not even need to be ZFS - any filesystem can hold ZFS streams.

dennismurphy
Put me on hold? I'll put YOU on hold
Premium Member
join:2002-11-19
Parsippany, NJ

dennismurphy to Black Box

Premium Member

to Black Box
Plug one of these in to a USB port and write a hash of the encrypted backup to it as the backup is done. Then you can do a compare before you 'trust' any data in it.

»www.amazon.com/SanDisk-F ··· 52355778
dennismurphy

dennismurphy to Shady Bimmer

Premium Member

to Shady Bimmer
said by Shady Bimmer:

said by dennismurphy:

I also have a Synology NAS at home ... so I backup to it, and then replicate across a VPN link to another Synology NAS I put at a family member's house.

Something like this sounds like a good option for the OP if there is a remote location with broadband available. The second server could also be kept onsite but "disconnected" until a backup is desired.

The VPN could be manually brought up to initiate a backup then torn back down when complete. This would address the concerns of the OP over compromise of everything online.

Bingo. That's essentially how my setup works. The remote site initiates a VPN tunnel, does a 'pull' of the data and tears down the tunnel.

At no time does the remote Synology expose any file services (NFS, CIFS et al.). It's purely a backup target.

So you'd need to smash my primary box, compromise the VPN server, the Synology backup service and the SSH tunnel it runs across, all within the "dial in" window of when the remote side connects.

And even then, I have version preservation turned on so at worst, you'd give me a junk copy. I just recover a known-good PIT and I'm good to go.

I do think I'll add that immutable flash drive and save the MD5 sums so I have something to compare against. I guess that's over-the-top paranoia.

guppy_fish
Premium Member
join:2003-12-09
Palm Harbor, FL

guppy_fish to tubbynet

Premium Member

to tubbynet
Actually its a third , I posted first and have had my Synology for 4 years now, best thing I have ever bought is how much I like the box. Its incredible what you can do with it.

Black Box
join:2002-12-21

Black Box to Shady Bimmer

Member

to Shady Bimmer
said by Shady Bimmer:

How would you know that your alternate copies are not also compromised? If you use any form of automated solution you run that same risk.

I don't. I have to assume that I catch the infection within a month. The buck has to stop somewhere. Otherwise I have to add long-term data archiving so I can revert to a point before the infection. dennismurphy See Profile has a good idea, storing a hash on a separate media to confirm that the files restored are the files that were visible at backup. I have to think about this.
said by Shady Bimmer:

ZFS snapshots are immutable: you can only destroy the snapshot and you can't modify the snapshot in any way.

Agreed that snapshots are immutable with the kernel driver. They can though be dropped (bye-bye data) or modified with direct access to the device, in the manner GRUB installs itself.
said by Shady Bimmer:

said by Black Box:

A compromised server would possibly mean compromised SpiderOak account/keys too. Recently malware started to go after passwords and keys to things like KeePass and 1Password so SpiderOak cannot be far behind.

This would be the case with any solution

Not if the storage media is offline. You cannot change data on a tape/disk sitting in a drawer somewhere away from the computers, even offsite. You can easily feed the backup media before the backup's scheduled start and remove it when the backup is complete. I hope that I didn't made it to the Equation Group sheet list (I'm not that important) to have to deal with firmware creepy crawlers, so checking backup from a known good bootable CD should be enough.
said by Shady Bimmer:

I only selectively transfer what I specifically want "offsite" - certainly not all 36TB. Encrypted containers and client-side-encrypted remote storage solutions provide what I need.

This is similar to my USB key/DVD backups reserved for the most important data.

That was a side question. I was just thinking what could be my architecture when I outgrow my 6 bays server, so I asked what is what you are using. Next question would have been "is it capable of ZFS"? Probably I should start a new topic, but it is too early for me.
Black Box

Black Box to tubbynet

Member

to tubbynet
said by tubbynet:

i purchased one for my home storage needs at the recommendation of dennismurphy See Profile (the guy does know what he's talking about).
purchased the ds1815+ with (8) 4tb hgst 7200 nas drives in an shr2 (basically a fancy raid6). gives me a little under 22tb usable.
plus -- there are articles out there to synch the whole thing to crashplan or so. haven't done that yet -- as i've had other committments -- but the box kicks much ass.

q.

Hey, that's in the price range of a LTO6 setup, maybe with a library attached too!

tubbynet
reminds me of the danse russe
MVM
join:2008-01-16
Gilbert, AZ

tubbynet

MVM

possibly -- but this houses all shared file storage, dlna to streaming devices, time machine and windows backups, as well as serves as storage for my vmware hosts (intel nuc that i boot from san).

much more than lto6 could offer.

q.
Shady Bimmer
Premium Member
join:2001-12-03

Shady Bimmer to Black Box

Premium Member

to Black Box
said by Black Box:

I don't. I have to assume that I catch the infection within a month. The buck has to stop somewhere.

That was exactly my point. You seem concerned over compromise and destruction of your backup of your backup - either you are or you are not.

Agreed that snapshots are immutable with the kernel driver. They can though be dropped (bye-bye data) or modified with direct access to the device, in the manner GRUB installs itself.

Yes, snapshots can only be destroyed. I don't know what you mean by direct access to the device - you may want to learn more about ZFS. It is not clear how that is relevant here, but if something external tries to modify the data on the disk directly, ZFS will detect and correct this (since you have double-parity).

if someone has physical access to your server they likely would have access to any other on-site copies you may have (even if offline). If that is a real concern then you really do need a remote solution, even if it means physically carrying your backup solution out the door with you.

You need to determine exactly what you are trying to protect against (you didn't mention physical access as a concern).

The idea is that you can use snapshots to create immutable copies. Those can then be sent to another storage solution (ZFS streams) - this is actually one common standard for backing up ZFS. It is not meant to use snapshots are your long-term backup solution itself.
said by Black Box:

Not if the storage media is offline. You cannot change data on a tape/disk sitting in a drawer somewhere away from the computers, even offsite.

Above you expressed concern over destruction of data - not just compromise. I can destroy a tape or disk sitting in a drawer very easily.

That was a side question. I was just thinking what could be my architecture when I outgrow my 6 bays server, so I asked what is what you are using. Next question would have been "is it capable of ZFS"? Probably I should start a new topic, but it is too early for me.

This was actually already answered.

Use another storage device to which you copy your ZFS streams. If that device supports ZFS you can do a send/receive to replicate your selected datasets and have your data immediately available when needed. If that device does not support ZFS you would simply store the streams as files for recovery at a a later time to your ZFS storage.

That alternate storage can be local or can be remote at a site you trust.

My ZFS storage services NFS, CIFS, AFS (apple, not andrew), and iSCSI. Among other things it is a backup target, and from there I replicate to other solutions as needed via multiple different methods.

Use of a second similar server in a remote location as noted by dennismurphy See Profile sounds like the best option for you. Since you are using ZFS you can either recursively snapshot and send all datasets/volumes in your pool(s) or selectively do so with only specific datasets/volumes.

Black Box
join:2002-12-21

Black Box

Member

said by Shady Bimmer:

That was exactly my point. You seem concerned over compromise and destruction of your backup of your backup - either you are or you are not.

My disaster scenario is: once a machine is compromised, the cracker easily jumps to all machines available online and deletes/overwrites/encrypts all data accessible. Offline backups are physically protected and inaccessible to the cracker. The intrusion is detected before the oldest backup is overwritten. Hardware failures are handled with online copies. A simultaneous physical compromise and software cracking cannot be handled and results in catastrophic data loss.
said by Shady Bimmer:

I don't know what you mean by direct access to the device - you may want to learn more about ZFS.

I've learned more than I care. I was hacking recently OpenIndiana's GRUB to install and boot from GPT partitions. The install process does not use the kernel ZFS driver. It goes directly to the raw device, updates the block list(s) in the stage1.5/stage2 files and fixes the hashes in the block pointers, so errors are not detected at access/resilver time.
Shady Bimmer
Premium Member
join:2001-12-03

Shady Bimmer

Premium Member

said by Black Box:

I've learned more than I care. I was hacking recently OpenIndiana's GRUB to install and boot from GPT partitions. The install process does not use the kernel ZFS driver. It goes directly to the raw device, updates the block list(s) in the stage1.5/stage2 files and fixes the hashes in the block pointers, so errors are not detected at access/resilver time.

Boot support with GRUB on a ZFS root pool is a special case, and is uniquely handled. In other words ZFS specifically facilitates the unique requirements of booting and managing booting.

Go ahead and try to do the same manipulations anywhere else in any ZFS pool. The real question is if you don't trust ZFS then why use it? No backup solution will help if you don't trust the integrity of the original data in the first place.

You asked for suggestions and I've identified what is typically done in enterprises, commercial installations, educational institutions, home installations, and others using ZFS If a more traditional backup infrastructure is not used. "zfs snapshot ..." periodically; "zfs send ... " ... "zfs receive ..." occasionally. Very simple, reliable, and efficient. Very simple architecture, and for what it is worth pretty well documented and described in various forums, wikis, knowledge bases, etc.

The two pools can be physically connected to the same server or the two pools can be connected to two physically different systems. The former is not an option for you so the latter is your choice.

From there, the two servers can be physically co-located or physically remote from each other. It is not clear but it does not sound like the former is suitable for you so the latter is your choice. (hint: This is what dennismurphy See Profile is suggesting)

A network connection between the two may be persistent, or transient for use only when the backup is occurring. Again the former is not an option for you so the latter is your choice. (hint: This is what dennismurphy See Profile is suggesting)

If you want to copy the entire ZFS pool then you need to have the same amount of storage at your destination. If you don't, then proper and judicious use of ZFS datasets (filesystems/volumes) is how you would control/manage what gets copied and does not (hint: This is exactly what I do).

Black Box
join:2002-12-21

Black Box

Member

I trust ZFS until the machine is compromised. Then all bets are off. Heck, even a :; cat /dev/zero > /dev/rdsk/c0t0d0 would to ZFS in! Try use data from yesterday's snapshot after that :D .

I don't understand why are you rejecting my disaster scenario. The proponents (including you) do not cover my scenario, and this is fine. Everybody is entitled to define his own scenarios and I respect that. I just want ideas on how to cover mine. I need to be able to nuke every online (permanent or on demand) machine before taking out the backup from its drawer. The solutions presented until now do not cover that.

The major weakness in my setup is that I don't have silos. I have been doing this kind of setup when I was the IT department at work, but I don't have the time nor the hardware to implement such thing at home. All machine have matched accounts, so I can easily access data across the network. If I can access everything anywhere easily, a cracker can access everything anywhere easily.

To be safe I need the backup to be inaccessible. Online/cloud backups do not cut it. Even with on-demand connection there is a chance that before detection the cracker hops to the "backup" as soon as the connection is manually established. Then simply trigger the destruction simultaneously. Curtains.

Could we please return to ideas on how best to cover this scenario? Thanks.
Shady Bimmer
Premium Member
join:2001-12-03

1 edit

Shady Bimmer

Premium Member

said by Black Box:

I trust ZFS until the machine is compromised. Then all bets are off. Heck, even a :; cat /dev/zero > /dev/rdsk/c0t0d0 would to ZFS in! Try use data from yesterday's snapshot after that :D .

Have you actually tried that?

Hint: that is one of the standard tests to validate ZFS since just about day 1. It is also the specific example well documented for ZFS reliability.

I don't understand why are you rejecting my disaster scenario.

I'm not rejecting. Just trying to understand your true requirement.

The proponents (including you) do not cover my scenario, and this is fine.

How do they not cover your scenario?

I need to be able to nuke every online (permanent or on demand) machine before taking out the backup from its drawer. The solutions presented until now do not cover that.

Again, how do they not cover that?

To be safe I need the backup to be inaccessible. Online/cloud backups do not cut it. Even with on-demand connection there is a chance that before detection the cracker hops to the "backup" as soon as the connection is manually established. Then simply trigger the destruction simultaneously. Curtains.

If your concern is that a compromise of your system will destroy your backup then you only have one option: write-once immutable media. Primarily this will be write-once optical media, particularly for the sizes you are looking for.

I do note a previous reply you made:
said by Black Box:

said by Shady Bimmer:

How would you know that your alternate copies are not also compromised? If you use any form of automated solution you run that same risk.

I don't. I have to assume that I catch the infection within a month. The buck has to stop somewhere. Otherwise I have to add long-term data archiving so I can revert to a point before the infection.

What you note here is different than what you are now saying. Hence the confusion. The reason I asked that question was specifically to scope your options.

Also for what it is worth your current solution of backing up to a USB key or DVD-RW media (as noted in your first post) is just as susceptible, if not more, as the suggestions made already.

There is no architecture involved here so it is not clear what else you need.
applerule
Premium Member
join:2012-12-23
Northeast TN
(Software) pfSense
ARRIS SB6183
Asus RT-N66

1 edit

applerule to Black Box

Premium Member

to Black Box
said by Black Box:

Could we please return to ideas on how best to cover this scenario? Thanks.

If you don't want to do online backups, the only viable option I'm aware of you're going to have involves implementing something like I mentioned before. Simply replace HDD with whatever "media" you want...tapes would be an option too.

You're talking at least $2k (and that's a probably a bare minimum for your data size) to have two identical SAN's if you wanted to rotate through 2 boxes (store one off site, and have one local). You could implement rotating hard drives with that amount of data for under $1500. A tape library is an option too but is going to be more expensive than drives.

You absolutely need some form of off-site data storage. That is going to be one of the most cost effective ways to do it without some form of cloud/online solution.

What is your budget like? When you're talking about a home setup I would expect most people to stay well under a $2k budget...

FWIW we have been hit with cryptolocker at my current employer before and were able to restore from snapshots with no issue at all. You're talking a pretty hardcore piece of malware that would have to be in the appliance to be able to corrupt that amount of data (as others have mentioned).

Black Box
join:2002-12-21

Black Box to Shady Bimmer

Member

to Shady Bimmer
said by Shady Bimmer:

said by Black Box:

I trust ZFS until the machine is compromised. Then all bets are off. Heck, even a :; cat /dev /zero > /dev/rdsk/c0t0d0 would to ZFS in! Try use data from yesterday's snapshot after that :D .

Have you actually tried that?

Hint: that is one of the standard tests to validate ZFS since just about day 1. It is also the specific example well documented for ZFS reliability.

Actually yes. You just have to repeat it for mirrors and RAIDZx, but you can reliably nuke the pool. Updated code to completely nuke a system:

# Run as root NOT!
 
for d in /dev/rdsk/c*t*d* do
    cat /dev /zero > $d &
done
 
said by Shady Bimmer:

How do they not cover your scenario?

The solutions proposed rely on online replicas/snapshots. If it is online in my scenario it is as good as gone.
said by Shady Bimmer:

If your concern is that a compromise of your system will destroy your backup then you only have one option: write-once immutable media. Primarily this will be write-once optical media, particularly for the sizes you are looking for.

Your statement misses one key word: "online". Online backups can be destroyed, offline (as in sitting in a drawer somewhere, even if writable) cannot. I cannot spend the time and money to properly lock down an online system so it won't go down if my credentials are compromised. Optical media could work, just it is not scalable. It is hard to manage terabytes of data dumped on myriad of discs.

Using alternating DVD+RW and USB keys is OK, because in my scenario I am able to discover the intrusion and initiate the recovery before data loss. The writable media would be connected in this case to a freshly nuked and paved system.