dslreports logo
site
 
    All Forums Hot Topics Gallery
spc

spacer




how-to block ads


Search Topic:
uniqs
4750
share rss forum feed


El Quintron
Resident Mouth Breather
Premium
join:2008-04-28
Etobicoke, ON
kudos:4
reply to Davesnothere

Re: DSLR Restore Progress

Wouldn't once a week, or even every day at midnight be necessary for a site with this volume?
--
Everything in Moderation... including moderation.


Davesnothere
No-BHELL-ity DOES have its Advantages
Premium
join:2009-06-15
START Today!
kudos:7

2 edits
reply to Paolo
 
@ El Quintron & Paolo :

Absolutely, but I was just making a general point regarding relative severity.

Incremental backups once daily at about 5 AM Eastern time might be optimal.

And now, having said all of that, I really ought to back up some more of my OWN personal stuff right away.


El Quintron
Resident Mouth Breather
Premium
join:2008-04-28
Etobicoke, ON
kudos:4
Reviews:
·TekSavvy Cable
·TekSavvy DSL
said by Davesnothere:

 
And having said all of that, I really ought to back up some more of my OWN stuff right away.

An easy way to prevent a catastrophe that I've found, is by keeping your datas and OSes on separate drives, at least that way if one drive goes you're not completely screwed.
--
Everything in Moderation... including moderation.


Davesnothere
No-BHELL-ity DOES have its Advantages
Premium
join:2009-06-15
START Today!
kudos:7
said by El Quintron:

An easy way to prevent a catastrophe that I've found, is by keeping your datas and OSes on separate drives, at least that way if one drive goes you're not completely screwed.

 
....which of course is not the Windoze default setting.

I do keep some of my data on other physical drives, even better on other PCs across my LAN, and/or on external USB hard drives.

But there is some with which I have been complacent and need to catch up - docs, bookmarks, & downloaded web pages - all of which default to C: .
--

START Forum »Start Communications
Or you can still use Canadian Broadband.
START Website »www.start.ca

MaynardKrebs
Heave Steve, for the good of the country
Premium
join:2009-06-17
kudos:4
reply to Davesnothere
Doing a database snapshot (ie. maintaining consistency) for backup purposes requires some sort of add-on for most SQL databases -- if you keep the db server running during the backup. Most of these add-ons cost real money.


El Quintron
Resident Mouth Breather
Premium
join:2008-04-28
Etobicoke, ON
kudos:4
Reviews:
·TekSavvy Cable
·TekSavvy DSL
reply to Davesnothere
said by Davesnothere:

....which of course is not the Windoze default setting.

Of course not... which is silly because both Linux and OS X separate the OS from the Data by default... even if they don't go all the way by using separate partions, recovery is substantially easier.


El Quintron
Resident Mouth Breather
Premium
join:2008-04-28
Etobicoke, ON
kudos:4
Reviews:
·TekSavvy Cable
·TekSavvy DSL
reply to MaynardKrebs
said by MaynardKrebs:

Doing a database snapshot (ie. maintaining consistency) for backup purposes requires some sort of add-on for most SQL databases -- if you keep the db server running during the backup. Most of these add-ons cost real money.

But you can build them with open source SQL software, and all you're doing is paying the dev for his/her time, as opposed to Dev time + hefty licensing fees.
--
Everything in Moderation... including moderation.

IamGimli

join:2004-02-28
Canada
kudos:2
reply to Davesnothere
said by Davesnothere:

said by IamGimli:

said by Chuckcar_t:

....Note: Always remember to do incremental backups no matter who you are !

You really should get a clue about the topic at hand before you go mouthing off about what other people should do. Right now, you just sound like a complete and utter tool.

 
But this time he makes a good point (amongst the other stuff).

No he doesn't. Incremental backups are notoriously unreliable as a general rule, and rarely ever work for database backups.

Full system backups or, even better, continuous sector-based shadow copy of the volumes are much more robust, reliable backup technologies when used in conjunction with the appropriate database agents.

None of that would have helped anyway in this case unless it was combined with off-site storage of those backups as there's very little backups can help with when a power outage takes out the storage arrays where those backups are stored and prevents them from being usable.

Hence my comment he should get a clue before telling anyone how to run their backups.


El Quintron
Resident Mouth Breather
Premium
join:2008-04-28
Etobicoke, ON
kudos:4
Reviews:
·TekSavvy Cable
·TekSavvy DSL
said by IamGimli:

Hence my comment he should get a clue before telling anyone how to run their backups.

I don't think the term "incremental backup" was the right term to use, but the spirit of what s/he was saying is backups could have avoided this mess.
--
Everything in Moderation... including moderation.


HiVolt
Premium
join:2000-12-28
Toronto, ON
kudos:21
I think they had database backups, problem is they were on the same storage array which was the biggest blunder I think.
--
BUCK FELL ,,!,,('-'),,!,,


Davesnothere
No-BHELL-ity DOES have its Advantages
Premium
join:2009-06-15
START Today!
kudos:7

1 edit
reply to El Quintron
said by El Quintron:

said by IamGimli:

Hence my comment he should get a clue before telling anyone how to run their backups.

 
I don't think the term "incremental backup" was the right term to use, but the spirit of what s/he was saying is backups could have avoided this mess.

 
So now we're haggling over the size of the increment, are we ?

I do happen to understand the term 'incremental', and was posting on the basis that the rest of us did too, including Chuck.

I was also posting on the basis that there DID need to exist a primary complete backup to which to apply the increments, and that we all knew that too.

YOUR point (Gimli) though is important to me, as in principle I also prefer full backups, as they are easier to safely and successfully administrate.

But given the circumstances that it looked as though there was NO full 'off-primary-media' backup, ANYTHING further might have proven to be better than what we seem to have so far.


El Quintron
Resident Mouth Breather
Premium
join:2008-04-28
Etobicoke, ON
kudos:4
Reviews:
·TekSavvy Cable
·TekSavvy DSL
reply to HiVolt
said by HiVolt:

I think they had database backups, problem is they were on the same storage array which was the biggest blunder I think.

I figured as much... my PCs get backed up to my NAS, and my Data is stored on separate drives than the OS on each individual PC...

Which solves two issues; if the OS fails, then I can re-install the OS while keeping the data, and if the whole thing were to melt down then I'd have the Data on my NAS.

It's the best low-maintenance homebrew solution I can think of.
--
Everything in Moderation... including moderation.


Davesnothere
No-BHELL-ity DOES have its Advantages
Premium
join:2009-06-15
START Today!
kudos:7
reply to HiVolt
said by HiVolt:

I think they had database backups, problem is they were on the same storage array which was the biggest blunder I think.

 
Yes, and if true, also an expensive one, on many levels.

I'm curious however, just how long that a FULL backup of a site and database of DSLR's pre-apocalypsoh size would take to do ?

And as such, how frequently it would be practical to perform one with the site up and running 24-7 ?

--

START Forum »Start Communications
Or you can still use Canadian Broadband.
START Website »www.start.ca


Davesnothere
No-BHELL-ity DOES have its Advantages
Premium
join:2009-06-15
START Today!
kudos:7
reply to El Quintron
said by El Quintron:

...Which solves two issues; if the OS fails, then I can re-install the OS while keeping the data, and if the whole thing were to melt down then I'd have the Data on my NAS.

It's the best low-maintenance homebrew solution I can think of.

 
Dat's NAS ! (pun intended)


El Quintron
Resident Mouth Breather
Premium
join:2008-04-28
Etobicoke, ON
kudos:4
Reviews:
·TekSavvy Cable
·TekSavvy DSL
said by Davesnothere:

Dat's NAS ! (pun intended)

LOL I can see your punny-ness wasn't lost with DSLr's data!
--
Everything in Moderation... including moderation.

MaynardKrebs
Heave Steve, for the good of the country
Premium
join:2009-06-17
kudos:4
reply to Davesnothere
I don't recall whether the Dell array was dual-ported or not.
Often people use dual-ported disk to be able to continue processing if a cpu box goes south. But that does not help in the situation DSLr had.

Real systems production systems use a wide variety of techniques to mitigate against storage failure - SANS, hierarchical storage controllers, and wide area clusters are a few of the implementation measures that can be taken. Larger cpu's and storage systems also tend to to have power supplies with monster capacitors which are better able to withstand power hits.

With relational databases you can do a synchronous 2PC using XA-compliant db's and TP monitors over a wide-area cluster spanning thousands of miles to ensure that each 'transaction' (write) is committed at a remote site. I have done this 20 years ago with OpenVMS systems running Rdb (only about 300 miles at that time).

You can also use db replication (in whole or in part) to synch remote databases. About 20 years ago I did this using Sybase Replication Server to synch databases in Toronto, NY, Tokyo, and London. This was a near-time endeavour, with hourly updates deemed to be sufficient due to bandwidth costs back then. Today (well actually for the past 10 years or so) the same sort of thing would be done as asynchronous transaction-by-transaction updates over TCP/IP sent using something like message-queuing middleware.


mlerner
Premium
join:2000-11-25
Nepean, ON
kudos:5
reply to HiVolt
said by HiVolt:

Yikes!

Good thing that they will restore the missing content.

Justin will have to have start a donation campaign, hehe.

Told you it was expensive!

I don't know still how much Justin is making on this site but I'm pretty sure he's close to losing money on this thing now.


Spike
Premium
join:2008-05-16
Toronto, ON
reply to Davesnothere
Just having a separate mysql slave server with its own storage array would had saved the day if he didn't want to do backups. Its simple stuff to do, you don't need a big expensive array just for a "dumb" backup slave replication server....

Its also MUCH easier to backup this slave server with no interruption or downtime.

Oh well, $28,000 later, I'm sure the lesson was learned.


damir52
SFRJ
Premium
join:2003-05-24
If he can afford 28k (restore price), i am sure he makes (yearly at least 2x times that) with this website.
--
I was born intelligent, but education ruined me ...


El Quintron
Resident Mouth Breather
Premium
join:2008-04-28
Etobicoke, ON
kudos:4
Reviews:
·TekSavvy Cable
·TekSavvy DSL
said by damir52:

If he can afford 28k (restore price), i am sure he makes (yearly at least 2x times that) with this website.

Or DSLr is something he loves or needs, making the 28k restore price a necessary expense.
--
Everything in Moderation... including moderation.


damir52
SFRJ
Premium
join:2003-05-24
I have websites too, i make money of them, they are not close the size of this, to be honest, in case like this, if happened (which would never would, because of how everything is configured = cheaper) i wouldn't be able to pay that much money for restorations, i mean i could take a credit and pay it, but wouldn't in that case.

There are priorities.

So, how i see it, there is lots of money here, which is good of course, this is is excellent website, i enjoy it from day one, so, to be able to pay money for this, i am still thinking he is making a lot, which saying again is VERY good, and i hope this is true.


--
I was born intelligent, but education ruined me ...

Bell_Abused

join:2006-10-07
reply to El Quintron
said by El Quintron:

said by damir52:

If he can afford 28k (restore price), i am sure he makes (yearly at least 2x times that) with this website.

Or DSLr is something he loves or needs, making the 28k restore price a necessary expense.

Don't you worry. More is made than you think. They have a budget of 50K+ just for people other than Karl to write news for the front page.

You are severely under-guessing.


El Quintron
Resident Mouth Breather
Premium
join:2008-04-28
Etobicoke, ON
kudos:4
Reviews:
·TekSavvy Cable
·TekSavvy DSL
said by Bell_Abused:

Don't you worry. More is made than you think. They have a budget of 50K+ just for people other than Karl to write news for the front page.

You are severely under-guessing.

The actual amount of cash doesn't matter, if DSLr is the only source of income, or "the plan" for his future then the 28k is a non-negotiable expense.
--
Everything in Moderation... including moderation.


Davesnothere
No-BHELL-ity DOES have its Advantages
Premium
join:2009-06-15
START Today!
kudos:7
said by El Quintron:

said by Bell_Abused:

Don't you worry. More is made than you think. They have a budget of 50K+ just for people other than Karl to write news for the front page.

You are severely under-guessing.

The actual amount of cash doesn't matter, if DSLr is the only source of income, or "the plan" for his future then the 28k is a non-negotiable expense.

 
BOTH of your points could be correct.

It might be more that he cannot afford NOT to put the site back online, moreso than the exact cost to do so.
--

START Forum »Start Communications
Or you can still use Canadian Broadband.
START Website »www.start.ca


El Quintron
Resident Mouth Breather
Premium
join:2008-04-28
Etobicoke, ON
kudos:4
Reviews:
·TekSavvy Cable
·TekSavvy DSL
said by Davesnothere:

It might be more that he cannot afford NOT to put the site back online, moreso than the exact cost to do so.

Those would be my exact thoughts as well.
--
Everything in Moderation... including moderation.


Davesnothere
No-BHELL-ity DOES have its Advantages
Premium
join:2009-06-15
START Today!
kudos:7

4 edits
said by El Quintron:

said by Davesnothere:

It might be more that he cannot afford NOT to put the site back online, moreso than the exact cost to do so.

Those would be my exact thoughts as well.

 
There HAS been STRONG circumstantial evidence to support such a hypothesis.

The OTHER arguments to support 'staying the course', at least insofar as the short to medium term would be :

(1) short term - COMPLETING the data recovery, as I'd expect that they are already on the hook for a portion of the recovery costs, REGARDLESS OF RESULT or how far they may choose to go,

(2) medium term - to recoup at least the costs of the complete data recovery.

The long term analysis would of course be the loss of revenue and reputation if they did not make their best effort to recover the DSLR site completely (as per our near above posts) .

There would also be the short term self-satisfaction of 'Slaying the Fierce Monster', and the all-term knowledge gained to assist in better avoidance of a similar future meltdown (PRICELESS).

And THAT's the way it is.... (or at least the way I see it)


Davesnothere
No-BHELL-ity DOES have its Advantages
Premium
join:2009-06-15
START Today!
kudos:7
reply to Davesnothere
 
Found during the past few moments, just below the headline ticker :
quote:
Announcement :

Data restoration is underway. User accounts, ISP reviews and news have been restored.

There will be some hours of down time scheduled for final restoration

 
I have checked and found some new encouragements, such as my total post count (though not any Kudos so far) has been restored.

Actual posts are not back yet, but I have a feeling that the merger of threads is what will require the downtime of which the announcement spoke.

I was not forced to login again as I had already set my credentials to what they were just before the meltdown.

My Left-Column 'Location' and my 'Post Signature' have each reverted to their last most recent iteration, but I have already backed up the newer ones, anticipating this.

So be sure to check under yer pilloze in the morning, in case there are any further improvements.

--

We have only 2 things about which to worry :
(1) That things may never get back to normal
(2) That they already HAVE !
-
START Forum »Start Communications
Or you can still use Canadian Broadband.



Davesnothere
No-BHELL-ity DOES have its Advantages
Premium
join:2009-06-15
START Today!
kudos:7

2 edits
reply to Davesnothere
 
Older IMs are not restored so far, though profiles are partially restored as I posted above.

Changing my left column 'Location' field still responds with an error page and no immediate change, but during this period of limited operation, any profile changes (other than password) which I DID achieve would do exactly the same as this, and then would 'magically' appear a few hours later, as if some database had likely just caught up.

EDIT : And sure enough, my left margin stuff has caught up.

dan mackay

join:2008-10-11
Barrie, ON
reply to Davesnothere
well, atleast my account works now


TwiztedZero
Nine Zero Burp Nine Six
Premium
join:2011-03-31
Toronto, ON
kudos:5
Mine too.