
how-to block ads
|
|
Share Topic  |
 |
|
 rblizz join:2001-12-16 North Richland Hills, TX Reviews:
·Callcentric
·Charter
| reply to borntochill
Re: CC Disaster said by borntochill:In this thread, and in the other epic CallCentric thread (post Sandy), the serious, glaring, and preventable shortcomings in CallCentric's "disaster recovery" plan have already been raised by numerous CallCentric customers and discussed exhaustively. If you wish to see it differently, that's your prerogative, but many CallCentric customers are not content with the status quo. Here's the way "I wish to see it." NY City, and especially lower Manhattan, was hit by a massive storm. Fourteen foot swells, when the all-time record before that was ten foot swells. Con Ed was swamped, they shut power off for safety and to lessen damage to their infrastructure. The electrical grid CallCentric is on also powers the New York Stock Exchange. Guess what's going to get priority in repair efforts in lower Manhattan? CallCentric was down for a total of 53 hours (a little over two days). The NYSE was down for two days (first time weather has taken them down for more than one day in over 100 years). Disasters happen. Expecting everyone to be able to stay up in a major disaster "no matter what" is unrealistic. Lower Manhattan had a major disaster and CallCentric recovered (quickly, considering the scope of the disaster). That is what "disaster recovery" is all about. What you're demanding is not disaster recovery but "disaster immunity." It won't happen. Not every situation can be managed, but armchair quarterbacks can sure pontificate, can't they? | |  madjeff join:2005-04-30 united state kudos:1 | said by rblizz:said by borntochill:In this thread, and in the other epic CallCentric thread (post Sandy), the serious, glaring, and preventable shortcomings in CallCentric's "disaster recovery" plan have already been raised by numerous CallCentric customers and discussed exhaustively. If you wish to see it differently, that's your prerogative, but many CallCentric customers are not content with the status quo. Disasters happen. Expecting everyone to be able to stay up in a major disaster "no matter what" is unrealistic. Lower Manhattan had a major disaster and CallCentric recovered (quickly, considering the scope of the disaster). That is what "disaster recovery" is all about. What you're demanding is not disaster recovery but "disaster immunity." It won't happen. Not every situation can be managed, but armchair quarterbacks can sure pontificate, can't they? There seems to be a disconnect here with people on exactly what constitutes "Disaster Recovery", as well as the distinction between DR and Business Continuity Planning (BCP). A lot of people here seem to be confusing the two, and there are subtle differences. Disaster recovery is the process by which you resume business after a disruptive event. Business Continuity Planning is a much broader plan that focuses on not only identifying your single points of failure and addressing them, but everything from how you are getting your employees to work, keeping them safe, and communicating with you clients regularly, as well as large number of other factors.
As soon as your company stops providing services, your clients will stop paying and leave you for competitors. But you can bet your vendors will still remain loyal and be more than happy to continue to charge you for their services regardless that you are down. In order to stop bleeding money in these downtime situations, you need a plan that will allow you to continue generating revenue and providing services on a temporary basis until the company has regained its bearings, even if it's only a small subset of total customers.
The trick to good Business Continuity planning is to only focus on the most critical business functions in order to reduce costs, and to enable these systems using the most cost-effective means possible without significantly affecting quality of service.
It appears in this particular case that CC's DR planning consisted of waiting until ConEd threw the switched back on. And in the isolated case of the servers in CC's office in NY, it was about all they could do in regards to getting services back online given their current architecture. I don't think anyone is arguing otherwise. I won't get into the fact that you had no backup power generation facilities as I think everyone understands that that was an obvious hole in your existing plan.
The point to this longwinded intro to BCP/DR is this. BCP planning is about identifying your weak spots for just these sorts of disasters and coming up with a plan to make sure you can continue to make money and stay in business, which is what all this boils down to. Sure, Sandy was a extremely rare event. But this has shown me that you wouldn't be able to recovery from a fire or a sprinkler system going rogue any better than a 100 year hurricane. In fact, those much more regular events would have been much much worse, as you would have physically lost your infrastructure. Having ConEd throw a switch so you can power everything back up is about the easiest DR recovery I can think of, and with all you guys are going through back there that is something to be thankful for. 
Gotta run, this was supposed to be a quick message. I have some other points but will try to get on later and wrap up my thoughts, what little they are worth. | | |
|
|