dslreports logo
 
    All Forums Hot Topics Gallery
spc
uniqs
61
« New Polycom with Bluetooth; VVX 600CallWithUs... strage issue. »
page: 1 · 2 · 3 · 4 · 5 · 6 · 7 · next
This is a sub-selection from CC Disaster
royrogers
join:2012-10-17

royrogers to Terabit

Member

to Terabit

Re: CC Disaster

How is it that Callcentric's power has been restored, but the news reports keep saying power for Manhattan will be restored Saturday?
rblizz
join:2001-12-16
North Richland Hills, TX

rblizz

Member

said by royrogers:

How is it that Callcentric's power has been restored, but the news reports keep saying power for Manhattan will be restored Saturday?

Some of lower Manhattan has been restored, some hasn't. I think the report means that power will be completely restored by Saturday.

Davesnothere
Change is NOT Necessarily Progress
Premium Member
join:2009-06-15
Canada

Davesnothere to royrogers

Premium Member

to royrogers
said by royrogers:

How is it that Callcentric's power has been restored, but the news reports keep saying power for Manhattan will be restored Saturday?

 
Caramilk secret.

Bluster
@conformal.com

Bluster

Anon

Re: CC Disaster - they will be all right

With CC back I think their problems with losing subscribers will be minor; most people won't care that their DID is with a non-redundant service.

They will figure the odds are against another big outage like Sandy, and will simply enjoy CC's good sound and reliability.

Iscream has made one statement here, that setting up geographically separate servers is not necessarily not in the plans, but that they work within a budget, etc. That is probably enough; any more would just get the haters, aka critics, down on him.

Things are quiet, calls are going through. CC will soon be back at the top of the charts. One example of how this will happen is a recent review cautioning that a backup or two would not hurt, but giving them 95 percent. That is what I call customer loyalty!

The only thing missing is PX Eliezer. I hope he got through Sandy with no problems and will soon re-join us here to celebrate CC's resurrection.
Mango
Use DMZ and you get a kick in the dick.
Premium Member
join:2008-12-25
www.toao.net

Mango

Premium Member

He's fine but still without electricity.
kaila
join:2000-10-11
Lincolnshire, IL

1 recommendation

kaila

Member

said by Mango:

He's fine but still without electricity.

You mean he doesn't have distributed redundant backup power sources? j/k, glad to hear he's fine.

josephf
join:2009-04-26

josephf to Mango

Member

to Mango
You know him?

Davesnothere
Change is NOT Necessarily Progress
Premium Member
join:2009-06-15
Canada

Davesnothere to kaila

Premium Member

to kaila
said by kaila:

said by Mango:

He's fine but still without electricity.

You mean he doesn't have distributed redundant backup power sources? j/k, glad to hear he's fine.

 
Yes, Good to hear !

= = = = = = =

On a related topic, then I expect that he's now forced to watch TV by Candlelight.
travelguy
join:1999-09-03
Bismarck, ND
Asus RT-AC68
Ubiquiti NSM5

travelguy to Bluster

Member

to Bluster
said by Bluster :

Things are quiet, calls are going through.

Things may be quiet, but they are not all back to normal. CC DIDs hosted in NYC are still out with no ETA.

I was fortunate enough to be using a Google Voice number as my public number, so it was easy enough for me to redirect calls to that number, but it is a bit of a pain.

ropeguru
Premium Member
join:2001-01-25
Mechanicsville, VA

ropeguru

Premium Member

said by travelguy:

said by Bluster :

Things are quiet, calls are going through.

Things may be quiet, but they are not all back to normal. CC DIDs hosted in NYC are still out with no ETA.

I was fortunate enough to be using a Google Voice number as my public number, so it was easy enough for me to redirect calls to that number, but it is a bit of a pain.

To clarify, this is NOT CC's fault. The equipment this is provided through in NYC is still down. So this is beyond CC's control.
dutchtender
join:2010-11-21
Sunnyvale, CA

dutchtender

Member

said by ropeguru:

said by travelguy:

said by Bluster :

Things are quiet, calls are going through.

Things may be quiet, but they are not all back to normal. CC DIDs hosted in NYC are still out with no ETA.

I was fortunate enough to be using a Google Voice number as my public number, so it was easy enough for me to redirect calls to that number, but it is a bit of a pain.

To clarify, this is NOT CC's fault. The equipment this is provided through in NYC is still down. So this is beyond CC's control.

my DIDs are 914 upstate. any idea why they aren't working? they are not in NYC.

josephf
join:2009-04-26

josephf

Member

914 is Westchester County, bordering NYC, which is also a declared Federal disaster area.

ropeguru
Premium Member
join:2001-01-25
Mechanicsville, VA

ropeguru to dutchtender

Premium Member

to dutchtender
said by dutchtender:

my DIDs are 914 upstate. any idea why they aren't working? they are not in NYC.

No I cannot. The DID's that travelguy See Profile was referring to and I clarified were New York City DID's.

Is yours a NY free DID? I remember something being mentioned about all the NY free DID's being down because of the way they get them from their carrier who is down and has an unknown ETA of being back up.

SpaethCo
Digital Plumber
MVM
join:2001-04-21
Minneapolis, MN

SpaethCo to dutchtender

MVM

to dutchtender
said by dutchtender:

my DIDs are 914 upstate. any idea why they aren't working? they are not in NYC.

The majority of the exchanges in area code 914 are LATA 132 (NYC Metro). Just because it's an "upstate number" doesn't mean the interconnect gear is physically located upstate; it can be located anywhere in the LATA network boundary which could include NYC proper.

josephf
join:2009-04-26

josephf

Member

I believe the entire area code 914 is in LATA 132.
borntochill
join:2003-02-09
united state

1 recommendation

borntochill to Bluster

Member

to Bluster

Re: CC Disaster

said by Bluster :

With CC back I think their problems with losing subscribers will be minor; most people won't care that their DID is with a non-redundant service.

They will figure the odds are against another big outage like Sandy, and will simply enjoy CC's good sound and reliability.

Iscream has made one statement here, that setting up geographically separate servers is not necessarily not in the plans, but that they work within a budget, etc.

After this outage, if CallCentric does not announce clear and specific intentions to put in place a proper disaster recovery plan, I cannot keep them as a primary provider for any of my clients. I will move DIDs to a service provider which DOES have disaster recovery.

Whether another "perfect storm" could hit NYC again in the near future is subject to debate and is probably unknowable. However, what if a fire destroys 11 Broadway, or there is a major plumbing flood? A prolonged power outage is not the only foreseeable event that could put CallCentric and incoming calls to its customers' phones out of commission for potentially much longer than 2 days and without any alternatives. My head will roll if that happens yet I could have prevented it by choosing a different primary service provider with proper disaster recovery procedures.

So again, I'd like to hear clear and specific plans from Iscream or another CC rep. For me, disaster recovery isn't an option; it's a requirement for a primary service provider. I will certainly continue to use CallCentric as a secondary provider because, lack of disaster recovery planning (and very poor communication when disasters happen), they are otherwise an excellent provider.
rblizz
join:2001-12-16
North Richland Hills, TX

rblizz to travelguy

Member

to travelguy

Re: CC Disaster - they will be all right

said by travelguy:

Things may be quiet, but they are not all back to normal. CC DIDs hosted in NYC are still out with no ETA.

Absolutely nothing CallCentric can do about that. They can't hand-off or have calls handed off to them when the local CO is powered down. It would be like me blaming CallCentric if a tornado took out my local CO in Dallas/Fort Worth. There is still a lot of repair left to do in the NY City area.
rblizz

rblizz to borntochill

Member

to borntochill

Re: CC Disaster

said by borntochill:

After this outage, if CallCentric does not announce clear and specific intentions to put in place a proper disaster recovery plan, ...

CallCentric already has a "proper" disaster recovery plan. They've already recovered. A large chunk of NY City still hasn't.
borntochill
join:2003-02-09
united state

2 recommendations

borntochill

Member

said by rblizz:

said by borntochill:

After this outage, if CallCentric does not announce clear and specific intentions to put in place a proper disaster recovery plan, ...

CallCentric already has a "proper" disaster recovery plan. They've already recovered. A large chunk of NY City still hasn't.

In this thread, and in the other epic CallCentric thread (post Sandy), the serious, glaring, and preventable shortcomings in CallCentric's "disaster recovery" plan have already been raised by numerous CallCentric customers and discussed exhaustively. If you wish to see it differently, that's your prerogative, but many CallCentric customers are not content with the status quo.
Iscream
Premium Member
join:2009-02-17
New York, NY

2 recommendations

Iscream

Premium Member

Gosh, you got me. I still have tons of things to do, but you have really gotten me to the last point. Just imagine words I could have expressed here (any one goes!).

I've just posted it in another thread where you had a chance to express your "expert's" opinion, here it's again:

In all due respect - you have zero technical understanding of matters you're trying to speculate about and/or - you're trying to be a judge for! What a shame and abuse of freedom of speech.

There is a saying - "don't judge and you won't be judged".

When time permits - I'll try to explain more what the terms "redundancy" and disaster recovery actually mean when applied to the Real Telephony.
rblizz
join:2001-12-16
North Richland Hills, TX

1 recommendation

rblizz to borntochill

Member

to borntochill
said by borntochill:

In this thread, and in the other epic CallCentric thread (post Sandy), the serious, glaring, and preventable shortcomings in CallCentric's "disaster recovery" plan have already been raised by numerous CallCentric customers and discussed exhaustively. If you wish to see it differently, that's your prerogative, but many CallCentric customers are not content with the status quo.

Here's the way "I wish to see it." NY City, and especially lower Manhattan, was hit by a massive storm. Fourteen foot swells, when the all-time record before that was ten foot swells. Con Ed was swamped, they shut power off for safety and to lessen damage to their infrastructure. The electrical grid CallCentric is on also powers the New York Stock Exchange. Guess what's going to get priority in repair efforts in lower Manhattan? CallCentric was down for a total of 53 hours (a little over two days). The NYSE was down for two days (first time weather has taken them down for more than one day in over 100 years). Disasters happen. Expecting everyone to be able to stay up in a major disaster "no matter what" is unrealistic. Lower Manhattan had a major disaster and CallCentric recovered (quickly, considering the scope of the disaster). That is what "disaster recovery" is all about. What you're demanding is not disaster recovery but "disaster immunity." It won't happen. Not every situation can be managed, but armchair quarterbacks can sure pontificate, can't they?
borntochill
join:2003-02-09
united state

1 recommendation

borntochill to Iscream

Member

to Iscream
said by Iscream:

Gosh, you got me. I still have tons of things to do, but you have really gotten me to the last point. Just imagine words I could have expressed here (any one goes!).

I've just posted it in another thread where you had a chance to express your "expert's" opinion, here it's again:

In all due respect - you have zero technical understanding of matters you're trying to speculate about and/or - you're trying to be a judge for! What a shame and abuse of freedom of speech.

There is a saying - "don't judge and you won't be judged".

When time permits - I'll try to explain more what the terms "redundancy" and disaster recovery actually mean when applied to the Real Telephony.

Iscream, I know you are busy now, but I look forward to your explanation about your notion of what "disaster recovery" means, minus the insults and dripping condescension.

My "judgments" about how CallCentric handled this disaster were clearly spelled out upthread in this post:

»Re: CC Disaster

That post received 5 "Approvals" from other users, including this forum's moderator. This is the most approvals I've received for a single post in the 10 years that I've been a member of DSLReports, so clearly I struck a chord with my remarks. If you respectfully want to tell me how and why I have it wrong, I look forward to that when you are ready, but I'm clearly not the only one concerned.
rblizz
join:2001-12-16
North Richland Hills, TX

rblizz

Member

said by borntochill:

Iscream, I know you are busy now, but I look forward to your explanation about your notion of what "disaster recovery" means, minus the insults and dripping condescension.

Look up "recovery" in the dictionary. Ah, never mind, I'll do it for you ...
quote:
1. The act of recovering, regaining, or retaking possession.
[1913 Webster]

or ...

n 1: return to an original state; "the recovery of the forest
after the fire was surprisingly rapid"

»www.dict.org/bin/Dict

To "recover" means something had to have been "altered" in the first place. Right? So, for example, when a "storm of the century" takes out a power grid that hardly ever goes out, causing wide-spread damage that would be the "alteration." Getting power back up, getting the phones back up -- that would be the "recovery." Correct?

This is what disaster recovery means.
nitzan
Premium Member
join:2008-02-27

1 recommendation

nitzan

Premium Member

said by rblizz:

Look up "recovery" in the dictionary.

Being a smart-ass aside, Disaster Recovery refers to standard practices of ensuring business continuity during natural disasters. A very common practice is to replicate your production environment to a remote standby backup facility in a different geographic location. This is a standard practice for any business that requires high availability.

Disaster recovery doesn't refer to "waiting until the power comes back on to recover" - that's essentially zero disaster recovery planning. A power outage is one thing, but can you imagine what happens if the building gets destroyed in a fire?
cell14
join:2012-01-04
Miami Beach, FL

cell14 to borntochill

Member

to borntochill
Outgoing calls working, NYC DID dead, calling card access number - 305 area code- not working.
Since CC is not my primary provider ( and after what happened it will not be) I am likely to keep them. One thing above all is absolutely inexcusable- the lack of any decent communication with customers. And this problem did not started with the storm, but already with DDoS.
rblizz
join:2001-12-16
North Richland Hills, TX

4 edits

1 recommendation

rblizz to nitzan

Member

to nitzan
said by nitzan:

Being a smart-ass aside, Disaster Recovery refers to standard practices of ensuring business continuity during natural disasters.

There is no guarantee that anyone will remain up, no matter what precautions are taken. Would you say the NYSE is well prepared for disasters? They, after all, had a back-up generator and all the resources in the world. And yet, somehow, they went down for two days also, didn't they?

Pissing and moaning after the fact doesn't accomplish a damn thing. Pontificating from the sidelines, especially when it's a competitor doing it, is (to be blunt) chickenshit and cowardly. Before Sandy, I had hoped that CallCentric would share what they've learned about thwarting DDoS attacks with other VoIP providers. Now that I see the true nature of some of these competitors I'm thinking "what the hell, let them sink or swim on their own."

Let's take an example of a "disaster" from my own life and look at the options as I saw them. I used to contract -- repaired, maintained and installed Nortel switches all over the country. One time I had been gone from my family for about a month and a half. I had been to Pennsylvania, North and South Carolina and then (when I was scheduled to go home to Idaho) I got sent to San Jose instead. Finally, on Christmas Eve, I got to go home. Except, when I got to Reno, I found out that the Boise, ID airport was fogged in and, since it was the holiday season, the soonest I could get another flight was in four days.

So, whom do I blame? Let's face it, that's the first thing we've got to do, fix blame. Because it can't possibly be bad weather, or lousy circumstances, or bad luck or something totally beyond the airline's control. The airline had to be blamed for it. I could have sat there all night pissing and moaning and demanding they do something now -- while whining about their lack of contingency plans and huffing and puffing about how I would never ever fly with this airline again -- or I could face reality and get over it.

I chose to rent a car (one of the last ones that was available), even though it meant I would have to drive all night, 450 miles, through a winter storm. I wanted to get home and I found a way to get it done. I got to see my kids on Christmas morning.

We live in a society now where affixing the blame has become the most important thing in some people's eyes. It can't possibly be Superstorm Sandy we blame, somebody should have anticipated 14 foot swells, instead of the expected 11 or 12 foot swells. Someone should have known that Con Ed was going to forced to shut down power to the Wall Street grid, even though such action almost never happens. Someone should have told everyone that CallCentric only had one server farm -- even though the fact was never hidden. "Someone's head must roll!"

No one could just say -- "It was a hell of storm, good job CallCentric for gracefully turning down the servers and successfully bringing them back up as soon as you got power back. Fantastic job of mitigating the effects of the storm damage. Thanks for giving us back top rate service in just slightly over two days."

No, they can't say that, because everyone lives in this weird imaginary world where nothing ever goes wrong, where there's never an act of God and where accidents just never happen. And, when things do go wrong, it's always someone's fault for not anticipating every possible contingency and planning for each and every one of them.

Shit happens. Get over it.

Davesnothere
Change is NOT Necessarily Progress
Premium Member
join:2009-06-15
Canada

2 edits

1 recommendation

Davesnothere to rblizz

Premium Member

to rblizz
said by rblizz:

said by borntochill:

After this outage, if CallCentric does not announce clear and specific intentions to put in place a proper disaster recovery plan, ...

CallCentric already has a "proper" disaster recovery plan....

 
Yeah.

CallCentric's 'proper' plan is : "Shut down servers and wait for Con Ed"

Insufficient, as borntochill See Profile and many others in this forum have stated.

Fair enough, it DOES protect their servers and our data.

However, I'm a private citizen and I have over 3000VA of UPS capacity at my place, containing 80Ah capacity of batteries.

( OK, I did used to operate a computer store, but still, it's a good feeling. )
Davesnothere

Davesnothere to borntochill

Premium Member

to borntochill
said by borntochill:

....So again, I'd like to hear clear and specific plans from Iscream or another CC rep. For me, disaster recovery isn't an option; it's a requirement for a primary service provider. I will certainly continue to use CallCentric as a secondary provider because, lack of disaster recovery planning (and very poor communication when disasters happen), they are otherwise an excellent provider.

 
+1 to all of that !
Davesnothere

Davesnothere to borntochill

Premium Member

to borntochill
said by borntochill:

....My "judgments" about how CallCentric handled this disaster were clearly spelled out upthread in this post:

»Re: CC Disaster

That post received 5 "Approvals" from other users, including this forum's moderator. This is the most approvals I've received for a single post in the 10 years that I've been a member of DSLReports, so clearly I struck a chord with my remarks. If you respectfully want to tell me how and why I have it wrong, I look forward to that when you are ready, but I'm clearly not the only one concerned.

 
Wrong, it got SIX approvals (including mine), and as you know, we each cannot see the other ones.

I did so blind and on-the-fly, i.e. before reading about the other 5.
Davesnothere

Davesnothere to rblizz

Premium Member

to rblizz
said by rblizz:

....No one could just say -- "It was a hell of storm, good job CallCentric for gracefully turning down the servers and successfully bringing them back up as soon as you got power back. Fantastic job of mitigating the effects of the storm damage. Thanks for giving us back top rate service in just slightly over two days." ....

 
Doing that much is de rigeur in this industry, in fact doing MORE than that much seems to be the standard, as I'm beginning to read in my thread »SERVER & GENERATOR LOCATIONS about server locations & power backup regimens.

It is good that they did that, but they were in a position to do no more (except maybe tell us more while waiting for Con Ed), and their 'plan', as it were, had constrained them to that.

= = = = = =

BTW, my pontifications are being brought to y'all LIVE from an old oak office chair which used to be my father's, and it has no arms nor moving parts.
« New Polycom with Bluetooth; VVX 600CallWithUs... strage issue. »
page: 1 · 2 · 3 · 4 · 5 · 6 · 7 · next
This is a sub-selection from CC Disaster