dslreports logo
site
 
    All Forums Hot Topics Gallery
spc

spacer




how-to block ads


Search Topic:
uniqs
55
share rss forum feed

ConstantineM

join:2011-09-02
San Jose, CA

1 recommendation

reply to Iscream

Back to worst-case-scenario to start with!

Yeap, my 519 number works, but my two free 845 numbers don't -- busy signals from T-Mo in NorCal. E.g. we're finally back to the situation that should have been the worst-case-scenario to start with.

I think it's pretty clear that there are hardly any plans for any kind of redundancy on CallCentric's part. Looking for a replacement DDI carrier for my 519; and, also, I will never route any calls from IPKall or UKDDI through CallCentric ever again (my final SIP registrations at OnSIP (which are in NYC, too, BTW) were never down).

Iscream
Premium
join:2009-02-17
New York, NY
kudos:6
Reviews:
·Verizon FiOS
Callcentric is 100% redundant locally, in all possible means - lines, equipment, databases, etc., but is NOT multi-site'd - it doesn't have geographical redundancy.

It's NOT clear [to me] that "there are hardly any plans.." - this is incorrect to say the least, it's rather that economics play a major role here - CC slowly builds its network while providing the best service quality and reliability possible FOR THE CURRENT BUDGET. When a city that never sleeps goes OFF because of a natural disaster - CC keeps its service as long as it can, but then it also goes off until it can restore its utility power.

Thanks for being Callcentric customer.

dutchtender

join:2010-11-21
New York, NY

1 edit
reply to ConstantineM
said by ConstantineM:

Yeap, my 519 number works, but my two free 845 numbers don't -- busy signals from T-Mo in NorCal. E.g. we're finally back to the situation that should have been the worst-case-scenario to start with.

I think it's pretty clear that there are hardly any plans for any kind of redundancy on CallCentric's part. Looking for a replacement DDI carrier for my 519; and, also, I will never route any calls from IPKall or UKDDI through CallCentric ever again (my final SIP registrations at OnSIP (which are in NYC, too, BTW) were never down).

my free 914 numbers don't work either. but not because of callcentric.


news4la

join:2009-01-05
Universal City, CA
reply to Iscream
How can I help with geographic redundancy. Yes that within reason does mean help with funds and equipment.If you need help with extra sites outside of NYC. There is a possibility I could help.
--
On AIR,On LINE and On the GO!

verix

join:2004-12-30
Oakland, CA

2 recommendations

reply to Iscream
said by Iscream:

Callcentric is 100% redundant locally, in all possible means - lines, equipment, databases, etc., but is NOT multi-site'd - it doesn't have geographical redundancy.

I look forward to the upcoming blog post/e-mail notice/press release/whatever before I decide on further action. Sorry this storm was unprecedented and that there may be budgetary reasons for your infrastructure. But, we need our lines.


Davesnothere
No-BHELL-ity DOES have its Advantages
Premium
join:2009-06-15
START Today!
kudos:7
said by verix:

said by Iscream:

Callcentric is 100% redundant locally, in all possible means - lines, equipment, databases, etc., but is NOT multi-site'd - it doesn't have geographical redundancy.

I look forward to the upcoming blog post/e-mail notice/press release/whatever before I decide on further action.

Sorry this storm was unprecedented and that there may be budgetary reasons for your infrastructure. But, we need our lines.

 
Agreed on that.

And WHERE is Tower of Power when we need them ?!

rblizz

join:2001-12-16
North Richland Hills, TX
reply to verix
said by verix:

I look forward to the upcoming blog post/e-mail notice/press release/whatever before I decide on further action. Sorry this storm was unprecedented and that there may be budgetary reasons for your infrastructure. But, we need our lines.

As if New Yorkers (and others along the Eastern Coast) don't need their power. Is this the first time you've ever heard a natural disaster? Did everything work without a glitch in Oakland after the 1989 earthquake?

rblizz

join:2001-12-16
North Richland Hills, TX
reply to verix
said by verix:

said by Iscream:

Callcentric is 100% redundant locally, in all possible means - lines, equipment, databases, etc., but is NOT multi-site'd - it doesn't have geographical redundancy.

I look forward to the upcoming blog post/e-mail notice/press release/whatever before I decide on further action. Sorry this storm was unprecedented and that there may be budgetary reasons for your infrastructure. But, we need our lines.

It was no secret that CallCentric had one server location. It's been discussed here several times.

verix

join:2004-12-30
Oakland, CA

2 recommendations

reply to rblizz
said by rblizz:

As if New Yorkers (and others along the Eastern Coast) don't need their power. Is this the first time you've ever heard a natural disaster? Did everything work without a glitch in Oakland after the 1989 earthquake?

Condolences to everyone affected by Sandy, but why should people nowhere near it have to suffer? What about the businesses in CA that have lines with CC?

verix

join:2004-12-30
Oakland, CA

2 recommendations

reply to rblizz
said by rblizz:

It was no secret that CallCentric had one server location. It's been discussed here several times.

I check this forum quite often and had no idea that CC had one server location until this event. Don't make assumptions of what people know.

rblizz

join:2001-12-16
North Richland Hills, TX
reply to verix
said by verix:

Condolences to everyone affected by Sandy, but why should people nowhere near it have to suffer? What about the businesses in CA that have lines with CC?

I don't know about Oakland, but in most of the world "reality happens." That's why they call the call earthquakes and storms like Sandy "natural disasters" ... "disasters" being the key word here. I don't know why so many here seem to think we live in a La La World where every inconvenience can always be averted -- and, if it can't, someone's head must roll.

Personally I'm happy it only took 3 days for CallCentric to get back up. Even the New York Stock Exchange, the world center of capitalism with nearly unlimited resources, was down for nearly two days. And this is the first time the NYSE was down for two days or more because of weather for over a century. This should tell you something about how powerful this storm was. I.E., it could have been a hell of a lot worse for CallCentric customers.

rblizz

join:2001-12-16
North Richland Hills, TX
reply to verix
said by verix:

[I check this forum quite often and had no idea that CC had one server location until this event. Don't make assumptions of what people know.

I like to know who I'm dealing with. One of CallCentric's big selling points was that it was on the same electrical grid as the NYSE.

From their promotional material ...

quote:
"CallCentric retail is backed by the resources of CallCentric Wholesale----

"Callcentric Wholesale, Inc. (parent of Callcentric) owns and operates its own TDM/SS7 network interconnected with major (as well as small ones) carriers, networks and voice exchanges....

"Callcentric Wholesale is also an ARIN registered ISP, operating its own IP network blocks and exchanging BGP-4 routing information with most ISP backbones (also nationally and internationally) via its own managed fiber optical network thus guarantying the best IP packet delivery to virtually anywhere in the world....

"We're NOT co-located with any providers - we operate our own data-center (with all underlying infrastructure, under 24x7 video surveillance) with redundant air-conditioning and AC power-plants; our dual-redundant AC sources reside on the same AC power grid with NY stock exchange (which is just across Broadway from us). In fact, we provide co-location, IP and TDM connection as well as origination and termination services to many international operators....."
Can you name the last time the NY Stock Exchange's electrical grid went down? I can't.

verix

join:2004-12-30
Oakland, CA

2 recommendations

reply to rblizz
said by rblizz:

I don't know about Oakland, but in most of the world "reality happens." That's why they call the call earthquakes and storms like Sandy "natural disasters" ... "disasters" being the key word here. I don't know why so many here seem to think we live in a La La World where every inconvenience can always be averted -- and, if it can't, someone's head must roll.

You didn't answer my questions. It's like you're trying to conflate my sympathies for Sandy victims to some sort of acceptance of CC's lack of service, when every other provider did not experience significant downtime. Even those that have a presence in NYC. And thank you for your lecture on natural disasters. That's why I'm waiting to hear how CC responds before I make a decision. Did I say I was leaving?

Personally I'm happy it only took 3 days for CallCentric to get back up.

Glad you're happy. A lot of people aren't.

rblizz

join:2001-12-16
North Richland Hills, TX
said by verix:

Glad you're happy. A lot of people aren't.

A lot of people like to bitch and confuse that with with "doing something." No one is happy about the downtime -- but I realize it could have been much, much worse -- so I'm happy that it wasn't.

It's also plain to me that CallCentric is proud of the time and effort they put into their infrastructure. There's a reason you and I chose CallCentric in the first place. It was their reputation for quality. So now they've been attacked from the outside with an unrelentlessness DDoS barrage (which they beat back) and a once in a century storm (that also took down the NYSE) and you're demanding they "explain themselves." They've already explained themselves. What exactly would make you happy at this point? I'm guessing nothing.

verix

join:2004-12-30
Oakland, CA
said by rblizz:

They've already explained themselves. What exactly would make you happy at this point? I'm guessing nothing.

I'm happy they restored service. I'm not happy this occurred in the first place. Happy?


Besant

@uwaterloo.ca
reply to verix
said by verix:

said by rblizz:

It was no secret that CallCentric had one server location. It's been discussed here several times.

I check this forum quite often and had no idea that CC had one server location until this event.

I did not know that either. All I recall is seeing 'redundant' in CC's description of its servers and assumed the redundant servers were geographically remote from the main ones.

For me it is not important: I use CC as a backup and have no DID with them; but if I had a business I would give them a wide berth until the day that real redundancy is finally in their budget.
Expand your moderator at work

Iscream
Premium
join:2009-02-17
New York, NY
kudos:6
Reviews:
·Verizon FiOS

1 recommendation

reply to verix

Re: Back to worst-case-scenario to start with!

Before I'm going to sleep a couple hours:

- took only 2 days (53 hours) to restore the service. NASDAQ and NYSE's floors are still down for general public. Perhaps Thursday or Friday.

- speaking of MAJOR national and world-wide carriers, about those who were and still are down - here is not compete list:

Belgacom, Telia, KPN (including former iBasis), TATA (not just BIG - this is THE LARGEST CARRIER in the world, no one is LARGER - TATA has TWO POPs in NY State - both were down), Verizon - still down, no even sign of life, ATT, T-Mobile, Sprint - either fully or still partially down along the way from Staten Island, through Brooklyn and Lower Manhattan including our offices. Speaking of "smaller" ones - Broadvox, iNetworks (that's Bandwidth and Dash-911 and Republic Wireless). Want even smaller ones - how about VoxBone ? Huh?

For those who want more detailed info - details available on the Net and specialized subscription lists.

Regarding NY Free DIDs - I wrote in my previous messages that those are currently NOT available, but NOT due to Callcentric's issues - because all TDM interconnects in LATA 132 (Big Apple) are still dead including SS7 network - at least in Lower Manhattan. We'll bring those DIDs up (as a matter of fact - they'll come up just by themselves) a soon as the PSTN network in New York restores.
Expand your moderator at work

rblizz

join:2001-12-16
North Richland Hills, TX
reply to Iscream

Re: Back to worst-case-scenario to start with!

said by Iscream:

- took only 2 days (53 hours) to restore the service. NASDAQ and NYSE's floors are still down for general public. Perhaps Thursday or Friday.

Sorry, I'm the one who's been messing that one up. So, just slightly over two days for full restoration of service.

rblizz

join:2001-12-16
North Richland Hills, TX
reply to Anon
said by rblizz:

Yep. Time for me to leave this thread for now. I'm allergic to the unreasonable.

Just to give some folks an idea of the scale of what's going on in New York (and elsewhere) ...

quote:
As blackouts linger, Northeasterners try to adjust

Sandy blacked out some of the nation's most densely populated cities and suburbs, instantly taking away modern conveniences from Virginia to Massachusetts and as far west as the Great Lakes.

For power companies, the scale of the destruction was unmatched -- more widespread than any blizzard or ice storm and worse than the aftermath of Hurricane Katrina.

"It's unprecedented: fallen trees, debris, the roads, water, snow. It's a little bit of everything," said Brian Wolff, senior vice president of the Edison Electric Institute, a group that lobbies for utilities.

Initially, about 60 million people were without power in 8.2 million homes and businesses. By Wednesday night, that number had fallen to roughly 44 million people in 6 million households and businesses.

Even as power slowly returned to some pockets, a new headache emerged: Backup batteries and generators running cellphone towers were running out of juice. One out of every five towers was down, according to the Federal Communications Commission.

»news.yahoo.com/blackouts-linger-···751.html


fukitol
Solon for President
Premium
join:2001-06-11
PonziWorld

1 recommendation

reply to Besant
Yeah, I don't buy the "it happened to a whole bunch of others, too" BULLSHIT. I don't give a flying fuck if a whole bunch of other companies didn't build a properly redundant network, either. If I ever used an excuse like that for bad behavior when I was growing up, my parents would ask me, "Well, if your friends jumped off a bridge, would you do it too?" I certainly hope that no one needed the E911 service for which they paid extra to CC over the past month. Oh, and where are the refunds? Hmm?
--
Vote Lucifer in 2012. It's real change you can believe in!
The truth is that the State is a conspiracy designed not only to exploit, but above all to corrupt its citizens. - Tolstoy


Davesnothere
No-BHELL-ity DOES have its Advantages
Premium
join:2009-06-15
START Today!
kudos:7

4 edits

1 recommendation

reply to rblizz
said by rblizz:

said by verix:

Condolences to everyone affected by Sandy, but why should people nowhere near it have to suffer? What about the businesses in CA that have lines with CC?

....it could have been a hell of a lot worse for CallCentric customers.

 
OR a lot BETTER, if their skinflint absentee owner(s) had kicked in the bucks to set up either a 2nd server farm in another city, or rent from at least one landlord who provides a proper generator - or do BOTH of these things.

Iscream was treading a fne line a few posts ago, when he spoke of budgetary constraints. - He also has his job to consider. - HE knows what it would take to make this right for the next time.

rblizz

join:2001-12-16
North Richland Hills, TX
reply to fukitol
said by fukitol:

Yeah, I don't buy the "it happened to a whole bunch of others, too" BULLSHIT. I don't give a flying fuck if a whole bunch of other companies didn't build a properly redundant network, either. If I ever used an excuse like that for bad behavior when I was growing up, my parents would ask me, "Well, if your friends jumped off a bridge, would you do it too?" I certainly hope that no one needed the E911 service for which they paid extra to CC over the past month. Oh, and where are the refunds? Hmm?

You know, I really don't give a flying fuck about what you don't give a flying fuck about. The narrow-minded, self contentedness here is beginning to piss me off. CallCentric never claimed to have geographic redundancy. Not once. Not ever. And it's not "bad behavior" not to have what they never advertised. What they always claimed was that their server farm was on the same electrical grid as the New York Stock Exchange. How often has the power gone out to the NYSE in the last thirty years? This is an extremely rare event and a once in a century storm. But stuff like this happens (rarely), regardless of how much posturing and huffing and puffing armchair quarterbacks spit out after the fact. And the reason the problems that the other companies are having is important to note, is because it this is a point of reference to show the scale of this storm and the damage it caused. It looks like the obvious is falling on deaf ears.

rblizz

join:2001-12-16
North Richland Hills, TX
reply to Davesnothere
said by Davesnothere:

OR, a lot BETTER, if their skinflint owner(s) kicked in the bucks to set up either a 2nd server farm in another city, or rent from at least one landlord who provides a proper generator - or do BOTH of these things.

If a company with a second server farm is important to you, then by all means, look for one -- I thought you already had moved on. Personally I don't care. I know how extremely rare an extended electrical outage is in lower Manhattan and especially one involving the New York Stock Exchange's grid. I don't expect another "storm of the century" to happen any time soon -- so I'm quite willing to "take my chances."


Davesnothere
No-BHELL-ity DOES have its Advantages
Premium
join:2009-06-15
START Today!
kudos:7

2 edits

1 recommendation

said by rblizz:

If a company with a second server farm is important to you, then by all means, look for one -- I thought you already had moved on. Personally I don't care. I know how extremely rare an extended electrical outage is in lower Manhattan and especially one involving the New York Stock Exchange's grid. I don't expect another "storm of the century" to happen any time soon -- so I'm quite willing to "take my chances."

 
IMNSHO, this is NOT about 'Playing the Odds'.

It's about customer service - at ALL levels of the operation, including planning and investment.

Below is part of what I said to someone yesturday in an IM :
quote:
I indeed AM angry that a company which is so professional in SO many other ways - can leave themselves WIDE open for such an epic fail in this regard.

One thing that this particular debacle illustrates and drives home - is the need for an even greater sophistication of process in doing our 'due diligence' when choosing a VoIP provider than most of us ever thought, as one poster [you, I believe] suggested to me.

I continue to have a lot of respect for CallCentric as a company - just not about THIS event's handling.

= = = = = = =

And BTW, in response to your reply to another poster :

What it would take to make ME shut my pie hole would be an open public announcement from Callcentric management that they COULD have done better if they had handled this event like VOIP.MS did in the same city, and what they will be doing in the next few months to CATCH UP with those CANUCKS in that regard.

rblizz

join:2001-12-16
North Richland Hills, TX

1 edit
said by Davesnothere:

This is not about 'Playing the Odds'.

It's about customer service - at ALL levels of the operation, including planning and investment.

I'm happy with the quality of service CallCentric has provided to me and I'm happy with their single server farm and the server redundancy they have there. It's not "playing the odds" -- the odds are in CallCentric's favor in my opinion. Until the DDoS attack and this storm, I've been blissfully using CallCentric and noting other provider's outages -- especially issues that seem to plague one of the companies that has multiple servers. Redundancy is nice, but it appears it can also introduce a whole new set of complications that can impact reliability.

You should also note "take my chances" was in quotes. In other words, I really don't think of it as taking a chance.

daneli

join:2012-10-23
Ann Arbor, MI

2 recommendations

reply to Iscream
I want to try to be constructive in my comments. Full redundancy with instant failover can be a very expensive and technically difficult challenge to meet. If a business can tolerate a longer time window (~ 8 hours) and can concentrate on only a subset of functions, then it can be much cheaper and easier to get some services back up and running in the face of a disruption. Certainly it should be possible for callcentric to provide a backup solution for its website and email that is more robust than the current solution. Perhaps a subset of functions - like voicemail or did forwarding - could also be made available from a remote location if the time window is extended to 12 hours or so?

What surprised me about Iscream's last post before the power went out was the revelation that callcentric has less than 60 minutes of backup power available in the event of a power failure. A diesel-powered generator isn't a perfect solution - recent events in New York confirm this - but it would go a long way to mitigate most of the risks associated with future power outages. (It isn't far-fetched to think that another power outage is likely in the future and that it could last more than 60 minutes.)

After the dust settles, I think it is reasonable for us to ask callcentric to be transparent about what steps if any it plans to take to mitigate these kinds of risks in the future. The risks I have in mind are not those associated with a "storm of the century" but the usual kinds of things - flood, fire, theft, etc - that businesses try to anticipate.

rblizz

join:2001-12-16
North Richland Hills, TX
said by daneli:

What surprised me about Iscream's last post before the power went out was the revelation that callcentric has less than 60 minutes of backup power available in the event of a power failure. A diesel-powered generator isn't a perfect solution - recent events in New York confirm this - but it would go a long way to mitigate most of the risks associated with future power outages. (It isn't far-fetched to think that another power outage is likely in the future and that it could last more than 60 minutes.)

Unfortunately it doesn't appear that a generator was an option in this building. But, maybe now, CallCentric can use this event as leverage with their landlords and get permission to install a generator.


Davesnothere
No-BHELL-ity DOES have its Advantages
Premium
join:2009-06-15
START Today!
kudos:7
reply to daneli
said by daneli:

I want to try to be constructive in my comments....

 
Absolutely, and I agree with pretty well all of them.

BTW, their UPS runtime might have been longer BEFORE they added a bunch of extra servers last month to help handle the DDoS attacks more effectively.