Cox → E-mail POP access down?

Yep. 24hrs and counting... I read a post saying Sunday!

One thing that bugs me is that we're given no information. Saying 6am and then needing double the time to resolve the problem looks very bad. It looks like they can't even estimate a fix to a system failure that should have never been an issue. It looks like gross incompotence.

All that that makes for grumbly customers. If we'd some information we could be more understanding. Unless that would make it look worse than a grossly incompotent scenario.

Oh well. I'm sure a few of us are getting closer to moving to FiOS over this.

At a minimum, I want (a) a good explaination about why the e-mail has been hinky for a week then down for over 24hrs, and (b) assurance that my e-mail has been secure.

Still no email in Hampton Roads! AAAHhhhhhhhhhhhhhhhh!

You are so correct - I am concerned about security of my email account and would appreciate some sort of response to let me know if security has been compromised - and what needs to be done by the customer to overcome it. More information would help relieve the concern of Cox customers so much.

the email server went down and needs to be replaced

ajwees41,

Thanks for posting the server issue. 26hrs to replace a server seems crazy to me. I couldn't let my customers be out of service for that long over such a fundamental.

Maybe they are checking to see if the server was also the cause of the outage the other day.

FWIW - From the support section of the Cox site...

Summary
Email Outage Update as of 8:30 pm, December 15th

Detail
As of 8:30 pm ET, the effort to restore residential customer email service continues. This work will continue through the night. We will provide additional updates early Sunday morning. We have no indication of any security breach that would put customer information at risk. We sincerely apologize for the inconvenience. All technical teams are fully engaged and committed to restoring service as quickly as we can.

$50 off our next bill would seem appropriate.

What I'm concerned about most is what is happening to e-mail that has been sent to me during this outage? Did it bounce? Did it go into oblivion? I don't use this address for business critical stuff but I would be upset to find out people have sent me stuff that never arrived.

Cox will never give us a credit. They are not the type of company that would do the right thing.

. It's being said that incoming mail is fine and will be available when the server comes back online.

It looks to me like cox decided to mess with the pop servers vs webmail servers. MS had the same sort of problem some years ago, and it was several days before things got sorted out.

They just posted another update...

"Restoration continues and we are actively exploring multiple approaches to restore email services. All available resources are engaged across multiple teams.These teams will continue working around the clock until your email is restored. Based on our investigation, event logs, and other data available to us, we have no indication that this incident is the result of a security breach that would put customer information at risk. We will provide additional updates by 9 am ET. We sincerely apologize for the inconvenience this is causing you."

During the week up to this. I know of two e-mails that were sent that I never received. There may have been one other but I can't confirm it.

So I have 98% faith that all our mail is hunky dory.

I have been down in CONNECTICUT SINCE FRIDAY

And here it is past 9 am ET and same message with no update. Anybody surprised?

Cox Support update... no end in sight.

Summary
Email Outage Update as of 9:30 am, December 16th

Detail
Since our update at 1 a.m., our teams continue to make progress in restoring email service to our customers in the Midwest and East Coast regions. Teams across all levels of our organization are focused on the recovery and we truly apologize for the inconvenience this has caused. At this time we cannot provide an estimated time for recovery, but will continue to work around the clock until email service is restored.

Appears to be down in FL too.

I would have never known if it wasn't for this thread. Haven't ever used the Cox email. Never did in case I needed to switch providers.

Restoration not expected today.

"Our teams are making progress in our email service restoration efforts. Through our investigation, we have isolated the cause of the failure to the email system platform which ultimately affected both the primary and secondary failover (back-up) systems. Currently, all incoming messages are queued in our system and we expect to deliver messages to customers on a rolling basis as we bring the systems back online.

Teams across all levels of our organization remain intensely focused on the restoration efforts. Unfortunately, we do not expect services to be restored today. We do understand the impact this is having on our customers, which is why we have deployed every resource possible to working this issue. While we are focused on restoring email access as quickly as possible, the multiple components and processes that make up our email system require care, and some time, to bring them back online."

This is really fustrating. I need a primary account to pay some bills, but I can't because of COX.

Can't pay bills due to email? Confused on that one. I never needed email at all to pay bills.

Some get billed through e-mail. That is, the invoice arrives by e-mail and there's a link to the service provider's site for a secure pay.

I would love to know what caused neither of the backup systems come online!

Heads should roll over this one and we should all get a credit on our bills. Whether or not people should use the ISP's email is not the issue, they offer the service and its a reasonable expectation to expect it to work. I don't really care when its out for 15 minutes in the middle of the night due to a maintenance window, but going on 48 hours with no end is sight is not acceptable.

per the article looks like only one backup and the samething that happened to the primary happened to the secondary.

Oklahoma City afternoon still down

ajwee41,

Yup, the servers fell like dominos. But what triggered the whole failure? What took the initial server down, then the failover servers? Corrupt data, overload of data, did a server glitch start it off, was some outside influence like a UPS failure in the data center... just what exactly happened?

We deserve to know and it will cut through any consumer-paranoia that's brewing... unless it was so fundamental Cox can only look foolish. Even then, I'd prefer a foolish honest ISP.

It would be nice, but as someone who has been in IT for a long time and with first-hand knowledge about several incidents which resulted in very high profile outages (bank ATM network, on-line retailer and others), you will probably be disappointed.

A. No one wants the liability. If WidgetCo publicly said "Our factory was shut down because an Acme server died and they couldn't fix it for 6 days" they would most likely be sued. In fact, the Acme server might have been brought down by a failed KoreaCo memory stick. The memory modules were supplied by SiliconCo. A replacement part was shipped via SpeedyCo, lost in transit and eventually installed backwards by a WidgetCo tech, etc.

B. It will take a while to sort out what went wrong.

C. Most of the time, it's human error - either at the root or along the way to exacerbate the situation.

I've never seen a full, factual, detailed accounting of a major outage by a large company.