|
to dlarkin_dc
Re: [VA] E-mail POP access down?Restoration not expected today.
"Our teams are making progress in our email service restoration efforts. Through our investigation, we have isolated the cause of the failure to the email system platform which ultimately affected both the primary and secondary failover (back-up) systems. Currently, all incoming messages are queued in our system and we expect to deliver messages to customers on a rolling basis as we bring the systems back online.
Teams across all levels of our organization remain intensely focused on the restoration efforts. Unfortunately, we do not expect services to be restored today. We do understand the impact this is having on our customers, which is why we have deployed every resource possible to working this issue. While we are focused on restoring email access as quickly as possible, the multiple components and processes that make up our email system require care, and some time, to bring them back online." |
|
|
JeffinNOLA to dlarkin_dc
Anon
2012-Dec-16 12:59 pm
to dlarkin_dc
This is really fustrating. I need a primary account to pay some bills, but I can't because of COX. |
|
digiblur Premium Member join:2002-06-03 Louisiana |
digiblur
Premium Member
2012-Dec-16 1:24 pm
Can't pay bills due to email? Confused on that one. I never needed email at all to pay bills. |
|
|
Some get billed through e-mail. That is, the invoice arrives by e-mail and there's a link to the service provider's site for a secure pay. |
|
dlarkin_dc |
to dmorack
I would love to know what caused neither of the backup systems come online! |
|
|
1 recommendation |
to dlarkin_dc
Heads should roll over this one and we should all get a credit on our bills. Whether or not people should use the ISP's email is not the issue, they offer the service and its a reasonable expectation to expect it to work. I don't really care when its out for 15 minutes in the middle of the night due to a maintenance window, but going on 48 hours with no end is sight is not acceptable. |
|
ajwees41 Premium Member join:2002-05-10 Omaha, NE |
to dlarkin_dc
per the article looks like only one backup and the samething that happened to the primary happened to the secondary. |
|
|
to BillRoland
Oklahoma City afternoon still down |
|
|
to ajwees41
ajwee41,
Yup, the servers fell like dominos. But what triggered the whole failure? What took the initial server down, then the failover servers? Corrupt data, overload of data, did a server glitch start it off, was some outside influence like a UPS failure in the data center... just what exactly happened?
We deserve to know and it will cut through any consumer-paranoia that's brewing... unless it was so fundamental Cox can only look foolish. Even then, I'd prefer a foolish honest ISP. |
|
|
It would be nice, but as someone who has been in IT for a long time and with first-hand knowledge about several incidents which resulted in very high profile outages (bank ATM network, on-line retailer and others), you will probably be disappointed.
A. No one wants the liability. If WidgetCo publicly said "Our factory was shut down because an Acme server died and they couldn't fix it for 6 days" they would most likely be sued. In fact, the Acme server might have been brought down by a failed KoreaCo memory stick. The memory modules were supplied by SiliconCo. A replacement part was shipped via SpeedyCo, lost in transit and eventually installed backwards by a WidgetCo tech, etc.
B. It will take a while to sort out what went wrong.
C. Most of the time, it's human error - either at the root or along the way to exacerbate the situation.
I've never seen a full, factual, detailed accounting of a major outage by a large company. |
|
|
to dlarkin_dc
Re: E-mail POP access down?I find it interesting that Outlook reports two different errors for two different Cox email accounts. For one e-mail it says "The operation timed out waiting for a response from the receiving (POP) server," the other e-mail says "The e-mail server rejected your login."
This is pure speculation on my part, but my guess is we're not looking at hardware failure, but some kind of update that was rolled out and started acting flaky, so they tried to roll it back and that's when everything went down the sewer. What we're waiting on now is for the vendor to figure out how to get it back working. |
|
chip89 Premium Member join:2012-07-05 Columbia Station, OH |
to ajwees41
Re: [VA] E-mail POP access down?You think they would of fought to put more backups in then just one system. |
|
|
to No_Strings
said by No_Strings:I've never seen a full, factual, detailed accounting of a major outage by a large company. Ditto that - after 25 years doing this. |
|
|
to dlarkin_dc
Re: E-mail POP access down?More thorough status up now - no intimate details but at least a lot more info. |
|
|
to KahunaNui
Re: [VA] E-mail POP access down?Yeah, it's a dream.
"...an issue within the email storage platform that supports Cox Residential Email" leaves a lot to interpretation but it might be as much as we ever get. |
|
|
said by dlarkin_dc:Yeah, it's a dream.
"...an issue within the email storage platform that supports Cox Residential Email" leaves a lot to interpretation but it might be as much as we ever get. I'd imagine you're right, I doubt we'll get anything clearer than that. |
|
1 edit |
to dlarkin_dc
Re: E-mail POP access down?I am in the Hampton Roads area of Virginia and I am now able to log into my email using Windows Live Mail without an error. I also have 3 emails in my inbox from this weekend. 2 show received yesterday and 1 is showing received today at around 12:21 am. I can also log into my email via the Cox web mail site which I couldn't do yesterday or earlier today. So it looks like they finally are bringing things back up.
Edit: They just updated the page and are saying this now...
"As of 6:00 pm ET, service restoration efforts are progressing. The system is beginning to come back online. Service could be intermittent as we work to fully recover. Some customers may be able to access emails starting tonight. This is a lengthy process and we still do not know when service will be restored for all customers. Once your email access is restored, you will immediately be able to send and receive new messages. Currently, all messages that have been received since Friday are queued in our system and we anticipate delivering messages to customers on a rolling basis as we bring the platform back online. Your queued emails will arrive gradually and may not be delivered in chronological order. You may continue to receive these queued emails over the course of several days after your service is restored. We are happy to share this positive progress with you and will remain steadfastly focused on this effort until fully resolved. We deeply regret the impact this has had, especially at a time when many customers are busy preparing for holiday celebrations with their families and friends. Please check back for another update at 9 p.m. ET." |
|
digiblur Premium Member join:2002-06-03 Louisiana |
to dlarkin_dc
Re: [VA] E-mail POP access down?said by dlarkin_dc:Some get billed through e-mail. That is, the invoice arrives by e-mail and there's a link to the service provider's site for a secure pay. Yes, but you should always keep the website addresses to bills you are responsible for. Plus there are other alternative methods. |
|
·Frontier FiberOp..
1 edit |
to dlarkin_dc
Re: E-mail POP access down?According to the Facebook page people in Virginia, Nebraska and Kansas are reporting that they are now receiving email. Others (including myself) can now access their primary email account. It appears Cox is beginning their "rolling access" to accounts. |
|
|
critarp
Anon
2012-Dec-16 8:25 pm
Nope, still no email in Hampton Roads. Worst provider ever. |
|
ajwees41 Premium Member join:2002-05-10 Omaha, NE |
to Ender3rd
Yep working, but slow retrieval in Omaha |
|
MarkRH Premium Member join:2005-02-08 Edmond, OK ARRIS BGW210-700 ARRIS TM3402 Asus RT-AC68
|
to dlarkin_dc
Looks like pop.east.cox.net is still down for the count (Oklahoma City). Timing out in Thunderbird. I only get email sent from Cox there so it doesn't affect me much. Haven't tried sending email, however.
Back during the days when I was moving around so much (dial-up) and changing ISPs somewhat frequently, I set up a separate email service so I'd have a consistent email address and I would only have to change the forwarding email address in that one spot rather than update my email in a bazillion places. I'm still using it. |
|
|
said by MarkRH:Looks like pop.east.cox.net is still down for the count (Oklahoma City). Timing out in Thunderbird. I only get email sent from Cox there so it doesn't affect me much. Haven't tried sending email, however.
Back during the days when I was moving around so much (dial-up) and changing ISPs somewhat frequently, I set up a separate email service so I'd have a consistent email address and I would only have to change the forwarding email address in that one spot rather than update my email in a bazillion places. I'm still using it. FWIW... I am now receiving email on our two Cox POP accounts. We are in the OKC market... |
|
|
Likewise here - in Oklahoma City, I am receiving email for 2 accounts - tho i'm not sure all of it has been processed yet. Anyway, it's up. |
|
1 edit |
I can now logon, get e-mail from Friday, Sunday and messages coming thorugh that are sent today. Still have a gap for Sunday mails.
UPDATE: Seems, just as Cox have on their alert update, the weekend mail is trickling in. |
|
|
to dlarkin_dc
I'm now getting e-mail as far back as last WEDNESDAY that is starting to show up. |
|
|
Maybe my mid-week lost mails will show up! |
|
|
al_snow
Anon
2012-Dec-17 2:22 pm
i'm happy everything working although i had to call for password reset and my hold time wait was short.so all is well and i will have to stay with outlook from cox too much to change. thanks for you guys info. okla city/moore area |
|
|
Yeah, I just got a mail from Friday. Well, everyting is going according to the plan/alert message from Cox.
Time to un-pin this thread? |
|