dslreports logo
 
    All Forums Hot Topics Gallery
spc
Search similar:


uniqs
9348
dmorack
join:2005-06-30
Herndon, VA

dmorack to dlarkin_dc

Member

to dlarkin_dc

Re: [VA] E-mail POP access down?

Restoration not expected today.

"Our teams are making progress in our email service restoration efforts. Through our investigation, we have isolated the cause of the failure to the email system platform which ultimately affected both the primary and secondary failover (back-up) systems. Currently, all incoming messages are queued in our system and we expect to deliver messages to customers on a rolling basis as we bring the systems back online.

Teams across all levels of our organization remain intensely focused on the restoration efforts. Unfortunately, we do not expect services to be restored today. We do understand the impact this is having on our customers, which is why we have deployed every resource possible to working this issue. While we are focused on restoring email access as quickly as possible, the multiple components and processes that make up our email system require care, and some time, to bring them back online."

JeffinNOLA
@cox.net

JeffinNOLA to dlarkin_dc

Anon

to dlarkin_dc
This is really fustrating. I need a primary account to pay some bills, but I can't because of COX.

digiblur
Premium Member
join:2002-06-03
Louisiana

digiblur

Premium Member

Can't pay bills due to email? Confused on that one. I never needed email at all to pay bills.

dlarkin_dc
join:2009-08-24
Alexandria, VA

dlarkin_dc

Member

Some get billed through e-mail. That is, the invoice arrives by e-mail and there's a link to the service provider's site for a secure pay.
dlarkin_dc

dlarkin_dc to dmorack

Member

to dmorack
I would love to know what caused neither of the backup systems come online!

BillRoland
Premium Member
join:2001-01-21
Ocala, FL

1 recommendation

BillRoland to dlarkin_dc

Premium Member

to dlarkin_dc
Heads should roll over this one and we should all get a credit on our bills. Whether or not people should use the ISP's email is not the issue, they offer the service and its a reasonable expectation to expect it to work. I don't really care when its out for 15 minutes in the middle of the night due to a maintenance window, but going on 48 hours with no end is sight is not acceptable.
ajwees41
Premium Member
join:2002-05-10
Omaha, NE

ajwees41 to dlarkin_dc

Premium Member

to dlarkin_dc
per the article looks like only one backup and the samething that happened to the primary happened to the secondary.

larry n
@cox.net

larry n to BillRoland

Anon

to BillRoland
Oklahoma City afternoon still down

dlarkin_dc
join:2009-08-24
Alexandria, VA

dlarkin_dc to ajwees41

Member

to ajwees41
ajwee41,

Yup, the servers fell like dominos. But what triggered the whole failure? What took the initial server down, then the failover servers? Corrupt data, overload of data, did a server glitch start it off, was some outside influence like a UPS failure in the data center... just what exactly happened?

We deserve to know and it will cut through any consumer-paranoia that's brewing... unless it was so fundamental Cox can only look foolish. Even then, I'd prefer a foolish honest ISP.

No_Strings

join:2001-11-22
The OC

No_Strings

It would be nice, but as someone who has been in IT for a long time and with first-hand knowledge about several incidents which resulted in very high profile outages (bank ATM network, on-line retailer and others), you will probably be disappointed.

A. No one wants the liability. If WidgetCo publicly said "Our factory was shut down because an Acme server died and they couldn't fix it for 6 days" they would most likely be sued. In fact, the Acme server might have been brought down by a failed KoreaCo memory stick. The memory modules were supplied by SiliconCo. A replacement part was shipped via SpeedyCo, lost in transit and eventually installed backwards by a WidgetCo tech, etc.

B. It will take a while to sort out what went wrong.

C. Most of the time, it's human error - either at the root or along the way to exacerbate the situation.

I've never seen a full, factual, detailed accounting of a major outage by a large company.

BillRoland
Premium Member
join:2001-01-21
Ocala, FL

BillRoland to dlarkin_dc

Premium Member

to dlarkin_dc

Re: E-mail POP access down?

I find it interesting that Outlook reports two different errors for two different Cox email accounts. For one e-mail it says "The operation timed out waiting for a response from the receiving (POP) server," the other e-mail says "The e-mail server rejected your login."

This is pure speculation on my part, but my guess is we're not looking at hardware failure, but some kind of update that was rolled out and started acting flaky, so they tried to roll it back and that's when everything went down the sewer. What we're waiting on now is for the vendor to figure out how to get it back working.

chip89
Premium Member
join:2012-07-05
Columbia Station, OH

chip89 to ajwees41

Premium Member

to ajwees41

Re: [VA] E-mail POP access down?

You think they would of fought to put more backups in then just one system.

KahunaNui
join:2000-05-01
Honolulu, HI

KahunaNui to No_Strings

Member

to No_Strings
said by No_Strings:

I've never seen a full, factual, detailed accounting of a major outage by a large company.

Ditto that - after 25 years doing this.
sullyguy
join:2011-08-25
Centreville, VA

sullyguy to dlarkin_dc

Member

to dlarkin_dc

Re: E-mail POP access down?

More thorough status up now - no intimate details but at least a lot more info.

dlarkin_dc
join:2009-08-24
Alexandria, VA

dlarkin_dc to KahunaNui

Member

to KahunaNui

Re: [VA] E-mail POP access down?

Yeah, it's a dream.

"...an issue within the email storage platform that supports Cox Residential Email" leaves a lot to interpretation but it might be as much as we ever get.

BillRoland
Premium Member
join:2001-01-21
Ocala, FL

BillRoland

Premium Member

said by dlarkin_dc:

Yeah, it's a dream.

"...an issue within the email storage platform that supports Cox Residential Email" leaves a lot to interpretation but it might be as much as we ever get.

I'd imagine you're right, I doubt we'll get anything clearer than that.
betterdan
join:2002-03-20
Portsmouth, VA

1 edit

betterdan to dlarkin_dc

Member

to dlarkin_dc

Re: E-mail POP access down?

I am in the Hampton Roads area of Virginia and I am now able to log into my email using Windows Live Mail without an error. I also have 3 emails in my inbox from this weekend. 2 show received yesterday and 1 is showing received today at around 12:21 am.
I can also log into my email via the Cox web mail site which I couldn't do yesterday or earlier today.
So it looks like they finally are bringing things back up.

Edit: They just updated the page and are saying this now...

"As of 6:00 pm ET, service restoration efforts are progressing. The system is beginning to come back online. Service could be intermittent as we work to fully recover. Some customers may be able to access emails starting tonight. This is a lengthy process and we still do not know when service will be restored for all customers. Once your email access is restored, you will immediately be able to send and receive new messages.
Currently, all messages that have been received since Friday are queued in our system and we anticipate delivering messages to customers on a rolling basis as we bring the platform back online. Your queued emails will arrive gradually and may not be delivered in chronological order. You may continue to receive these queued emails over the course of several days after your service is restored.
We are happy to share this positive progress with you and will remain steadfastly focused on this effort until fully resolved. We deeply regret the impact this has had, especially at a time when many customers are busy preparing for holiday celebrations with their families and friends. Please check back for another update at 9 p.m. ET."

digiblur
Premium Member
join:2002-06-03
Louisiana

digiblur to dlarkin_dc

Premium Member

to dlarkin_dc

Re: [VA] E-mail POP access down?

said by dlarkin_dc:

Some get billed through e-mail. That is, the invoice arrives by e-mail and there's a link to the service provider's site for a secure pay.

Yes, but you should always keep the website addresses to bills you are responsible for. Plus there are other alternative methods.

Ender3rd
join:2001-07-15
Connecticut
·Frontier FiberOp..

1 edit

Ender3rd to dlarkin_dc

Member

to dlarkin_dc

Re: E-mail POP access down?

According to the Facebook page people in Virginia, Nebraska and Kansas are reporting that they are now receiving email. Others (including myself) can now access their primary email account. It appears Cox is beginning their "rolling access" to accounts.

critarp
@cox.net

critarp

Anon

Nope, still no email in Hampton Roads. Worst provider ever.
ajwees41
Premium Member
join:2002-05-10
Omaha, NE

ajwees41 to Ender3rd

Premium Member

to Ender3rd
Yep working, but slow retrieval in Omaha

MarkRH
Premium Member
join:2005-02-08
Edmond, OK
ARRIS BGW210-700
ARRIS TM3402
Asus RT-AC68

MarkRH to dlarkin_dc

Premium Member

to dlarkin_dc
Looks like pop.east.cox.net is still down for the count (Oklahoma City). Timing out in Thunderbird. I only get email sent from Cox there so it doesn't affect me much. Haven't tried sending email, however.

Back during the days when I was moving around so much (dial-up) and changing ISPs somewhat frequently, I set up a separate email service so I'd have a consistent email address and I would only have to change the forwarding email address in that one spot rather than update my email in a bazillion places. I'm still using it.

SoonerAl
MVM
join:2002-07-23
Norman, OK

SoonerAl

MVM

said by MarkRH:

Looks like pop.east.cox.net is still down for the count (Oklahoma City). Timing out in Thunderbird. I only get email sent from Cox there so it doesn't affect me much. Haven't tried sending email, however.

Back during the days when I was moving around so much (dial-up) and changing ISPs somewhat frequently, I set up a separate email service so I'd have a consistent email address and I would only have to change the forwarding email address in that one spot rather than update my email in a bazillion places. I'm still using it.

FWIW...

I am now receiving email on our two Cox POP accounts. We are in the OKC market...

johnburns
join:2004-10-14
Oklahoma City, OK

johnburns

Member

Likewise here - in Oklahoma City, I am receiving email for 2 accounts - tho i'm not sure all of it has been processed yet. Anyway, it's up.

dlarkin_dc
join:2009-08-24
Alexandria, VA

1 edit

dlarkin_dc

Member

I can now logon, get e-mail from Friday, Sunday and messages coming thorugh that are sent today. Still have a gap for Sunday mails.

UPDATE: Seems, just as Cox have on their alert update, the weekend mail is trickling in.

BillRoland
Premium Member
join:2001-01-21
Ocala, FL

BillRoland to dlarkin_dc

Premium Member

to dlarkin_dc
I'm now getting e-mail as far back as last WEDNESDAY that is starting to show up.

dlarkin_dc
join:2009-08-24
Alexandria, VA

dlarkin_dc

Member

Maybe my mid-week lost mails will show up!

al_snow
@cox.net

al_snow

Anon

i'm happy everything working although i had to call for password reset
and my hold time wait was short.so all is well and i will have to stay with outlook from cox too much to change. thanks for you guys info. okla city/moore area

dlarkin_dc
join:2009-08-24
Alexandria, VA

dlarkin_dc

Member

Yeah, I just got a mail from Friday. Well, everyting is going according to the plan/alert message from Cox.

Time to un-pin this thread?