republican-creole
site Search:


 
   
story category
Sidekick Users Get Outage AND Total Data Loss
And it's apparently courtesy of Microsoft?
by Karl Bode Monday 12-Oct-2009 tags: business · bandwidth · trouble · consumers
Tipped by Z80 See Profile
Users of the T-Mobile Sidekick have been writing in to complain about a week-plus outage, and now they're being informed that they'll be losing personal data as well. According to a post made over at the T-Mobile website and the Wall Street Journal, problems with Microsoft servers have resulted in a total data loss for any data not backed up on users' personal devices. How's Microsoft involved? Microsoft took over the guts of the Sidekick service and software when they acquired Danger Inc. early last year. The outage of course begs questions about the stability of the so-called "cloud" (though Oracle's CEO is one of many who think that's a dumb name for computers on a network), but it also begs the age old question: where was the redundancy?

Update:It appears that customers will at least be getting a month's service credit. T-Mobile is also halting sales of new service and is letting customers under contract cancel service without penalty.

view: topics flat text 
Post a:
page: 1 · 2

CurGeorge8

join:2005-05-02
South Park, PA

Fail

Epic. Fail.

45071419

join:2006-07-30

Re: Fail

said by CurGeorge8:

Epic. Fail.
Big time. So who got fired this weekend?

P.S. "The cloud" term is dumb.

CurGeorge8

join:2005-05-02
South Park, PA

Re: Fail

said by 45071419:

said by CurGeorge8:

Epic. Fail.
Big time. So who got fired this weekend?

P.S. "The cloud" term is dumb.
I'm sure everyone was canned after having faced the wrath of Paris Hilton.

ReVeLaTeD
Premium
join:2001-11-10
San Diego, CA

Re: Fail

said by CurGeorge8:

said by 45071419:

said by CurGeorge8:

Epic. Fail.
Big time. So who got fired this weekend?

P.S. "The cloud" term is dumb.
I'm sure everyone was canned after having faced the wrath of Paris Hilton.
Paris went to BlackBerry ages ago. But the joke is quite comedy.

IT's mighty funny though that this type of thing happens after Microsoft bought Danger. Prior to that, Danger would be down, but rarely if ever was data lost.
Kdee

join:2005-08-26
Etobicoke, ON
I think you mean Perez Hilton. Apparently he's pretty pissed.

ReformCRTC
Support Your Independent ISP

join:2004-03-07
Canada
I agree cloud is dumb. Like their head's in a Fog...I don't deal in fogs, I deal in backing up data to physical media.
speedy99

join:2002-09-25
Saratoga, CA
If they actually used "the cloud" - or any secondary data storage system - they would have backups. This has nothing to do with "the cloud", and everything to do with foolishly depending on a SAN with no backups.

ThrowDemsOut
If you can't convince 'em, confuse 'em
Premium
join:2002-03-03
Mullica Hill, NJ
kudos:4
said by CurGeorge8:

Epic. Fail.
Servers should be BACKED UP daily and weekly & monthly backups should be held offsite.

milnoc

join:2001-03-05
H3B
kudos:1

Re: Fail

What's the best way to back up thousands of terabytes (or petabytes) of data? A second system maybe?
glinc

join:2009-04-07
New York, NY

Re: Fail

exactly, they should have all hdd's set up with raid-1
itguy05

join:2005-06-17

Re: Fail

said by glinc:

exactly, they should have all hdd's set up with raid-1
I've seen a RAID 1 controller puke and take both drives out. Or even worse, corruption on 1 drive immediately gets replicated to the other.

And, yes, I've seen even RAID 5 arrays go tits-up.

Of course people need to realize that even with tape backups, a full system restore of lots and lots of TB pretty much never happens. Where I work we have many, many (90 maybe) TB of data. It has been calculated that a full database restore from tape would take almost a month. I forget if that takes into account the validation work or not. Needless to say we have a redundant system somewhere else.

However, I wonder where it was said it was a SAN - the articles I've saw said "server"....

Working in IT for as long as I have, I've seen it all. Most likely what happened this time was there was:

1) No $$ for that spare SAN.
2) The risk was supposed to be so low as to not cause an issue.

My bet is on #2 - I've seen way too much of those types of things cause issues. Including work on 1/2 of a fully redundant SAN take the whole thing down where the vendor is still scratching their heads years later.
bjbrock

join:2002-10-28
Mcalester, OK

Re: Fail

Properly configured RAID1 uses 2 controllers.

tmh

@verizon.net
said by itguy05:

And, yes, I've seen even RAID 5 arrays go tits-up.
We had a 16 bay Promise VTrak die after a power surge (yes, it was on a UPS). All but one drive went south, so even RAID 5 was useless.

We did have backups fortunately.

John McClane
yippee ki yay
Premium
join:2005-03-19
even better would be to have several different sites all doing the same thing. and then have backups of those.

Matt
All noise, no signal.
Premium
join:2003-07-20
Jamestown, NC
kudos:12
said by milnoc:

What's the best way to back up thousands of terabytes (or petabytes) of data? A second system maybe?
It's a SAN that failed, apparently one that was being upgraded for Microsoft by Hitachi. So the better question would be, why were live customers accessing a SAN that was being upgraded and why did the upgrade nullify the redundancy?
--
"What is conservatism? Is it not adherence to the old and tried, against the new and untried?" - Abraham Lincoln
patcat88

join:2002-04-05
Jamaica, NY
kudos:1

Re: Fail

said by Matt:

It's a SAN that failed, apparently one that was being upgraded for Microsoft by Hitachi. So the better question would be, why were live customers accessing a SAN that was being upgraded and why did the upgrade nullify the redundancy?
Because the blond sales agent from Hitachi said they solution can do it, and the boss ordered the IT minions to do it, the blond sales agent promised it to him.

powerhog
Stinkin' up the joint
Premium
join:2000-12-14
Owasso, OK

Re: Fail

LOL- so true!

I worked at a company where one of the vendor reps was a hot blonde woman. Needless to say, the datacenter quickly filled with her company's servers even though they were, by far, the most prone to failure of all the systems we had.
dsless

join:2001-05-16
Pittsburgh, PA
Ugh Hitachi!

lolwhat
You're getting warmer
Premium
join:2001-06-11
PonziWorld
Reviews:
·Callcentric
Anything. Anything at all. Microsoft isn't nicknamed M$ for nothing; they have gobs of money, and they could've used it to do something for a backup.
--
The Greater Depression: Coming soon to a country near you!
F*** the Republicrats.

ReformCRTC
Support Your Independent ISP

join:2004-03-07
Canada
PETabytes? Sounds kinky...

John McClane
yippee ki yay
Premium
join:2005-03-19

Re: Fail

its petabytes. your thinking of pedobytes

james

join:2001-02-26
CWCville USA

Re: Fail

said by John McClane:

its petabytes. your thinking of pedobytes
Why don't you have a seat right there?
--
said by Metatron2008:

But people who download thousands of movies and games.... Yes, they are as bad as any murderer

John McClane
yippee ki yay
Premium
join:2005-03-19

Re: Fail

you forgot this

tiger72
SexaT duorP
Premium
join:2001-03-28
Saint Louis, MO
kudos:1
Reviews:
·T-Mobile US
·AT&T DSL Service
precisely.

From what I read, the loss of personal data was due to a system upgrade. For some reason, Microsoft/Danger did not do a data backup before making the big upgrade. In fact, they apparently hadn't done ANY data backups EVER?! They don't have any "old" backups to fall back on.

I'm sure heads will roll at Microsoft for this. And deservedly so. They'll have cost Microsoft and T-Mobile $millions in damages.
--
"What makes us omniscient? Have we a record of omniscience? ...If we can't persuade nations with comparable values of the merit of our cause, we'd better reexamine our reasoning."
-United States Secretary of Defense (1961-1968) Robert S. McNamara
moonpuppy

join:2000-08-21
Glen Burnie, MD

Re: Fail

said by tiger72:

I'm sure heads will roll at Microsoft for this. And deservedly so. They'll have cost Microsoft and T-Mobile $millions in damages.
At least T-Mobile can have Google to fall back on.

45071419

join:2006-07-30
said by tiger72:

From what I read, the loss of personal data was due to a system upgrade. For some reason, Microsoft/Danger did not do a data backup before making the big upgrade. In fact, they apparently hadn't done ANY data backups EVER?!
I guess "Change Management" to them means "Let's upgrade and start over from scratch".

Camelot One
Premium,MVM
join:2001-11-21
Austin, TX
kudos:1
said by tiger72:

I'm sure heads will roll at Microsoft for this. And deservedly so. They'll have cost Microsoft and T-Mobile $millions in damages.
I'm trying to figure out how exactly. The users are all under contract, so they are stuck with the service no matter what. And I guarantee you there is something in the TOS stating they aren't responsible for data loss.

tiger72
SexaT duorP
Premium
join:2001-03-28
Saint Louis, MO
kudos:1
Reviews:
·T-Mobile US
·AT&T DSL Service

Re: Fail

said by Camelot One:

said by tiger72:

I'm sure heads will roll at Microsoft for this. And deservedly so. They'll have cost Microsoft and T-Mobile $millions in damages.
I'm trying to figure out how exactly. The users are all under contract, so they are stuck with the service no matter what. And I guarantee you there is something in the TOS stating they aren't responsible for data loss.
TMO has already given customers a free month of service - that in itself is an extremely expensive move. Not to mention the likely class-action suit that will no doubt be brought forth...
--
"What makes us omniscient? Have we a record of omniscience? ...If we can't persuade nations with comparable values of the merit of our cause, we'd better reexamine our reasoning."
-United States Secretary of Defense (1961-1968) Robert S. McNamara

r81984
Fair and Balanced
Premium
join:2001-11-14
Katy, TX
Reviews:
·AT&T U-Verse
·AT&T DSL Service
·row44
said by tiger72:

They'll have cost Microsoft and T-Mobile $millions in damages.
Damages for what? Did they guarantee storage of data?
--
Democrats are not Socialists any more than Republicans are.

tiger72
SexaT duorP
Premium
join:2001-03-28
Saint Louis, MO
kudos:1
Reviews:
·T-Mobile US
·AT&T DSL Service

Re: Fail

said by r81984:

said by tiger72:

They'll have cost Microsoft and T-Mobile $millions in damages.
Damages for what? Did they guarantee storage of data?
It doesn't matter. If a class-action suit is brought forward, TMO and Microsoft would want to settle early and quickly to get it all out of the news. They try to build their images on "reliability", and the longer this is in the news, the more likely they'll lose future customers. A settlement (regardless of what the actual contract states) would get it out of the news - but at a price.
--
"What makes us omniscient? Have we a record of omniscience? ...If we can't persuade nations with comparable values of the merit of our cause, we'd better reexamine our reasoning."
-United States Secretary of Defense (1961-1968) Robert S. McNamara

ptrowski
Got Helix?
Premium
join:2005-03-14
Putnam, CT
kudos:4
Reviews:
·VOIPo
·Choice One Commu..
said by r81984:

said by tiger72:

They'll have cost Microsoft and T-Mobile $millions in damages.
Damages for what? Did they guarantee storage of data?
Very good point.
--
"So, Lone Starr, now you see that evil will always triumph because good is dumb."

Have you been touched by his noodly appendage? »www.venganza.org

John McClane
yippee ki yay
Premium
join:2005-03-19

Re: Fail

for one, i don't know, but I don't think that T-mobile does more than provide voice and data service. they do not control the information on the devices and therefore how its backed up.. or am i wrong?
bjbrock

join:2002-10-28
Mcalester, OK
It's called merchantability. They sold a working service and there is minimum expected performance on their part. Plus there are some things you can't "TOS" your way out of.

There will be a law suit, or a settlement.

r81984
Fair and Balanced
Premium
join:2001-11-14
Katy, TX
Reviews:
·AT&T U-Verse
·AT&T DSL Service
·row44

Re: Fail

said by bjbrock:

It's called merchantability. They sold a working service and there is minimum expected performance on their part. Plus there are some things you can't "TOS" your way out of.

There will be a law suit, or a settlement.
With it comes to technology the minimum expected performance is you know to back up your data yourself. I thought everyone knew that.

I am sorry, but lawsuit for losing data that no one guaranteed is pointless. This is a case that Tmobile knows it will win so if they settle out of court Tmobile will only pay what it would lose paying to win the court case to save time. Once divided up people will get like a $1 and lawyers will get most of the money.
Also it will be hard to claim damages over lost phone numbers and emails.
--
Democrats are not Socialists any more than Republicans are.
rahvin112

join:2002-05-24
Sandy, UT

Re: Fail

They made claims in advertising and contract documents for usability and they were expected hold to minimum network and management practices. Those expected standards include backups. When they get to court they will have to explain to the jury why they didn't follow backup standards that every single enterprise grade company in they country follows. Regardless of what the TOS says you can't TOS your way out of negligence. Not following the minimum enterprise standards for backups is negligent. Negligent behavior gets you sued, the deeper the pockets the more lawsuits it will generate. Consumer anger is going to be very high, signups to the class action suits will be very significant.

The Class action suits are likely being written up as I type this and will be filed the day they restore service so the lawsuits will have the correct number of days. T-mobile will settle and they will execute liablity terms in their contract with Danger, likely causing Danger to be responsible for everything but a small "deducible". This isn't speculative, it's practically guaranteed.

Heads should roll at Danger management for not following standards in enterprise IT management that the rest of the world uses. Backups are critical, expected and negligent if not provided.

tiger72
SexaT duorP
Premium
join:2001-03-28
Saint Louis, MO
kudos:1
Reviews:
·T-Mobile US
·AT&T DSL Service
said by r81984:

I am sorry, but lawsuit for losing data that no one guaranteed is pointless. This is a case that Tmobile knows it will win so if they settle out of court Tmobile will only pay what it would lose paying to win the court case to save time. Once divided up people will get like a $1 and lawyers will get most of the money.
Also it will be hard to claim damages over lost phone numbers and emails.
It doesn't matter if TMO wins the case if they take a massive PR hit to their much vaunted customer service. That's, really, the point of the class action suit. Not to win large damage awards, but to inflict PR damage to corporations...
--
"What makes us omniscient? Have we a record of omniscience? ...If we can't persuade nations with comparable values of the merit of our cause, we'd better reexamine our reasoning."
-United States Secretary of Defense (1961-1968) Robert S. McNamara

Steve
I know your IP address
Consultant
join:2001-03-10
Yorba Linda, CA
kudos:5
said by r81984:

With it comes to technology the minimum expected performance is you know to back up your data yourself. I thought everyone knew that.
There wasn't a way for users to back it up themselves
Kdee

join:2005-08-26
Etobicoke, ON
T-Mobile apparently kept telling users that they didn't have to backup their data because it was all kept securely in a data center.

I guess that worked out well.

N3OGH
Certified GLG-20
Premium
join:2003-11-11
Philly burbs
kudos:1
Reviews:
·Verizon FiOS
All aboard the fail boat!

ReformCRTC
Support Your Independent ISP

join:2004-03-07
Canada

Re: Fail

Epic Fail x100
moonpuppy

join:2000-08-21
Glen Burnie, MD

Looks like Microsoft has some explaining to do...

....or could this be a way to get people off of Sidekicks and into Windows Mobile phones.

Let the conspiracy theories fly.

en102
Canadian, eh?

join:2001-01-26
Valencia, CA

Re: Looks like Microsoft has some explaining to do...

Windows as a server OS is painful enough... do we really want to force WinMo handsets as well ?

I do use WinMo handsets - and it works, but has to be rebooted from time to time when it freezes up.

ReformCRTC
Support Your Independent ISP

join:2004-03-07
Canada

Re: Looks like Microsoft has some explaining to do...

BSOD on a phone. Now that's a useless fail.
AVonGauss
Premium,MVM
join:2007-11-01
Boynton Beach, FL

Clouds...

A nice cloudy day, then it rains, the clouds dissipate and then there are clear skies without a cloud in sight... My point, if there was crucial information of yours in the system that you did not have stored in an alternative location, who's really at fault here?

swintec
Premium,VIP
join:2003-12-19
Alfred, ME
kudos:3
Reviews:
·RapidVPS
·Sprint Mobile Br..
·VoicePulse
·RoadRunner Cable

All GONE!

My girlfriends sidekick lost all of her ring tones. She told me yesterday "they simply have not come back yet, since being reset"....I guess I will have to tell her the bad news, that they will never come back.

She did get a text message yesterday that she is getting a months worth of data credits to her account.

I have never given her phone much attention, but is everything stored off the device??? If so, why? If she takes the battery out of her phone, or the battery dies, it is many hours - several days before all of her settings and functions return..seems very stupid to me.
--
Block Accounts | UseNet Now

See 6 replies to this post

jake9234

@rcn.com

awful

something like this WOULD NOT HAPPEN on verizon or at&t. tmobile sucks which is why they are far behind all carriers.

anon82837

@198.111.16.x

Re: awful

said by jake9234 :

tmobile sucks which is why they are far behind all carriers.
You sir have committed a fallacy. A statement that makes absolutely no sense. T-mobile is a very big Euro based company that one day decided to start a network in the us. They recently have put a great deal of money to beef up their US resources to very soon rival ATT to the tune of a new 3g market every few days. If you read the article, they don't run the sidekick data store. Danger aka Microsoft does. I don't have t-mobile because their coverage is horrible by me (like att) but give em' a break, they're working on it and showing true promise.
moonpuppy

join:2000-08-21
Glen Burnie, MD
said by jake9234 :

something like this WOULD NOT HAPPEN on verizon or at&t. tmobile sucks which is why they are far behind all carriers.
More BS from the anonymous poster.

Guess you forgot the ATT debacle when they couldn't add, change or remove service to their mobile systems because their IT department did an untested upgrade that screwed everything up.


John McClane
yippee ki yay
Premium
join:2005-03-19
umm.. please do not post here anymore. what you said is complete BS. T-Mobile has nothing to do with the outage. they just happen to be the wireless carrier for voice and data service. they do not control how the devices worked.

Mang

@yahoo.com

End of contract...

So i guess this would be a good enough reason to be able to exit yuor long term contract because the carrier FAILED it's ToS of the contract.

1) A Week+ of no service
2) Loss of data
3) "Are you smarter then a 5th grader" level of data backup schedule in place means they were negligent with customers property.

I'm sure a class action suit can get the attorney's very rich on this slam dunk case as well as get people their money back for the time+money they spent on the service since the contract began.

I've never liked T-Mobile, tried them for one year in 2005 and i realized while they were cheaper then AT&T... i was just getting what i paid for (and often times less)

See 6 replies to this post

NO to ESPN

@sbcglobal.net

Maybe It is Part of the Plan

Danger transition to Mobile Explorer version 48.9. (Maybe by then it will be decent.)

MovieLover76

join:2009-09-11
Reviews:
·Verizon FiOS
·DIRECTV
·Optimum Online
·Cablevision

This is why backup should be done yourself

I've never been a fan of online backup services, and have always backed up my own data personnally, trusted your data to microsoft run servers seems doomed to failure at some point lol. No one is going to care for your data as well as you do yourself, it's really that simple.

Steve
I know your IP address
Consultant
join:2001-03-10
Yorba Linda, CA
kudos:5

Re: This is why backup should be done yourself

Not if there's no way to back up this particular data yourself
sedu

join:2002-01-10
Freeland, MD

Danger Will Robinson, Danger

What more can you say.

jaynap

@198.91.4.x

Can we say Mobile Make over

This gives a how new meaning to the latest ad campaign.

Maggs
Premium
join:2002-11-29
Woodside, NY

It's Microsoft

Give them something good, and they'll fuck it up. Hell they had UNIX programmers code Windows to appease to morons.

Gbcue
Almost P.E.
Premium
join:2001-09-30
Santa Rosa, CA
kudos:8

Android Users

What's the best way to backup an android device?

Does Google back everything up?

sapo
Cruising Down Memory Lane
Premium
join:2002-09-16
Sacramento, CA
kudos:1

Re: Android Users

Android interfaces with Google directly for Gmail, Calendar, and Contacts. For the G1 specifically a microsd card is required and all stuff like photos, music, etc are kept on there. That is all.
--
DO DO

Gbcue
Almost P.E.
Premium
join:2001-09-30
Santa Rosa, CA
kudos:8
Reviews:
·AT&T U-Verse

Re: Android Users

said by sapo:

Android interfaces with Google directly for Gmail, Calendar, and Contacts. For the G1 specifically a microsd card is required and all stuff like photos, music, etc are kept on there. That is all.
I know, but how can we be sure Google is backing all the contacts, gmail, the calendar?
--
My BLOG!
Black Friday Ads

John McClane
yippee ki yay
Premium
join:2005-03-19
Reviews:
·Comcast
·T-Mobile US
·AT&T U-Verse
·Vonage

Re: Android Users

There are apps to backup all of your email/sms/mms/contacts/etc to your sd card.

i'm not worried about it. if gmail goes down I at least have a SLA to fall back on. I have Google Apps through my College and thats what I use on my phone to login.

toddbs98

join:2000-07-08
North Little Rock, AR

What do you expect..

This story doesn't surprise me at all since it took 13 calls totaling 6 hours on the phone over a period of 2 months for T-Mobile to get my voice mail to work. If however the information on your sidekick was that vital to users they should have been responsible enough to back it up.
--
Patriots always speak of dying for their country never killing for it. Bertrand Russell
gadgetboyj

join:2009-08-25
Staten Island, NY
kudos:2

Fuck

Damn it I lost everything. All my contacts, all my texts, all my pictures, all my... everything. Its gone and I was never offered a free month of service. WTF is this? And to the person who said it was stupid to trust Microsoft to our stuff, Microsoft only very recently acquired Danger who ran the servers before that. This only makes me want to find the people over there that come up with these stupid ideas and put them to an entire life of torture. Everything on my phone is GONE.

John McClane
yippee ki yay
Premium
join:2005-03-19

Re: Fuck

pro tip: its just a phone
old_wiz_60

join:2005-06-03
Bedford, MA

Glad I don't have T-Mobile!

Wonder what happens to t-mobile stock after this?

John McClane
yippee ki yay
Premium
join:2005-03-19

Re: Glad I don't have T-Mobile!

t-mobile had nothing to do with this.
bilbusb

join:2003-04-10
Tucker, GA

.

lol all these comments about raid 1.

glinc: lol this had to be a SAN, and no san is raid1 ... we are talking many many terabytes here.

bjbrock: raid 1 with 2 controllers .. lol what? Is that a joke?

God, looks like one of their many san failed .. and they did not have a proper backup strategy.

There is no way a server could handle that volume of data.

Also there is no indication that it has anything to do with the servers ... so sure it's MS's fault .. but necessarily due to their software. Sounds like hardware to me.

As for backing up huge amounts of data, they have san replication and robotic tape libraries for that.
antidelldude

join:2003-12-22
Rochester, MI
Reviews:
·Comcast

Not Surprising

Seriously guys? You find this surprising? T-mobiles systems have been breached by hackers more than once. Sidekick's data have been hacked more than once. If you didn't have at-leased your contacts backed up somewhere, you're an idiot. Lesson Learned eh?

John McClane
yippee ki yay
Premium
join:2005-03-19

Re: Not Surprising

wow another one.

t-mobile had nothing to do with this.
antidelldude

join:2003-12-22
Rochester, MI
Reviews:
·Comcast

Re: Not Surprising

Oh noes, It's not T-mobiles fault. Oh wait, who hired what once was Danger and now is Microsoft to handle their Sidekicks customers data? Who didn't care to oversee any operations or put forth any rules to protect consumer data? Umhmm. There is enough blame to spread around.

John McClane
yippee ki yay
Premium
join:2005-03-19

Re: Not Surprising

are you kidding me? they don't tell the ppl that are in charge of the sidekick backend how to do their job. they have a wireless network to run. that would be something that AT&T would try to do.

MemphisPCGuy
Senior Systems Engineer
Premium
join:2004-05-09
Memphis, TN
Reviews:
·Comcast

2 edits

I don't want my data backed up

Why didn't they just spin "privacy" and give rebate/gift. I wouldn't want my information backed up and hope I could opt out if it was.

Edit: for context, I'm on Verizon. They back it up a few times I'm sure.
--
Onsite Computer Support in Memphis
»www.memphispcguy.com
dsless

join:2001-05-16
Pittsburgh, PA

Big Iron

Should have purchased a Mainframe! Microsoft and data center sounds like a contradiction in terms.

burgerwars

join:2004-09-11
Northridge, CA

Dogfooding or Sabotage?

Link to story: »www.appleinsider.com/articles/09···age.html

Saturday, 11-Feb 15:49:52 Terms of Use & Privacy | feedback | contact | Hosting by nac.net - DSL,Hosting & Co-lo
over 12.5 years online! © 1999-2012 dslreports.com.