dslreports logo
 story category
Google Suffers 'Embarrasing' Outage
Though it only impacted 14% of users, company says...
Some of you might have noticed that Google today suffered through some outages impacting not only GMail, but Google news. While some might credit this to the coming bandwidth apocalypse, Google posted a brief explanation to their blog stating that the outage only impacted about 14% of users, while redefining vague with their explanation. "An error in one of our systems caused us to direct some of our web traffic through Asia, which created a traffic jam," says Google SVP of Operations, Urs Hoelzle. "We've been working hard to make our services ultrafast and "always on," so it's especially embarrassing when a glitch like this one happens," Hoelzle says.
view:
topics flat nest 
page: 1 · 2 · next

POB
Res Firma Mitescere Nescit
Premium Member
join:2003-02-13
Stepford, CA

POB

Premium Member

The Sky Has Fallen...

While some might credit this to the coming bandwidth apocalypse...
Yeah, the outage was all because of the ::LOOMING:: bandwidth apocalypse. The only people who will swallow that are clueless whores representatives and the corporate lobbyists who own them.

jwalk42
jwalking
join:2008-02-04
Old Orchard Beach, ME

jwalk42

Member

Re: The Sky Has Fallen...

agreed.....and...the bandwidth apocalypse makes me smile.

S_engineer
Premium Member
join:2007-05-16
Chicago, IL

S_engineer

Premium Member

Re: The Sky Has Fallen...

said by jwalk42:

agreed.....and...the bandwidth apocalypse makes me smile.
if only we had listened to Craig Moffet...

Eagles1221
join:2009-04-29
Vincentown, NJ

Eagles1221 to POB

Member

to POB
Google was out for me yesterday and HP was out today.

Well, Google worked over my VZ DSL but not my MetroE. BGP issues?
iansltx
join:2007-02-19
Austin, TX

iansltx

Member

Re: The Sky Has Fallen...

Appeared to be. Qwest connections seemed to be hit pretty hard.
phills_suck
join:2004-10-11
Burlington, NJ

phills_suck

Member

Me!

I was one of those 14%.

IowaMan
Premium Member
join:2008-08-21
Grinnell, IA

IowaMan

Premium Member

Re: Me!

Me Too Here is where the traffic died
Tracing route to www.l.google.com [72.14.213.99]
over a maximum of 30 hops:

1 20 ms 7 ms 9 ms 12-215-25-254.client.mchsi.com [12.215.25.254]
2 15 ms 12 ms 11 ms 12-215-4-22.client.mchsi.com [12.215.4.22]
3 63 ms 60 ms 64 ms cr2.cgcil.ip.att.net [12.122.132.98]
4 75 ms 59 ms 63 ms cr2.st6wa.ip.att.net [12.122.31.130]
5 79 ms 77 ms 75 ms 12.122.146.157
6 * * * Request timed out.
7 * * * Request timed out.
8 274 ms * * 216.239.46.208
9 * * * Request timed out.
10 * * * Request timed out.
11 282 ms 283 ms * pv-in-f99.google.com [72.14.213.99]
12 * 381 ms * pv-in-f99.google.com [72.14.213.99]
13 381 ms * 268 ms pv-in-f99.google.com [72.14.213.99]

Trace complete.

Parogadi
What? Stop Looking At Me Like That
Premium Member
join:2003-03-31
Racine, WI

Parogadi

Premium Member

Re: Me!

Likewise, I had done a 100 ping to google test and got 66% packet loss, though the ping was within ms spec of normal.

Tzale
Proud Libertarian Conservative
Premium Member
join:2004-01-06
NYC Metro

Tzale to phills_suck

Premium Member

to phills_suck
said by phills_suck:

I was one of those 14%.
Me too.

cork1958
Cork
Premium Member
join:2000-02-26

cork1958 to phills_suck

Premium Member

to phills_suck
said by phills_suck:

I was one of those 14%.
Me too.

Personally,
I think Google stinks anyway. Is no big thing when Google goes down for me.
RolteC
The Need for Speed
join:2001-05-20
New York, NY

RolteC

Member

No wonder why I couldn't access Google / Gmail

hmm I see now why.
ctggzg
Premium Member
join:2005-02-11
USA

ctggzg

Premium Member

Re: No wonder why I couldn't access Google / Gmail

Thanks for the insight.

cchhat01
Dr. Zoidberg
join:2001-05-01
Elmhurst, NY

cchhat01

Member

Re: No wonder why I couldn't access Google / Gmail

said by ctggzg:

Thanks for the insight.
I cracked up here! Seriously you guys make me laugh!

v35_pilot
Whoops, there goes another AMU
Premium Member
join:2005-12-12
Fayetteville, NY

v35_pilot to RolteC

Premium Member

to RolteC
said by RolteC:

hmm I see now why.
Yoda uses Gmail and posts to DSLR. Cool.

Uncle Paul
join:2003-02-04
USA

Uncle Paul

Member

I wonder

I wonder if China enjoyed what they pulled off the wire today...

Matt3
All noise, no signal.
Premium Member
join:2003-07-20
Jamestown, NC

Matt3

Premium Member

Re: I wonder

said by Uncle Paul:

I wonder if China enjoyed what they pulled off the wire today...
Wow, what a racist comment. Is China all of Asia now?

The effected users were being routed through Japan. See this post: »Re: YouTube Down?

Sir Meowmix III
@205.255.240.x

Sir Meowmix III

Anon

Re: I wonder

No, just the biggest land-mass there...

morbo
Complete Your Transaction
join:2002-01-22
00000

1 edit

morbo to Matt3

Member

to Matt3
said by Matt3 See ProfileWow, what a racist comment. Is China all of Asia now?


no, but it is one of two acceptable answers based on population density in Asia: China and India. the Maldives would have been an unacceptable answer.

POB
Res Firma Mitescere Nescit
Premium Member
join:2003-02-13
Stepford, CA

POB

Premium Member

Re: I wonder

said by morbo:

no, but it is one of two acceptable answers based on population density in Asia: China and India. the Maldives would have been an unacceptable answer.
LMAO - Like anyone else here even know what the Maldives are.

zalternate
join:2007-02-22
freedom land

zalternate to Matt3

Member

to Matt3
Remember when Japan was called the Orient?

Matt3
All noise, no signal.
Premium Member
join:2003-07-20
Jamestown, NC

Matt3

Premium Member

More than one outage

They had more than one outage then. Some folks were being directed through Asia but me and a ton of others had Level 3 routing issues in Atlanta and Washington at the peering points.

»YouTube Down?

Karl Bode
News Guy
join:2000-03-02

Karl Bode

News Guy

Re: More than one outage

For about three hours, Google News wasn't available from anywhere as far as I can tell, so yeah...

FFH5
Premium Member
join:2002-03-03
Tavistock NJ

FFH5 to Matt3

Premium Member

to Matt3

Cause was IPV6 conversion causing network routing problems

»news.cnet.com/8301-1023_ ··· 1_3-0-20
Google this morning attempted to make changes to key Internet routing numbers--known as autonomous system numbers--as part of its ongoing transition from an older networking standard to a newer one called IPv6. An unknown "bug" inside Google's network involving some sort of hardware failure or glitch prevented Internet service providers from finding Google's new ASNs on the Internet--effectively sealing it off from many customers, he said.

AT&T or Verizon--appeared to be disproportionately hurt because large ISPs "peer" with Google, or interconnect their networks with Google's networks in order to improve speed and reduce bandwith costs, Alperovitch said.
They can use that old saying: "Temporary disruption; but permanent improvements".

Logan 5
What a long strange trip its been
Premium Member
join:2001-05-25
San Francisco, CA

Logan 5

Premium Member

Apocalypse "now"?

It was the RIAA, Global Warming and Mark Cuban all working in concert to punish those evil "Googlealians"....
firewire9999
join:2004-07-11
Livonia, MI

firewire9999

Member

Re: Apocalypse "now"?

Hmmmm you forgot the following to blame:

George W. Bush
Carl Rove
Dick Cheney
Republican Party (in general)

Logan 5
What a long strange trip its been
Premium Member
join:2001-05-25
San Francisco, CA

Logan 5

Premium Member

Wait... can't forget the Illuminati as well as the cast of the smash Broadway play "GitMo on Ice" either....

Any others we can blame for this?

Doctor Four
My other vehicle is a TARDIS
Premium Member
join:2000-09-05
Dallas, TX

Doctor Four

Premium Member

Re: Apocalypse "now"?

said by Logan 5:

Wait... can't forget the Illuminati as well as the cast of the smash Broadway play "GitMo on Ice" either....

Any others we can blame for this?

What about Rupert Murdoch?

C0deZer0
Oc'D To Rhythm And Police
Premium Member
join:2001-10-03
Tempe, AZ

C0deZer0 to Logan 5

Premium Member

to Logan 5
said by Logan 5:

Any others we can blame for this?
Yes.

Blame Canada.

»www.youtube.com/watch?v= ··· YipouABI

HarleyDude
join:2002-01-04
Kennett Square, PA

HarleyDude

Member

Re: Apocalypse "now"?

I don't care who you are, THATS FUNNY!

iam x
Sungazer
Premium Member
join:2005-02-23

iam x to Logan 5

Premium Member

to Logan 5
said by Logan 5:

Any others we can blame for this?

Sure, the Rothschild's.
They are responsible for everything.
patcat88
join:2002-04-05
Jamaica, NY

patcat88 to Logan 5

Member

to Logan 5
said by Logan 5:

Wait... can't forget the Illuminati as well as the cast of the smash Broadway play "GitMo on Ice" either....

Any others we can blame for this?

Forgot about the Freemasons.
staticfive
join:2006-08-01
Santa Rosa, CA

staticfive

Member

Re: Apocalypse "now"?

Wow, did I miss something?? When did BBR become a soap box for right-wing pundits?

HarleyDude
join:2002-01-04
Kennett Square, PA

HarleyDude

Member

You all got it wrong

I has to be Global Warming..
zenafu
join:2007-06-12
Brooklyn, NY

zenafu

Member

Mo Knows All

"You know what I blame this on the breakdown of? Society."

-Mo Syzlak
ke4pym
Premium Member
join:2004-07-24
Charlotte, NC

ke4pym

Premium Member

Never down ... HAH!

You know, Google Co. sure has had their share of outages over the past 6-8 months.

Seems like it started when one of their GSA reps was in our building touting how we should dump our in house Exchange platform (and Office) for their always on, always available system (yes, yes, I know, not quite the same thing ... but still..).

insomniac84
join:2002-01-03
Schererville, IN

insomniac84

Member

What outage?

No problems at all in KC.
Stojko
Premium Member
join:2007-10-20
St John's NL

Stojko

Premium Member

Re: What outage?

said by insomniac84:

No problems at all in KC.
No problems here either.

rob_in_chatt
Premium Member
join:2004-09-17
Chattanooga, TN

rob_in_chatt

Premium Member

what outage?

no problems here in Chattanooga TN with google or GMail. it has been up and running all day as far as i can tell.

AnonDOG
@kaballero.com

AnonDOG

Anon

No part of this is vague...

"An error in one of our systems caused us to direct some of our web traffic through Asia, which created a traffic jam," says Google SVP or Operations, Urs Hoelzle.

No part of that quote is vague, unless you wanted IP routes and addresses. It says that they screwed up a route and sent some (probably a lot) of web traffic through Asia (during the day time in the Western world) and that choked a pipe in Asia...

So, I dunno, no part of that is vague to me... but what do I know, I'm not a journalist.
Maze
join:2006-02-02
USA, NYC

1 edit

Maze

Member

Include NYC

Me and my friend had no gmail here in Brooklyn / Manhattan. Both of us though were routed to a temporary unavailable errors and what to look for. But that did not fix the issue as google was having it. Like to add google.com was not resolving instantly but reported by my friend.

NY Tel
Premium Member
join:2004-04-09
Smithtown, NY

NY Tel

Premium Member

Google Suffers 'Embarrasing' Outage

I couldn't get to it from my work pc but was able to send and receive GMAIL from my Blackberry.

CharterandATT
@charter.com

CharterandATT

Anon

Some up some down

Google was down earlier for my AT&T DSL, so I switched to my Charter connection and it worked fine, near Chattanooga, TN.
ShellMMG
join:2009-04-16
Grass Lake, MI

ShellMMG

Member

Talk about bad timing...

I've been having a signal war with Alltel/Verizon/Sprint (Alltel can't use Sprint towers to roam anymore...and I was affected), I'd just installed a Cradlepoint router and was working out the kinks...and Google goes down. I was tearing my hair out, thinking I'd screwed up yet ANOTHER setting, and it was all Google.

There ain't enough aspirin on the planet to erase THAT headache.

Michail
Premium Member
join:2000-08-02
Boynton Beach, FL

Michail

Premium Member

Saw it too

I was one of the 14% also.

Usually if google is down I just assume the entire Internet is. But no, OMG, MSN was there!

AnonDOG
@kaballero.net

AnonDOG

Anon

Re: Saw it too

said by Michail:

I was one of the 14% also.

Usually if google is down I just assume the entire Internet is. But no, OMG, MSN was there!
Ayup, that sucks when it happens. Thankfully, Microsoft is gradually loosing ground to better programmers, better systems, and better software.

It will take time, but eventually they will be relegated to a place in the market which they deserve and have earned on their merits. It will always be a significant place, but it will not be the dominant place.
FrontirCynic
join:2006-10-25
Long Beach, CA

FrontirCynic

Member

nostradamus sez....

Google was down totally for fios here in socal from 730 am to about 10am

look everyone claims 99.99% uptime. they dont like to say .o1% downtime. But you gotta squeeze that in somewhere. If humans touch it there is no such thing as 100% perfect.

Google is free and makes Yahoo and MS look like 1997 internet. (the pre-cambrian period)

We just need to have redundancy and backups built in. That is why I have two other email providers other than google. Just like verizon or earthlink or anyone else, they do tend to go down.

What is funny is when these "hiccups" (f-ups) come along now in any system (like oh say the financial system) we are all on their ass in 30 seconds with email, bloggs and now tweets. As soon as I saw google was down I said oh man will they have hell to pay today. Someone will face a firing squad.

plain old me
@comcast.net

plain old me

Anon

outsourcing

Rerouted over Asia - coincidence? Maybe, maybe not?
It seems that's what you get from outsourcing to Asia.
You get people writing code, whose only code writing experience is that of copying their homework assignments from the guy next to them in school.
Somebody probably ended up hard-coding Asian routing information into some piece of software.
And since QA is either outsourced too or just skipped entirely these days, no one catches the mistake until after implementation...
ditka_b
Premium Member
join:2001-10-05
Barrington, IL

ditka_b

Premium Member

Google

Didn't pay their bill lol

Hpower
join:2000-06-08
Canyon Country, CA

Hpower

Member

Re: Google

lol that's what I thought too. "WTH SEND THE BILL ASAP!!!"
page: 1 · 2 · next