Fire at the Planet in Webhosting

Fire at the Planet in Webhosting http://www.dslreports.com/forum/r20570844 en Fri, 27 Nov 2009 09:18:08 EDT Fri, 27 Nov 2009 09:18:08 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20587537 ff1324 :

said by pokesph

:

I just can't believe they weren't allowed to run the backup power solutions..
Now, why bother having one if its a) not a secondary system, with redundant wiring/sysyems b) not usable (in this case not used)

Because electrocuting firefighters is usually considered a bad thing. One of the first things we do when a structure is on fire is cut the power to the building. This includes accounting for ALL secondary power supplies. Fire tends to melt the insulation off of wires. Touching those wires (if they haven't already shorted causing additional damage to structure and electrical equipment) has killed or injured firefighters. Plus, it's a really cool sensation when you touch a pool of water on the floor and it makes your whole arm tingle.

yeah, I'm a little biased on the subject, though.
--
What do you want to do to the world, Ronald?
Burn it all.
See you next year, Ronald.]]> http://www.dslreports.com/forum/remark,20587537 Tue, 03 Jun 2008 23:49:52 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20581153 jazzman916 : Update page:
»service-update.theplanet.com/

Edit: Direct link to The Plant CEO's audio message:
»service-update.theplanet.com/Erw···2-08.wav

]]> http://www.dslreports.com/forum/remark,20581153 Mon, 02 Jun 2008 23:18:22 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20579467 jazzman916 : Another interesting story about The Planet's incident:
»www.gawkwire.com/web_hosting/pla···ion.html]]> http://www.dslreports.com/forum/remark,20579467 Mon, 02 Jun 2008 18:08:07 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20576577 CHPMustang : Our sites are ran thru the Dallas DataCenter so we were unaffected fortunately :)

Blame the H1 going down on the rasberry ants :D
--
Got XM?]]> http://www.dslreports.com/forum/remark,20576577 Mon, 02 Jun 2008 09:08:29 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20576181 leevis : WOOT! My server just came back online! Earlier this evening I was able to determine that mine was on the second floor so I was hopeful it would be back up soon. I just feel bad for anyone on the first floor as they now say it will be late tomorrow before those power on.

I was able to remote access it and everything is running fine. In the "Unscheduled shutdown" event box, for "reason" I put: "You wouldn't believe me if I told you."

said by jasonc :

some other forum posts are claiming it as a cover up...

It's exactly what they said it was, nothing more, nothing less (and I'd tell you what really happened if that black helicopter wasn't circling above my house :D )

Lee
--
"I've got a Caribbean soul I can barely control..." -JB]]> http://www.dslreports.com/forum/remark,20576181 Mon, 02 Jun 2008 05:46:59 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20575531 joako : What happened to your business contingency plan? You sell wigets what do you do when the wiget makers go on strike? You sell on website what happen when there is no site? At the very least you can have a backup MX so no messages are lost. Keep a cheap hosting account elsewhere and you could have changed the DNS last night and be taking customers.
--
09:F9:11:02:9D:74:E3:5B:D8:41:56:C5:63:56:88:C0]]> http://www.dslreports.com/forum/remark,20575531 Sun, 01 Jun 2008 23:49:55 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20574479 anon : some other forum posts are claiming it as a cover up...
if it is true why no photos?]]> http://www.dslreports.com/forum/remark,20574479 Sun, 01 Jun 2008 20:05:03 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20574271 anon : Hummmm, I think I discovered the cause of the Planet Server Provider of Thousands of Websites with the Transformer OVERHEATING Disaster!!!
This was posted a few months ago,
»www.thehostingnews.com/news-dedi···306.html it claims "Mr. Lowenberg conducted a six-month trial to reduce power consumption and increase data center operating efficiency. Initial results demonstrate that while critical server loads increased by 5 percent, power used for cooling decreased by 31 percent. Overall, the company experienced power reductions of up to 13.5 percent through a broad range of improvements. The new green initiatives were conducted across its six world-class data centers." and also The Planet operates more than 150 30-ton computer room air conditioning CRAC units across its six data centers. In one data center alone, the company was able to turn off four of the units. The cooling requirement on two of the units was reduced to 50 percent of capacity, while another nine now operate at 25 percent of capacity. The company also extended the return air plenums on all of its down-flow CRAC units to optimize efficiency. "

Lets turn the Air Conditioning DOWN to HEAT things up so they Explode!!! You don't need to be Sherlock Holmes to figure this one out.]]> http://www.dslreports.com/forum/remark,20574271 Sun, 01 Jun 2008 19:20:28 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20573758 anon : I run a free email website w/ +/- 20k signups & a hosting service for French Canadaians w/ 30 sites hosted.

Imagine the email volume being bounced back!
Imagine lost signups and lost customers!

Much business lost for me and much much business and personal email lost to my customers.

Not to mention lost revenue.....

Alexandre Cloutier]]> http://www.dslreports.com/forum/remark,20573758 Sun, 01 Jun 2008 17:26:56 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20573660 anon : I've had a dedicated server with them (Well, RackShack, then EV1) since '01 and this is the first time my server has gone down except for when I've had it down for scheduled maintenance.
Thinking about the problems with some of their management and DNS systems "single homed" at H1, I wonder how many server farms EV1 had and if The Planet had not yet set up redundant systems in their other farms. Just an idea.]]> http://www.dslreports.com/forum/remark,20573660 Sun, 01 Jun 2008 17:08:12 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20573244 JoelC707 : It really all depends on how the power system was designed. I'm sure there could have been ways to design the system so that the backups were independent from the mains but there is always still a risk and the fire marshal has the final and ultimate say (they can and will escort you out and padlock your building if you defy them). Generally the generators are wired in almost in parallel with the main feed from the grid. This allows them to have one set of power equipment and more space for the money making equipment.]]> http://www.dslreports.com/forum/remark,20573244 Sun, 01 Jun 2008 15:10:11 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20573096 leevis :

said by pokesph

:

while I'm not impacted ...

Why did I have to hear about this from DSLR and not from The Planet directly?

I was impacted, and I did hear about it directly from The Planet. With something of this magnitude (+/-7500 customers' servers down), I can understand why they only corresponded with those directly effected.

Lee
--
"I've got a Caribbean soul I can barely control..." -JB]]> http://www.dslreports.com/forum/remark,20573096 Sun, 01 Jun 2008 14:23:01 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20572823 anon :

said by texans20

said by hes :

Though it is an accident and the magnitude and special characteristics of the explosion cant be anticipated, dont this guys need to have a mirror, a 2nd location or something like that? They even had DNS and managment services runing there with no data redundancy? What can you say about a specialized company that has this kind of failure?

I realy hope this will make many of the clients run to a better provider and impact them economicaly so they will take measures for anticipating this in the future. They advertise that now right?

Sure everyone in the computer industry knows that, but it seems the guys at the planet don't, many of their basic tools and services were located in that facility, once down, services down. If you check their posts on the accident you'll see they didn't have those services runing on 2 data centers, even though they own them. They had to physically move the equipment to another data center, install it and have them run in this new location. Nice right?

]]> http://www.dslreports.com/forum/remark,20572823 Sun, 01 Jun 2008 13:10:46 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20572819 joako : I think what the last poster was getting at is why didn't ThePlanet implement those practices? Why did they have, in their own words, "single homed" DNS servers? Why weren't there redundant copies of management system that went down?
--
09:F9:11:02:9D:74:E3:5B:D8:41:56:C5:63:56:88:C0]]> http://www.dslreports.com/forum/remark,20572819 Sun, 01 Jun 2008 13:10:11 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20572765 texans20 :

The Planet has multiple data centers, only one is down. Anyone in the computer industry knows never put all your eggs in one basket. If you absolutely can not be down, then pay extra to host servers in one or two other data centers. All the redundant systems in the world aren't going to help you if there is a problem with the building.
--
"I sincerely believe the banking institutions having the issuing power of money are more dangerous to liberty than standing armies." Thomas Jefferson]]> http://www.dslreports.com/forum/remark,20572765 Sun, 01 Jun 2008 12:55:55 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20572651 anon : Though it is an accident and the magnitude and special characteristics of the explosion cant be anticipated, dont this guys need to have a mirror, a 2nd location or something like that? They even had DNS and managment services runing there with no data redundancy? What can you say about a specialized company that has this kind of failure?

I realy hope this will make many of the clients run to a better provider and impact them economicaly so they will take measures for anticipating this in the future. They advertise that now right?]]> http://www.dslreports.com/forum/remark,20572651 Sun, 01 Jun 2008 12:25:29 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20572619 joako : Well there was an electrical fire/explosion so that's a safety precaution from the fire marshall.

I don't know exactly how their setup is in that datacenter but it sounds as if they don't have a fully redundant power setup.
--
09:F9:11:02:9D:74:E3:5B:D8:41:56:C5:63:56:88:C0]]> http://www.dslreports.com/forum/remark,20572619 Sun, 01 Jun 2008 12:16:14 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20572494 pokesph : WOW this is crazy.. while I'm not impacted, I am a bit peaved.
I also use The Planet (out of the Dalltx2 DC) so after reading all the posts on the planet's forum, I just can't believe they weren't allowed to run the backup power solutions..
Now, why bother having one if its a) not a secondary system, with redundant wiring/sysyems b) not usable (in this case not used)

Why did I have to hear about this from DSLR and not from The Planet directly?

I hope they readdress their backup and disaster recovery plans after this.
--
Webmaster - Steve
- - - - - - - - - - - -
»www.1-gb.net
»www.ppnstudio.com]]> http://www.dslreports.com/forum/remark,20572494 Sun, 01 Jun 2008 11:41:40 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20571922 elnino :

said by HardwareGeek

:

Anyone else feeling the pinch from this?

Yep, I'm feeling it too. My church's website is hosted out of that Datacenter. What stinks is that online signups for our church directory were supposed to start today so EVERYONE is going to be trying to get in. I'm hoping things will be up today]]> http://www.dslreports.com/forum/remark,20571922 Sun, 01 Jun 2008 07:39:20 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20571680 HardwareGeek : I've only been a customer for 3 months.

I'm not too worried, since they said the servers aren't damaged.]]> http://www.dslreports.com/forum/remark,20571680 Sun, 01 Jun 2008 03:33:31 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20571644 leevis :

said by HardwareGeek

:

Anyone else feeling the pinch from this?

Yep, my server is currently off-line. I'm a customer from the old "Rackshack" days, so naturally mine is in that datacenter. What's really crazy is that just one week ago, my server went off-line because the power-supply burned out. That was the first failure in two years.

Lee
--
"I've got a Caribbean soul I can barely control..." -JB]]> http://www.dslreports.com/forum/remark,20571644 Sun, 01 Jun 2008 03:16:00 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20571559 jazzman916 : Interesting discussions in their customer forums:

»forums.theplanet.com/index.php?s···ry592415
»forums.theplanet.com/index.php?s···ic=90191
--
"Hoaxes use weaknesses in human behaviour to ensure they are replicated and distributed. In other words, hoaxes prey on the Human Operating System." --Stewart Kirkpatrick]]> http://www.dslreports.com/forum/remark,20571559 Sun, 01 Jun 2008 02:19:43 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20571553 TearAbite : hehe.. i just got this.. good thing nobody uses EntreCard:

said by EntreCard :
From: Entrecard Team
Sent: Saturday, May 31, 2008 11:06 PM
To: Me
Subject: Entrecard Outage / Electrical fire

Greetings,

Unfortunately, there has been an electrical fire at the data center housing Entrecard's servers. The servers have not been damaged, but the site may be down for as long as 12 hours.

Entrecard is hosted with ThePlanet, and this fire has effected over 9000 websites, as many of the world's most popualar websites are hosted with ThePlanet. They are working around the clock to get their data center back online, and we are hoping that they get it back online fast.

Thank you for your pateince and understanding with this rare and frustrating problem. Unfortunately, there is absolutely nothing we can do but wait for The Planet to get their datacenter back online.

Please check back with Entrecard in 12 to 24 hours.

Thanks,
Entrecard Team

]]> http://www.dslreports.com/forum/remark,20571553 Sun, 01 Jun 2008 02:15:31 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20571345 removed : Looks like everyone seems to be having power issues this month. We did earlier, and now it's The Planet's time as well.

A friend of mine has a box in that datacenter with 300 customers on it. Hopefully they'll get things going again, though it doesn't sound like it'll happen anytime soon given the damage...
--
irc.removed.us - #dslr | DSLR Phishtracker | Email: removed@dslr.net]]> http://www.dslreports.com/forum/remark,20571345 Sun, 01 Jun 2008 00:47:39 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20571324 jazzman916 :

said by The Planet CEO Doug Erwin :

This evening at 4:55pm CDT in our H1 data center, electrical gear shorted, creating an explosion and fire that knocked down three walls surrounding our electrical equipment room. Thankfully, no one was injured. In addition, no customer servers were damaged or lost.

We have just been allowed into the building to physically inspect the damage. Early indications are that the short was in a high-volume wire conduit. We were not allowed to activate our backup generator plan based on instructions from the fire department.

This is a significant outage, impacting approximately 9,000 servers and 7,500 customers. All members of our support team are in, and all vendors who supply us with data center equipment are on site. Our initial assessment, although early, points to being able to have some service restored by mid-afternoon on Sunday. Rest assured we are working around the clock.

We are in the process of communicating with all affected customers. we are planning to post updates every hour via our forum and in our customer portal. Our interactive voice response system is updating customers as well.

There is no impact in any of our other five data centers.

I am sorry that this accident has occurred and I apologize for the impact.

.
--
"Hoaxes use weaknesses in human behaviour to ensure they are replicated and distributed. In other words, hoaxes prey on the Human Operating System." --Stewart Kirkpatrick]]> http://www.dslreports.com/forum/remark,20571324 Sun, 01 Jun 2008 00:37:13 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20571299 HardwareGeek : Apparently there was an explosion that knocked down three walls.
--
Email/MSN: Michael at hardwaregeeks.com AIM: MikeR35292]]> http://www.dslreports.com/forum/remark,20571299 Sun, 01 Jun 2008 00:27:48 EDT Re: Fire at the Planet http://www.dslreports.com/forum/remark,20571282 ashs2dst : I am feeling it.

one customer on the box with 6 different domains.. all down.. hoping they get it back up tommorrow... Otherwise tommorrow night my phone will ring uncontrollably. ]]> http://www.dslreports.com/forum/remark,20571282 Sun, 01 Jun 2008 00:22:07 EDT Fire at the Planet http://www.dslreports.com/forum/remark,20570844 HardwareGeek : Appears they had a fire at the planet, no servers or anything damaged except for the transformer that caught fire.

Anyone else feeling the pinch from this?
--
Email/MSN: Michael at hardwaregeeks.com AIM: MikeR35292]]> http://www.dslreports.com/forum/remark,20570844 Sat, 31 May 2008 21:57:14 EDT