milnoc
join:2001-03-05
H3B
 ·TekSavvy Solutions..
| Blade servers
All right! Uber Geek Alert! In other words, I don't know where else to ask this question. 
As some of you know, I'm working on launching a cable television channel later this year. The automated broadcast system for this channel will be entirely computer based, and the distribution will be for the most part IP based. The performance requirements for the system will not be too high, but I'll still need quite a bit of equipment, and at least 10 terabytes of hard drive space for the video files.
To make life easier, I'm considering getting a blade server package which will be installed at a datacenter. I figured I can squeeze everything I'll need for the next two years in a single IBM Bladecenter S chassis. Throw in a keyboard/monitor drawer, and I can fit everything in a 1/4 rack with 2U to spare.
The question is: are IBM Bladecenters reliable and cost effective these days? I'm having a lot of trouble finding ANY reviews on the product, including credible complaints!
I'm also checking out the offerings by HP and Dell, but haven't figured out if everything I'll need for the next two years will fit in that same 1/4 rack. If I have to upgrade to a 1/2 rack and extra power, the monthly co-location costs will double, possibly negating any potential cost savings achieved by going with a cheaper equipment provider (the equipment will be leased, not purchased).
So any opinions out there? Has anyone here worked with IBM Bladecenters or know someone who has worked with them? Was the experience on the whole positive or negative? And how was the cost, set-up and long term maintenance? Good or bad?
Thanks! |
|
twizlar
I dont think so.
Premium
join:2003-12-24
Brantford, ON
| Beyond the cost and expandability issues down the road, the datacenters that WILL let you run that high of power density will be well out of your price range for a 1/4 rack. Fully loaded you are talking a power density of ~120-130 amps(usable) per rack, which isn't going to happen in most datacenters.
You would be better off to start small with a basic storage server with a couple front end units and upgrade as its needed from there. As far as IBM goes, they make good equipment, personally I've always found their stuff overpriced compared to most of the other major players like HP, Dell, Supermicro etc.
--
Broadline Networks Inc. |
|
HeadSpinning
join:2005-05-29
Windsor, ON
1 edit | reply to milnoc
When we built our data centre, we designed for around 5kw average dissipation per cabinet, or around 300 watts per square foot. We will let customers go over that power density, but will charge accordingly for power (and implictily, cooling).
You'll find that some data centres might not allow you to put higher power density in, so the money spent on blade servers might not be justified.
Find out the costs of the equipment, the savings in space, and the cost of power at the data centre you select, and do the math before you make a decision. |
|
milnoc
join:2001-03-05
H3B
·TekSavvy Solutions..
| reply to milnoc
The datacenter I'm considering charges $100/mo for the 1/4 rack, and $250/mo for 120V/15A power. The blade configuration I'm considering will have a maximum capacity of 1900/2900 watts depending on the voltage (I don't know which is 120 volts and which is 240 volts). The chassis itself will start off with two blades each with a single 2.5 GHz Xeon quad-core processor.
The preliminary monthly costs are as follows:
1/4 rack with 120V/15A UPS/generator backed-up power: $350
Bladecenter lease: $750 (approx.)
50 Mbps unlimited bandwidth: $350
Total: $1450 per month.
If I get a system that takes up more room and consumes more power, the extra 1/4 rack and power will cost me an extra $350 per month. If the monthly price difference between the IBM blade system and a competitor's blade system is less than $350 for similar specs, then I might as well stick with the IBM system.
But how reliable is the IBM system? If historically it's had many more problems than its competitors, then the monthly cost savings are totally meaningless. |
|
nitzguy
Premium
join:2002-07-11
Sudbury, ON
·TekSavvy Solutions..
 ·Rogers Hi-Speed
| Rock Solid Milnoc. If it has IBM on it, you can count on it. Since our company picked up doing server support for the company I work for now, large mining outfit, I've had to go to the Core room for our IBM equipment 0 times. HP Server...had to reboot it manually about a half dozen times...stupid netapp....garbage I don't even want to talk about how that thing wakes me up in an inconvenient manner.
We have some xSeries equpiment that runs our print servers, and some other stuff that I can't remember off the top of my head. But, for the most part, IBM Server = ultra reliable, even if its a little more pricey than its competitors. If you're getting it at a good price, then I say go for it.
But that's just my opinion . |
|
NeTwOrKDawg
Networking is a lifestyle
join:2005-04-25
Brantford, ON
| reply to milnoc
I have a client which had a full rack of IBM XSeries machines colo'd at Telus. Out of 7 servers, every one had the RAID controller changed at least 4 times. EACH.
Since migrating over to HP DL380's... ZERO raid controller issues. And the iLO on the systems gives great remote console access.
Even my employer refuses to use an IBM servers in our own datacenter. And we are a very large IBM partner to boot. But we only use HP in the datacenter. Why? Because years of experience of IBM vs. HP, HP proves more reliable with less parts replacements. |
|
CanerisErik
Caneris
Premium,VIP
join:2007-10-03
Toronto, ON
| said by NeTwOrKDawg  :Since migrating over to HP DL380's... ZERO raid controller issues. And the iLO on the systems gives great remote console access. ... But we only use HP in the datacenter. Same here. Our racks are filled with HP ProLiants (and ProCurve switches, but that's off-topic). No issues yet; they are rock solid hardware, we're able to hot-swap most components and iLO has eliminated KVM cables entirely.
The dedicated servers that we let our customers use are all HP ProLiants, but we also eat our own dogfood and use them for internal infrastructure.
--
Erik - Caneris - Internet solutions and more. |
|
milnoc
join:2001-03-05
H3B
·TekSavvy Solutions..
1 edit | reply to nitzguy
That reminds me. I have to check my previous employer's old IBM Netfinity 5500 server. It's still used as a Linux server for their Mantis system, but one of the drives in the RAID array may have died, and there's no more hot swap. (they stopped fixing it. It has a 500 MHz P3 for crying out loud! Break down already!)
No worries. The database is backed up daily to the Windows server, which is backed up daily on the external backup device (external 2.5" hard drives).
As for the IBM system I'm currently configuring, everything will be redundant including power supplies, RAID controller, and network interface. Even the two blades will be clones of each other so that if one blade breaks down, the other blade will keep on working.
BTW, the datacenter I'm considering has already dealt with IBM blades, so no problems there.
I'll check out the ProLiant series now. |
|
twizlar
I dont think so.
Premium
join:2003-12-24
Brantford, ON | reply to milnoc
Going to have to agree about HP, thats all we use for our own services and dedicated servers for customers. They have been rock solid across the board.
--
Broadline Networks Inc. |
|
jfmezei
Premium
join:2007-01-03
Beaconsfield, QC
·ELECTRONICBOX
| reply to milnoc
If you have a blade chassis with only 2 blades in it, you are probably spending a lot of money for the blade itself and colo costs.
If you really know that you will needto fully populate the blade cabinet "quickly", then yes, you can start with a sparely populated cabinet and fill it out. But if this will happen over a period of years, you can to get some some of commitment from the vendor that "years from now", the boards for additional CPUs on that model of blades will still be available.
Don't trust what HP says unless it is backed by a contract. They change products at a whim.
You also need to look at total network throughput on the blade cabinet. If all blades communicate through the same backbone to reach their respective ethernet cards, you may encounter some congestion in the cabinet. Since video is bandwidth intensive, you want to make sure you can grow ethernet bandwidth linearly with each added blade and not hit some limit before your blade cabinet is fully populated. |
|
lcdguy
join:2008-01-17
·TekSavvy Solutions..
| reply to milnoc
You should also consider HP's offerings. we use ton's of HP hardware at work and it's all rock solid. The other reason is i believe both HP and IBM have blade solutions that can run off of standard power and not just 3 phase. We use ours for visualization and are running around 20-30 servers of 3 blades. just my 2 cents. |
|
milnoc
join:2001-03-05
H3B
·TekSavvy Solutions..
| reply to jfmezei
jf, I'll definitely check out the bandwidth of the internal network. This shouldn't be a problem for the video files, but it may become a problem one day for the Internet access.
The reason for the blade system is simple. Because I have to retain about three months of programming material, and to properly host all the BitTorrent files my channel will generate, I'll need between 10 to 20 terabytes of storage on a single RAID array, accessible to all of the servers. I also need a solution where I can easily add a server whenever the increased popularity of my channel warrants it, and where every single component has a redundant backup component in case the main component goes down, allowing my channel to remain on the air without interruption. A blade system satisfies all of these needs perfectly.
The RAID array will be split in two. The first array will contain eight 2 TB SATA drives, and the second array will contain four 1 TB SAS drives. Both arrays will each have one parity drive and one hot swap drive, or two parity drives if the blade system can support RAID-6 P+Q. The slower SATA drives will hold all the never-updated-always-read video files, while the faster SAS drives will hold all the I/O intensive files such as the automated broadcast system's master database.
Do note that the blade system will be used mainly for data storage as well as IP based program distribution to the smaller cable companies. For the live broadcasts via dedicated fibre, that will most likely be done on a separate external server which will be installed only when I can justify the high cost of the dedicated fibre connection, or if ever the larger cable companies agree to cover the costs. |
|
twizlar
I dont think so.
Premium
join:2003-12-24
Brantford, ON
| Im not sure why you would want to use a blade center for such specific and generic tasks. You can slap a 10TB box together for much less than the cost of the bladecenter and drives.
--
Broadline Networks Inc. |
|
jfmezei
Premium
join:2007-01-03
Beaconsfield, QC
·ELECTRONICBOX
| reply to milnoc
Your storage system can connect via fibre to many different hosts. Not really different than many hosts on a single blade.
You pay a premium for a blade. It starts to pay off only once the blade is fully loaded.
Also, with a blade, you need to consider the hot swappable capabilities. You don't want to have to power off the whole array to add/remove a faulty component (blade in front, network and other cards in the back).
I think the HP product has dual power and ventilation, but not sure about hot swap capabilities.
Blades generally do provide you with better systen management capabilities over separate systems (especially for non windows systems that don't require a VGA console). The HP one (and I assume the IBM one too) have a management console that lets you connect to any of the systems in the blade from a single location.
However, blades and generally 1U systems don't give you the latest and greatest in terms of CPU performance due to space/heat consideration.
In terms of remaining on air no matter what, you need to consider what sort of hardware/sofware combination will give you that, especially if you want to do load balancing during normal times so all yoru resources are used to provide the service. |
|
CanerisErik
Caneris
Premium,VIP
join:2007-10-03
Toronto, ON
| reply to milnoc
said by milnoc :Do note that the blade system will be used mainly for data storage as well as IP based program distribution to the smaller cable companies. For the live broadcasts via dedicated fibre, that will most likely be done on a separate external server which will be installed only when I can justify the high cost of the dedicated fibre connection, or if ever the larger cable companies agree to cover the costs. Too bad you're not at 151 Front, this "high cost of the dedicated fibre connection" would have been peanuts...many of the large cable companies are here, including Rogers, Videotron, Shaw, and others, so you'd be able to connect to their networks easily.
--
Erik - Caneris - Internet solutions and more. |
|
Stormlord
join:2009-03-27
| reply to milnoc
Hi, First the best way to go is take san for your data put it in iscsi , for hardware , personnely I use IBM Xseries Server but I also work with Hp and Dell and no problem , may be dell is little cheeper. Take san with 2 or 4 giaga port and switch dedicated for you san. Use Vmware Vsphere on 2u servers for beginning , you can add more server when load is need.
Good luck with your project. |
|
freejazz_RdJ
join:2009-03-10
| reply to milnoc
If you're willing to take a look, Sun has extremely good pricing on their hardware. Their open storage platform, an ZFS more generally, is great to work with and offers great io/sec/$ performance.
Even better if you could get your stack running on Solaris on Sparc. Their T2 based servers offer incredible price/performance for non-HPC loads.
If you insist on a blade server, which makes no sense to me, they also have some offerings.
Cons of the blade server:
Proprietary form factors
Poor storage density unless you buy overpriced storage blades, which sometimes require you to add a SAN switch to the chassis
Extremely expensive startup costs to equip the chassis with PSU, networking blades and such when you've only got a couple blades.
Vendor lock in because of the above proprietary parts.
These and many more are why the blade server never took off like everyone predicted it would several years ago. I'd much rather buy 5-6 rack servers, hook them to a 10G Arista switch and run my storage, internal and external networking over a unified infrastructure than buy any blade server and have to shell out for expensive network blades, PSU's and an external SAN since most blades offer 2 2.5-inch SAS drives at best. |
|
CanerisErik
Caneris
Premium,VIP
join:2007-10-03
Toronto, ON
| reply to jfmezei
said by jfmezei :In terms of remaining on air no matter what, you need to consider what sort of hardware/sofware combination will give you that, especially if you want to do load balancing during normal times so all yoru resources are used to provide the service. Indeed. If it's any help, we've designed and built, and currently maintain, a system with similar "always on the air" requirements which runs the content for a certain satellite radio channel and phone-in app. While the storage and other requirements are certainly an order of magnitude lower than milnoc's, some of the ideas are the same. Not to sound like an ad for HP again, but we actually don't have blades anywhere, and have found the rack-mounted ProLiants to meet the high availability requirements. You can actually hot swap almost everything: power, fans, drives, NICs, RAM (well, "hot add" is more correct for that one), and you have at least two of almost everything, even on the 1U models.
You would also have more flexibility with storage in a non-blade setup.
--
Erik - Caneris - Internet solutions and more. |
|
Stormlord
join:2009-03-27 | reply to Stormlord
For correction I have make a mistake , use 1u server in addition with your san , you don't need more then 2 hdd on vm servers. |
|
curious 1
@videotron.ca
| I have nothing of value to add to this topic.
However. I'm interested in knowing more about milnoc-TV and what you will be running. If that's ok w/ you.
Also, which cable will be carrying it?
Is it french? English? Bilingual?
Got a page up? Info anywhere? |
|
