dslreports logo
 
    All Forums Hot Topics Gallery
spc
uniqs
60

koitsu
MVM
join:2002-07-16
Mountain View, CA
Humax BGW320-500

2 edits

koitsu to pandora

MVM

to pandora

Re: Need suggestions for a 1U or 2U rack mount server

Please read everything I've written below. What you've bought (from what I can tell) is not going to work without some additional purchases!

First and foremost: I almost certainly would not have gone with a Core i7 given that you're using 1U and you want a quieter system. The Core i7s tend to run hotter than the i5s (by sometimes up to 10-15C). It wouldn't be a problem assuming you're fine with the chassis fans running at full speed, but I'm pretty certain you're going to go into the system BIOS and drop the speed down due to the noise, which may cause heating/cooling problems for the CPU. I'm being paranoid here, absolutely. You'll see why this is more of a concern when I get to the HSF situation at the bottom...

Secondly: the CSE-512F-350B is one of those half-depth systems (I have one sitting in my garage -- in the video I linked, that system in the white box I mention? Half-depth Supermicro box). I love half-depth systems, but they are a serious PITA. Please don't let my comments depress you or bring you down though!

You need to be aware up front that the most painful part of the system is getting the hard disks in there and getting them wired up. Here's the user manual for the chassis:

»www.supermicro.com/manua ··· C512.pdf

See pages 5-3 and 5-4. Visually it looks like no problem, right? Yeah, trust me, it's a serious pain. Furthermore, with a 2.5" SSD, you might run into some issues mounting that in there, because the hard disks are assumed to be 3.5" and are screwed in/mounted via the bottom of the chassis (really). Maybe the 512F has mounting holes for 2.5" drives, but I strongly doubt it; they tend to focus on 3.5" drives. If it lacks 2.5" mounting holes, the easiest solution (since there's no 2.5-to-3.5 bracket that will work with underside-mounted screws) is to use some velcro tape. It's an SSD, no moving parts, yay.

The other complexity relating to the hard drives is the power connector. Now, the 512F series chassis is newer than what I've used, so this may be different, but: on the older half-depth models, all the power connectors positioned/intended for use with the hard disks were 4-pin Molex, not SATA. Given how little room there is in the case, I was only able to get 1 hard disk mounted because I had to use a Molex-to-SATA power adapter cable. See Figure A-2 in the chassis PDF (its near the bottom) for an example of the mess.

I can't find pictures of the chassis once opened up so I can't see what power connectors/etc. are in there.

I see with the 512F they changed the system cooling fan from a large circular blower to 2 or 3 square-shaped blowers, which is good (this is is what's used on all their higher-grade servers). Part of the power cable problem I mentioned on the half-depth I have pertained to how gigantic that circular blower was.

If you'd like me to make a short video showing you my half-depth Supermicro box and going over some of the annoyances mentioned above, I can do so. Just let me know.

Now, the other complexity -- this has to do with 1U boxes:

The model of chassis you bought DOES NOT come with a heatsink for the CPU. Read the Standards Parts List at the bottom:

»www.supermicro.com/produ ··· -350.cfm

You cannot use the stock HSF that comes with the CPU becuase 1U is too short (height-wise) for it to fit. You're going to need to buy a Supermicro HSF. The reason you need one of their HSFs is so that you can guarantee compatibility with the air shroud used to route the air coming off the chassis fans across the CPU. See Figure 5-10 (page 5-9) in the documentation. A third-party one might be too high/too wide and not work.

All Supermicro HSFs are bolt-through and come with thermal paste pre-applied. You can see at the above page under Optional Parts List an item at the bottom called Heatsink / Retention which takes you to a link consisting of a zillion HSFs and you won't be able to quickly discern what to buy.

It seems pretty obvious at first though, right? You want one of the ones that lacks a fan, but mounts properly with your motherboard (which is socket LGA1155). Again, see Figure A-2 for an example of the kind of HSF you're going to need.

But here's the problem: because you're using an Asus motherboard, there's absolutely **zero** guarantee that the board has enough clearance around the CPU for a Supermicro HSF. There may be capacitors or VRMs or other parts around the CPU area which won't allow the HSF to mount correctly. If you had bought a Supermicro motherboard confirmed to work in that chassis, you wouldn't have that risk. So you're just going to have to hope for the best here.

So which Supermicro HSF do you need? I think the one you want/need is the SNK-P0046P.

The SNK-P0047PS+ is for a unique chassis/motherboard, and the SNK-P0046A4 is too tall to fit in 1U. They only make 3 LGA1155-compatible HSFs, so those are your choices. I think the SNK-P0046P is what you want, and it will be compatible with the air shroud, but like I said, I can't guarantee it'll work with an Asus motherboard.

Edit: there's one other aspect that just occurred to me: the CPU socket location on the Asus motherboard might not be positioned at the same location as on a Supermicro motherboard, which means the air shroud might not line up properly at all (it might be off by say half an inch, or maybe even an inch). That's really bad. I don't know how to solve this problem other than modifying the shroud, but if you cut pieces off to make it work, you're going to lose airflow, which is bad (especially since you bought an i7). :/ This is why it's almost always best to go single-vendor (i.e. Supermicro chassis + Supermicro motherboard) when it comes to building a server system.

Welcome to all the "fun stuff" you get to deal with when it comes to 1U chassis. With 2U this isn't as big of a problem because you usually just use the stock HSF provided with the CPU, or if it won't fit, you get one that's similar to the classic Core 2 Duo / Core 2 Quad HSFs. I forget what my Core i7 came with (I always use third-party HSFs on my desktops), but if it's like this one, then yeah, it works in a 2U but is too high to fit in a 1U.
pandora
Premium Member
join:2001-06-01
Outland

1 edit

pandora

Premium Member

Thanks for the cautions. I'll report on the build. If the case doesn't work out, since it's from Amazon, I can pay to return it and select a 2U version. Looking at the thermal specification at the Intel site - »www.intel.com/content/ww ··· ide.html it appears Intel lumps thermal specs for all 77 watt 3rd generation i5 / i7 CPU's together. As if they are related.

The i5 and i7 versions seem very similar electronically. At least based on the Intel literature.
Thordrune
Premium Member
join:2005-08-03
Lakeport, CA

Thordrune

Premium Member

Based on my experience, HyperThreading does increase temperatures on a heavily-loaded CPU. TDP is a worst-case scenario, they probably lumped them together to make it easier/less effort for them.
pandora
Premium Member
join:2001-06-01
Outland

pandora

Premium Member

said by Thordrune:

Based on my experience, HyperThreading does increase temperatures on a heavily-loaded CPU. TDP is a worst-case scenario, they probably lumped them together to make it easier/less effort for them.

It wouldn't surprise me. Should I put up pictures of the build? For me, this is my first rack mount PC. I have a 12U wall mount rack, the system will be integrated with the entire home for lighting, security, video and phone. Prior to this, my only concern was building desktop for surfing or gaming. This is more of a home server appliance.

I was amazed at how prices for things like RAM have come down since my last i7 builds (circa mid 2010). 16 GB for $55? That seemed amazing.

koitsu
MVM
join:2002-07-16
Mountain View, CA
Humax BGW320-500

koitsu

MVM

Yeah, DDR3 right now is pathetically cheap. If you have a board that supports lots of RAM, might as well max it out considering the cost.

Hard disks still haven't come down in price since the Thailand flooding (Slashdot just ran an article proving it -- I've been trying to tell people... sigh...), while SSDs are under $0.50 a gigabyte now. Come on 1TB SSDs! Get here so I can replace my damn storage arrays with low-latency low-power stuff!
pandora
Premium Member
join:2001-06-01
Outland

pandora to koitsu

Premium Member

to koitsu
The Supermicro rack mount case arrived today. It's got an 80 gold (says 88% at half load) efficient 350 watt. It measures 19" by 15", and had 2 SATA power adapters (which connect to the older molex). SATA data cables, and motherboard, rack, drive mounting hardware are included.

The case has no shroud, instead it has 2 sets of rectangular fans attached in what is effectively a rectangle, with 2 fans ganged one behind the other (4 fans total). So it's 2 fans wide and 2 fans deep. The fan sets (left and right) each have 1 4 wire cable coming out (I assume this means a pair dual fan motherboard speed controllable case fans). I believe the Asus motherboard can support up to 4 speed controllable fans, and the BIOS has a feature that can run a thermal / power test to optimize motherboard operation based on the case cooling. If you go about 3 or so minutes into this video, an ASUS representative discusses it - »www.youtube.com/watch?fe ··· B6BB5628

Removing the case cover required taking out 4 Phillips screws. The case includes mounting hardware and brackets for an ATX motherboard plus a hole schematic / layout (of the holes in the case, which I guess is to help when placing the motherboard).

A regular PC type power connector was supplied for the case, there is an on / off button for the power supply, an on / off for the case and a reset. Additionally various status indicator lights on front of the case.

The front of the case is black as is the front bracket mounting hardware, but the rest is unpainted metal (probably steel) inside and out.

There is room to mount 2 drives in the front of the unit, there are only 2 screws on the bottom of the case for drive mounting, but there seems to be some sort of adapters included. I'll know more when the motherboard arrives. With any luck this weekend I'll be assembling the unit. I wonder how loud it'll be.

Looking at the case, and the video linked above, it appears the CPU may be partially occluded by RAM. Other than RAM there is nothing to block air flow around the CPU to or from the 4 fans in the case as best I can tell at this time.

koitsu
MVM
join:2002-07-16
Mountain View, CA
Humax BGW320-500

1 edit

koitsu

MVM

Ah, I see. Yeah, on server-class boards, the RAM is usually positioned length-wise (i.e. the sockets are rotated 90 degrees compared to that Asus board) so that the cooling fans can blow air across everything. Welcome to desktop vs. server design.

I would definitely be concerned about CPU cooling at this point. I'm left with the impression that the fans are going to cool a single DIMM on your system and very little else -- and that's bad. An i5 and i7 (especially an i7) needs some form of cooling vs. just a heat sink.

The chassis setup/assembly/etc. sounds normal (almost identical to the half-depth Supermicro I have in my garage). Maybe a few changes for the better, but what you describe sounds about the same.

As far as how loud it'll be, I can tell you right now it's going to be loud. The chassis fans are FAN-0087L4, PWM-driven, and are spec'd at between 12000 and 18000rpm. That isn't a typo: 12 thousand and 18 thousand RPM. These fans are what cool the entire system (RAM and CPU mainly).

I'm sure you can hook the fans up to your Asus board (not sure if chassis fans use 3-pin or 4-pin headers) and adjust the RPM levels via the BIOS. How much of an adjustment is unknown. You'll find out once you get the motherboard of course.

Supermicro may offer compatible fans (i.e. same size/depth) but run at a lower RPM; I don't know. You might be able to find third-party fans that are of the exact same size (40x56mm) and run at a lower RPM, but you'll need to make sure they can be used/mounted in the chassis properly. I tend to recommend folks just stick with Supermicro-authorised products. The smaller a fan is (in size), the faster it has to run to push air -- that's why things like 120mm fans can run at 1200rpm and cool effectively. 40mm fans are a different story.

When it comes to server-class systems, noise is almost never a focal point. This is why when I hear of people putting racks in their homes, I chuckle and think "I hope it's downstairs in a basement, or in a garage".

An alternate solution is not to go with 1U or 2U servers at all, but instead invest in a chassis like this which is a standard desktop case except supports being rotated on its side and mounted in a rack (has mounting holes for brackets and rails which are purchased separately -- see picture on right). You can then use case fans of your own choice, stock HSFs, and so on -- it's a desktop case. They do make a black version of that chassis as well (model number ends with "B"). The downside is that it takes up 4U of rack space vs. 1U or 2U.

There's a slightly cheaper solution than buying a case like that too, but it's incredibly ghetto: simply buy a desktop case of your choice, and then buy a rack shelf that fits your rack and place the chassis on its side on the rack. I see people in datacenters doing this all the time, and I shake my head because it's incredibly unprofessional, but it does work when you have unique requirements/constraints.
pandora
Premium Member
join:2001-06-01
Outland

pandora

Premium Member

Thanks for the thoughts ... you've been helpful and patient with me.

When the motherboard arrives, I'll install it and see what sort of noise it makes. The plan is to install it in a normally occupied area, but if it really can't be silenced, I'll either change fans or look for another solution. If I get the motherboard on Friday, I'll have more answers and maybe a short video by Saturday.
pandora

pandora to koitsu

Premium Member

to koitsu
Just an update before going to bed ... Newegg shipped it's motherboard to me from Memphis (I'm in Connecticut). They missed Tuesday (I ordered a bit before 4 PM). Tracking says the motherboard won't be here until Monday.

Sigh.
pandora

1 edit

pandora to koitsu

Premium Member

to koitsu
The motherboard came in early, and unfortunately the Xenon motherboard front cable connector on the Supermicro 1U case is 16 pin, but my supermicro motherboard requires a 20 pin connector.

I ordered the recommended adapter part (Supermicro CBL-0068L) from an Amazon store, it may take a week or more to arrive (Thanksgiving is next week). Supermicro's manual for the case offers a number of suggested cables to solve issues like this.

The pin outs for the motherboard front panel cable provided with the case CBL-0156L were not provided on any site on supermicro or directly via google search. When checking front panel header layouts on other 16 pin Supermicro motherboard (X9SCM, X9SCM-F, X9SCL, X9SCL-F, X9SCL+-F) the pin outs were very different than the Asus motherboard. I could try to disconnect and rewire the connector so at least power and reset would work, but it'd probably be better to just wait for the appropriate cable.

The motherboard mounted on the case, but the back panel didn't fit from the case, the Asus back panel was too high for the 1U case. As it is, I'm leaving it open for now.

At least one of the case fans will need an extension cable as the supplied fan cables can't reach more than one of the motherboard fan controllers.

The Intel CPU fan is too high to close the case. A smaller fan will have to be found.