Dennis Mod join:2001-01-26 Algonquin, IL |
Dennis
Mod
2014-Apr-15 4:07 pm
Dell poweredge T620 painfully slow on Windows Server 2012 R2Brand new, out of the box from Dell and the computer is running slower than molasses. I tried tweaking the priorities from background to applications: » www.win2012workstation.c ··· ormance/But even just starting up, shutting down or installing updates takes hours! I'm kind of at my wits end...is this common? |
|
Badger3kWe Don't Need No Stinkin Badgers Premium Member join:2001-09-27 Franklin, OH |
Badger3k
Premium Member
2014-Apr-15 4:14 pm
Is the BIOS set for Performance or one of the power saving modes? Have seen performance issues with that before. |
|
lorennerol Premium Member join:2003-10-29 Seattle, WA |
to Dennis
Have had massive ongoing issues with two T620s going back to last year. After months of troubleshooting, it seems to be related to the read queue on the RAID controller getting backed up.
It easily triggered by creating shadow copies and/or running Windows Server Backup (which uses Shadow Copies).
In my cases they are running Server 2012 (not R2). RAID1 SAS for host OS and RAID5 SAS for data. |
|
Dennis Mod join:2001-01-26 Algonquin, IL |
to Badger3k
It just now finished rebooting, and in the BIOS it says: System Profile--------------Performance Per Watt (DAPC).
Drop down gives a choice of: Performance per Watt (DAPC) Performance per Wat (OS) Performance Dense Configuration Custom
Any recommendation? |
|
Dennis |
to lorennerol
said by lorennerol:Have had massive ongoing issues with two T620s going back to last year. After months of troubleshooting, it seems to be related to the read queue on the RAID controller getting backed up.
It easily triggered by creating shadow copies and/or running Windows Server Backup (which uses Shadow Copies).
In my cases they are running Server 2012 (not R2). RAID1 SAS for host OS and RAID5 SAS for data. This is running RAID 5 as a whole. |
|
|
Badger3kWe Don't Need No Stinkin Badgers Premium Member join:2001-09-27 Franklin, OH |
to Dennis
We have them set to just the Performance option straight from Dell. To my knowledge we haven't had any complaints of performance. Most of ours go out as Hyper-V hosts running 2-4 VMs. |
|
lorennerol Premium Member join:2003-10-29 Seattle, WA |
to Dennis
said by Dennis:This is running RAID 5 as a whole. Look at the disk queue in ResMon. In our case it runs from 2-30 seconds when the problem is happening. |
|
Dennis Mod join:2001-01-26 Algonquin, IL |
Dennis
Mod
2014-Apr-15 5:29 pm
I got the computer rebooted, made the BIOS change and so far it doesn't seem to be the main issue.
I'm sitting here watching the resource monitor after I tried to launch FireFox and all I can hear are the hard drives thrashing. The disk queue hasn't moved from 7.35 but it almost seems like it's frozen. The disk I/O rate never got above 2.
At this point it's just locked up, gonna leave it alone to see if it comes back to the living. |
|
Dennis |
Dennis
Mod
2014-Apr-15 5:52 pm
It "woke" up and I took a few screen shots. The first was right after it finally started responding and the second was about 5 minutes later. It's still sitting here just going crazy. This is my first time touching a poweredge or Server 2012 so I appreciate the guidance. |
|
lorennerol Premium Member join:2003-10-29 Seattle, WA |
to Dennis
That looks painfully familiar. I've seen the problem get so severe that ResMon stops recording data- the graphs go blank and the lists of data either stop updating or also go blank.
Call them and tell them you seem to have the same issue as SR 886302801
I can't say for certain that it is the same problem, but it sure looks similar. |
|
Dennis Mod join:2001-01-26 Algonquin, IL |
Dennis
Mod
2014-Apr-16 5:28 pm
I left it running overnight and when I came back it was pretty stable but then it took an 1/2 hour to shut down again. On the phone with Dell right now so we'll see what happens. I checked the PERC S110 raid controller during a reboot and all the drives are green across the board. |
|
lorennerol Premium Member join:2003-10-29 Seattle, WA |
to Dennis
Good luck. Please post back so we all know the result. |
|
Dennis Mod join:2001-01-26 Algonquin, IL |
Dennis
Mod
2014-Apr-16 6:07 pm
I'm being told that because the server is setup as RAID 5 that the PERC s110 really isn't the best choice. Basically the box is choking on itself and we need to upgrade to a hardware based RAID controller.
Funny thing is we would have readily we just didn't know it would be this painfully bad. |
|
OvrQualifiedSlightly Ahead Of Time Premium Member join:2002-01-27 Winter Park, FL |
to Dennis
You're not the only one running into this with RAID5 on the S110: » en.community.dell.com/su ··· 489.aspx |
|
lorennerol Premium Member join:2003-10-29 Seattle, WA |
to Dennis
said by Dennis:I'm being told that because the server is setup as RAID 5 that the PERC s110 really isn't the best choice. Basically the box is choking on itself and we need to upgrade to a hardware based RAID controller.
Funny thing is we would have readily we just didn't know it would be this painfully bad. They told me the exact same thing, except the two T620s we're working with shipped with H310 controllers. Dell sent H710P controllers, which I installed. Made no difference. They also told us it was the USB 3.0 controllers we'd added to the boxes to facilitate backup to external disk. It wasn't that either. |
|
lorennerol |
to Dennis
Also, my opinion...if their hardware can't do something as incredibly basic as booting a server OS from a RAID5 array on that RAID controller, they shouldn't allow that as a configurable option. It's like selling a 3 ton automobile with a 5HP engine, but not telling people the engine won't actually move the car. |
|
Dennis Mod join:2001-01-26 Algonquin, IL |
Dennis
Mod
2014-Apr-16 7:53 pm
said by lorennerol:Also, my opinion...if their hardware can't do something as incredibly basic as booting a server OS from a RAID5 array on that RAID controller, they shouldn't allow that as a configurable option. I kind of voiced that opinion while on the phone. Really it shouldn't be an option at all. |
|
|
Seen this before on these.. You opted for the "fakeraid" ie software raid. The problem is it is slow as sin. I have 4 of these running a private stack... and even on linux they are slow as sin. disk i/o wait is so high it makes them useless. the only thing I could do was update the drivers from their site and run windows on em.
Update the driver and the bios. |
|
drh Premium Member join:2008-10-20 Olympia, WA |
drh to Dennis
Premium Member
2014-Apr-17 11:38 pm
to Dennis
Couple things to be aware of. Yes - you have the fakeRaid (software raid). Not going to be nearly as fast as the H710 or H710P. Another thing to be aware of... When you setup Raid 5 (any raid set really) the controller will initialize the array in the background. Performance can be slow during this time. Total time will depend on the size of the drives/array. I've seen 3TB 7.2K drives in a 5+ disk Raid 5 array take more than a 36 hours to finish. I didn't see you list the drive size. I'll add my .02 here and say the use of Raid 5 is almost never a good idea. There has to be conditions in place for me to consider it. Raid 1 or Raid 10. I Raid 10 almost all the time now. At work we have some PE R720xd with the H710P controllers and 14 disk Raid 10 using 10K 1.2TB disks. They scream! |
|
lorennerol Premium Member join:2003-10-29 Seattle, WA |
said by drh:At work we have some PE R720xd with the H710P controllers and 14 disk Raid 10 using 10K 1.2TB disks. They scream! I've found several reports that indicate that the disk I/O issue that seems to plague the T420 and T620 servers does not affect the R720. In our case we replaced older X900 series servers with 3-disk RAID5 arrays with new T620s with 3-disk RAID5 arrays (and 15k SAS disks instead of 10k SCSI) and we're seeing a massive performance drop. We're not talking about high-end applications here- the OP has a server that barely boots and in our case we can't run backups of the host without the VMs slowing down so much that they appear to be offline to the end users. This is basic, basic functionality. |
|
drh Premium Member join:2008-10-20 Olympia, WA |
drh
Premium Member
2014-Apr-19 11:02 am
I'm not saying you guys are seeing ghosts, but I'm going to need more info on the application/need for such a setup. The T620 in it's smallest setup holds 8 3.5 Hot-swap drives. Why are we talking about 3 disk Raid5 arrays. I wouldn't put a 3 disk raid5 array at my house even to store a backup of my backup. Are we saying there is a 'known' issue among users of T420/620 with IO issues? - what is the common setup here then? H310 controllers? SATA 7.2 drives? Three disk arrays? But I agree, Dell should remove the S110 as an option. For light storage use a H310 in Raid 1 or 10 would be fine (again, no Raid 5 please). I know cost can become a factor, but the performance gain of a H710P with the correct Raid/disk types for the application work great. One big Raid 10 and Hyper-V these things. |
|
Dennis Mod join:2001-01-26 Algonquin, IL |
Dennis
Mod
2014-Apr-19 5:02 pm
RAID 5 was listed as a requirement for the software to be installed on it. It will be an SQL server holding medical records for a friends business. |
|
drh Premium Member join:2008-10-20 Olympia, WA |
drh
Premium Member
2014-Apr-19 8:13 pm
I have a number of SQL Servers at work and we will not put any data on a RAID5. SQL requirements/recommendations by vendors are almost always outdated and behind the times. With the changes in all things storage related 'new best practices' are now considered the norm. I still see quotes from SQL guys based on things Microsoft wrote when SQL 7 came out. Mega tons of reading on this site - if not a member, become one. »community.spiceworks.com ··· che-mode See the links in the first response to this persons question. For all intents and purposes Raid 5 is dead. Not just my .02 I don't know the budget and system requirement of the project for this T620, but I can think of a reason where using 3 disks in a Raid 5 would be a good idea. Heck - I'd buy used drives on Ebay and put them in a big Raid 10 with spares standing by and risk a drive failure then 3 brand new drives in Raid 5. |
|
|
to Dennis
will they take RAID 6 as well? It's more redundant then RAID 5 but it really needs a hardware raid card to work at a good speed. |
|
AsherN Premium Member join:2010-08-23 Thornhill, ON |
to Dennis
said by Dennis:RAID 5 was listed as a requirement for the software to be installed on it. It will be an SQL server holding medical records for a friends business. The RAID5 requirement is old and severely outdated. RAID5 was OK for a small number of small drives. RAID6 is an OK alternative, but I really only consider it for archival or backup storage. Sure, it's more redundant than RAID5, until you lose your first drive. Also, in arrays with a lot of writes, the IO penalty of parity based arrays only grows as the number of drives grows. with a 3 drive array, add a 4th drive and use RAID10. You will get faster reads overall, and a consistent write penalty regardless of the number of drives in the array. |
|
AsherN |
to Joe12345678
said by Joe12345678:will they take RAID 6 as well? It's more redundant then RAID 5 but it really needs a hardware raid card to work at a good speed. If we're talking about a 3 drive array, adding an additional drive to do RAID6 makes no sense. Might as well add the drive and go RAID10 |
|
|
to lorennerol
said by lorennerol:said by drh:At work we have some PE R720xd with the H710P controllers and 14 disk Raid 10 using 10K 1.2TB disks. They scream! I've found several reports that indicate that the disk I/O issue that seems to plague the T420 and T620 servers does not affect the R720. We have a bunch of Server 2012-equipped T620s with H710s; they all haul ass. Granted, all of them are running RAID 10s. |
|
Dennis Mod join:2001-01-26 Algonquin, IL |
Dennis
Mod
2014-Apr-27 7:48 pm
I appreciate the feed back. I have an H710 card and will be converting (re-installing) from the PERC SP110 to the new setup this week. |
|
AsherN Premium Member join:2010-08-23 Thornhill, ON |
AsherN
Premium Member
2014-Apr-27 10:10 pm
Do yourself a favour, at the same time, get a new drive and re-create the array as RAID10. |
|
mikefxu join:2004-10-05 Titusville, FL |
Just note that you cannot expand a RAID 10 array on the H710. |
|