dslreports logo
site
 
    All Forums Hot Topics Gallery
spc

spacer




how-to block ads


Search Topic:
uniqs
878
share rss forum feed


The WeaseL
Premium
join:2001-12-03
Minnesota

Debian box locking up

I'm a fairly novice Linux user, so speak slowly, use small words.

I've got a Debian box running MySQL and Drupal. Runs fine, save for the occasional complete lock up. Cannot even access it from a console. Only option is to do hard reboot.

I have no idea where to start as far as digging into logs to see if I can find something, so I am hoping for a few pointers.
--
How lucky am I to have known someone who is so hard to say good-bye to.


VVSneakEh

join:2003-02-17
Toronto, ON
Probably hardware related, has this box run another OS successfully?


graysonf
Premium,MVM
join:1999-07-16
Fort Lauderdale, FL
kudos:2
reply to The WeaseL
Test the memory with Memtest86 or similar.


The WeaseL
Premium
join:2001-12-03
Minnesota
reply to The WeaseL
Running in VMWare on our cluster, all other servers are running ok.


Selenia
Gentoo Convert
Premium
join:2006-09-22
Fort Smith, AR
kudos:2
reply to The WeaseL
Debian is rock solid. So rock solid that the only time it did this on me was when my hard drive was failing. Could be bad memory or a bad driver too. Do you see anything above warnings, like "error", "fatal" or "critical" in dmesg? Just type it in the terminal to see it. Do you run any other OS? If so, does it have the same issue? That could narrow this down a lot. Debian is solid, but drivers can suck. I remember when I needed fglrx and it was so bad that lockups during OpenGL apps was considered the norm for some graphics cards in any Linux distribution. Glad those days are gone. Still, you see some stragglers here and there, as drivers go. I just had to drop a defective bleeding edge xorg update because enabling compositing=crash(at least for the GUI, but could terminal in). Running beautiful again now that I rolled back.
--
A fool thinks they know everything.

A wise person knows enough to know they couldn't possibly know everything.

There are zealots for every OS, like every religion. They do not represent the majority of users for either.


jimkyle
Btrieve Guy
Premium
join:2002-10-20
Oklahoma City, OK
kudos:2
Reviews:
·AT&T Southwest
reply to The WeaseL
You say you're fairly novice, so are you aware of the "virtual terminals" available by the CTRL-ALT-Fn hotkeys, where n = 1 through 6? When you get one of the complete lockups, try all six of them before doing the hard reboot. If you get a command-line login prompt at any of them, use your UserName and password to log in, then type "ps ax >~/crashlog" to save a list of processes into your home directory under the name "crashlog" so you can examine it later. Then at the next prompt, type "sudo shutdown -r now" to do a soft reboot, and type in your user password when asked for it. Don't expect to see any keystrokes echoed to the screen while you type the password; it's going in okay anyway.

That should give you some detail to help diagnose the problem, in addition to the system logs (which in case of a lockup may well not be fully up to date anyway).
--
Jim Kyle


Selenia
Gentoo Convert
Premium
join:2006-09-22
Fort Smith, AR
kudos:2

1 edit
Sudo does not work with a default Debian install. It's disabled by default. This is not Ubuntu. Likely, a novice has not bothered to enable sudo for any reason. He could login as root/root password or login as user/user password then type su then root password to do that command(without the sudo). Good idea on the crashlogs. He should also check dmesg, as I said. Often times, hardware issues cause minor errors before the big crash. My failing hard drive had the writing on the wall in dmesg "I/O buffer error at sector xxxxxx block xxxxxx on /dev/sdxx" before my backup and soon after, complete hard drive failure. This happened after several lockups, but at least examining dmesg warned me.

Edit: User said he cannot access from console. I assume he means ssh. Ssh does not run off of X. An X lockup(had plenty of those with fglrx) is out of the question. Likewise, ssh may work when the virtual terminals do not, in the case of certain hard lockups of X where the keyboard input driver gets frozen and would not acknowledge ctrl+alt+Fx(1-6).
--
A fool thinks they know everything.

A wise person knows enough to know they couldn't possibly know everything.

There are zealots for every OS, like every religion. They do not represent the majority of users for either.


The WeaseL
Premium
join:2001-12-03
Minnesota
reply to The WeaseL
Actually by console I meant the VMWare console as if I was sitting physically in front of the box trying to get in via the GUI. Root is enabled, I did most of drupal and mysql install via command line, but I did not try SSH yet so when it locks up again I will check to see if I can SSH in and get the ps ax crashlog.

Appreciate the help so far, I hate that the issue is a wait for it to break type.
--
How lucky am I to have known someone who is so hard to say good-bye to.


ironwalker
World Renowned
Premium,MVM
join:2001-08-31
Keansburg, NJ
reply to The WeaseL
more info,
What kernel?
How is the vmware config, setup?
Have you checked logs after restart?
I asked about the config because it could be running out of memory or hard drive space.


The WeaseL
Premium
join:2001-12-03
Minnesota
reply to The WeaseL
Debian Release 6.0.5
Kernel 2.6.32-5-686

In vmware it is configured to have two CPUs (servers have Xeon X5650's), and 2 GB of RAM. Checking the System Monitor at any one random time, under the resources tab shows the CPU 10%, RAM 400 MB, and either no swap used or a few MB.


The WeaseL
Premium
join:2001-12-03
Minnesota
reply to The WeaseL
Well it hard locked on me today, my only way in was through SSH and was able to do a ps ax to a crashlog file, but also did 'top' to see if anything was suddenly maxing out the CPU's and noticed the load average: 0.21, 37.75, 84.67

My understanding here is the 1 minute average is fine, but the 5 and 15 minute ones are way high.

Edit: To add, while doing this, the box eventually become responsive again and began working normally without having to do a reboot. It was hung up for --
How lucky am I to have known someone who is so hard to say good-bye to.


YouOwnedBro

@pnap.net
Are you sure you're not compromised? You chrooted Drupal? AppArmor/SELinux? Up and up on patches? PHP set to not permit url_fopen, url_include, and you've disabled functions like curl_init, exec, system, pass_thru, etc?


Selenia
Gentoo Convert
Premium
join:2006-09-22
Fort Smith, AR
kudos:2
reply to The WeaseL
Check your memory ASAP(thought I asked you to before). Hard locks point to kernel or hardware. How does the machine run without the virtual machines? Wonder if something in the virtual machines is interacting poorly with the kernel modules VMware installs. If it runs fine without them enable them 1 at a time on single core so you have some play if it starts kicking the CPU. I found VirtualBox a little trickier to set up, but found its kernel modules to be more adaptive to system upgrades and the like. But first check your memory to keep that out the realm of possibilities. That will mess up a system just as quick as a bad hard drive due to corrupt disk writes.
--
A fool thinks they know everything.

A wise person knows enough to know they couldn't possibly know everything.

There are zealots for every OS, like every religion. They do not represent the majority of users for either.