before booting: Memtest86 - this is what I use to check my RAM before I attempt booting into windows.
Put the app on a bootable floppy, and as you boot from it, it will run its memory diagnostic app. Test#5 & 8 are known to be good for conditioning, but I prefer to run tests 1-7 to ensure my memory is error-free at this point. And I never boot to the OS at a new FSB/timings set until it passes memtest.
in windows: Sisoft Sandra - its memory bandwidth is good for testing memory speed, as well as helping to condition it (w/ the burn-in wizard).
Prime95. This helps burn in CPU and RAM, and is very sensitive, so if you can get this thing running its torture test for a long time error-free, chances are your system is stable. -- Why I hate VIA