I would say it's the GPU. From my experience, RAM-related artifacts tend to be in patterns of stripes or blocks of dots (
example), and GPU artifacts tend to be random-colored or "snow" pixels in various areas, or large polygons (
example).
As for the RAMDAC, that would only be an issue if you're not using DVI/HDMI or DisplayPort. How is it connected to your monitor?