Flash your router perhaps? I did mine a long while ago, and speeds weren't top notch, I flashed the latest I could and needless to say, speeds went up 40%... -- I hate rogers and their CRAP tech. support... But, MAJOR kudos to the TekSavvy team!
My Asus WL-520gu wasn't up for the task with Tomato or dd-wrt installed with a 28/1 and QOS. With VOIP that was pretty much mandatory. I don't know if it was a fault in my settings or the router but I ended up buying a new router which would more than meet my needs for years to come.
But definately what helps is turning off all the features you don't need (ie QOS). A quick google search showed that with your router turning off WMM helped some other guy. I wish you the best of luck tracking down the problem. It can be tough. Like other users have suggested... I've also had good experiences with both DD-WRT and Tomato firmwares.