First things first, you should get your terminology in line. You've said megabytes a few times, What you are paying for is MegaBITS.
20Megabits = 2.44Megabytes.
Next I'd suggest thinking of Internet access like a road system, as ilianame suggested. Your paying for a freeway to your home with a speed limit of 20mbits. There may be congestion on that freeway which will slow you down, and once you leave that freeway and go onto a different road system (the servers Internet connection) it might slow down.
Finally you keep saying what the average for your city is, which is meaningless. They might be paying for dial up (its still around), or 1.5mbit connections, which is going to skew the average. What you need to be concerned with is your connection. You haven't actually given any stats about your connection.
Please use an actual speed test to check this »
speedtest.shaw.ca/ would be your best bet since thats between you and shaw, without getting the outside world into effect.
Test this at multiple times during the day, and feel free to post them so people can analyze.