dslreports logo
site
spacer

spacer
 
   
spc
story category
Charter, Cox Comcast Tops PC Magazine Speed Rankings
As Mag Turns to Ookla to Shore up Test Issues
by Karl Bode 04:48PM Friday Sep 02 2011
For several years now PC Magazine has been conducting a somewhat controversial ranking of broadband ISPs by speed, using the PC Magazine Surfspeed application. Criticism over the years has grown about the magazine's methodology, and their decision to rank ISPs based on browsing speed in the age of 100 Mbps connections and Internet video. In 2007, many of our readers took issue with the magazine's results, noting the findings listed ISPs in incorrect states, and even managed to confuse gigabit and kilobit measurements.

Click for full size
Last year, popular speed measuring site Ookla was pointedly critical of the magazine's methodology, stating the PC Mag test didn't represent the performance of an ISP or a modern web browsing experience:
quote:
...the test happens in serial using two connections at a time, but modern web browsers go well beyond this old HTTP limitation. For example Firefox 3 and Internet Explorer 8 default to six parallel connections to a host when retrieving content. PC Magazine claims that they are trying to represent the speed of an actual web browsing experience, but apparently it is not a modern one. Ignoring that issue the test could still be used to compare rudimentary web browsing performance, but does that really represent the performance of an ISP? No.
PC Magazine was listening, and this year finally changed their measuring methodology -- by hiring Ookla to do it for them:
quote:
In previous years, our tests focused on browsing download speeds. However, with video streaming and massive uploads/downloads becoming the norm, we realized we needed to change our methodology. We listened to your feedback and turned our attention to Ookla, a leader in broadband testing and Web-based network diagnostic applications, and its popular (and proven) product Speedtest.net. Speedtest.net is almost the de facto standard for testing connection speeds. Ookla set us up with our own PCMag branded version of the test.
As a result, this year's tests look considerably different, with Comcast, Cox and Charter all tying for top honors based on 58,300 tests completed from May 6, 2011 until July 29, 2011 at pcmag.speedtest.net. Verizon, who normally dominated PC Mag's charts courtesy of FiOS, falls slightly this year because Ookla averages both FiOS and slower DSL customers into a single score. The magazine's test also breaks down ISP speed by region, fastest business ISPs, as well as fastest ISPs by state, city and globally.

You can find this year's PC Mag test methodology detailed here, and you can compare the results to Ookla's Net Index website.

view:
topics flat nest 

Gbcue
Premium
join:2001-09-30
Santa Rosa, CA
kudos:8

1 recommendation

Sonic.net

Where's Sonic's gigabit service?

Metatron2008
Premium
join:2008-09-02
united state

Where is At&t?

Verse gets decent speeds... are they seriously slower then earthlink?
en103

join:2011-05-02
Reviews:
·Time Warner Cable

1 edit

1 recommendation

Re: Where is At&t?

They're not grabbing the top 5% of all users, but taking an 'average' of all users.
AT&T's Uverse internet starts out with 3Mbps/1Mbps and charges extra. If they add in the ADSL users, many never make it past 3Mbps (2.4Mbps after ATM overhead).

TWC (and most cable providers) first bundle in +10Mbps/1Mbps (they'll charge more for a 1.5Mbps if you wanted to downgrade in a bundle!), 'then' they add on 'Boost' to fluff the numbers with a 20-30Mbps download rate for the first 10-20 seconds.

Eg. My '10/1Mbps package'


said by pcmag :
Who scored the lowest? Folks in Alabama, South Carolina and Mississippi appear to stumble thanks to AT&T South (formerly Bellsouth). All had an index of well below 5.0 (Alabama's AT&T speed is the lowest on the chart, with an index of 2.56.). The District of Columbia comes close, with Verizon getting an index score of 3.19 (despite the fact that FiOS is available in D.C.).

spewak
R.I.P Dadkins
Premium
join:2001-08-07
Elk Grove, CA
kudos:1
Reviews:
·SureWest Internet

Re: Where is At&t?

said by en103:

They're not grabbing the top 5% of all users, but taking an 'average' of all users.

I didn't think so:


--
The weekend is here, grab a can of beer!
con100

join:2011-08-03
Westlake, OH
I used to have Uverse and it was slow. Got slower speeds then what I paid for.

houkouonchi

join:2002-07-22
Ontario, CA

DSL users brinigng down VZ

The reason VZ is so low on that has to be because its including both FIOS and DSL users... too bad they couldn't separate them.
--
150/75 mbit Verizon FiOS connection FTW!

RedCaliSS
Premium
join:2004-08-21
Murrieta, CA

1 recommendation

Re: DSL users brinigng down VZ


this study is worthless
but what can you expect from PCrag magazine...
had they used the test results from the FCC & Sam knows to get a true accurate picture this so called study from PCrag would be quite different.
speedtest.net is unable to distinguish between DSL and Fiber?
these test results are inaccurate IMHO. Thought VZ used different IP blocks for DSL & Fiber?
en103

join:2011-05-02

Re: DSL users brinigng down VZ

Beyond that.... most cable results are fluffed with 'Boost' and many DSL lines (AT&T) are killed with ATM overhead.

C_Chipperson
Monster Rain
Premium
join:2009-01-17
00000
kudos:3

1 recommendation

Fios isn't nationwide... nor is verizon for that matter...but i gues none of the ISP's listed are truly nationwide right? hmmm... "Fastest ISPs Nationwide"...what an odd list!
tmc8080

join:2004-04-24
Brooklyn, NY
Reviews:
·ooma
·Optimum Online
·Verizon FiOS
said by houkouonchi:

The reason VZ is so low on that has to be because its including both FIOS and DSL users... too bad they couldn't separate them.

Well, then there's just one thing to do then... DOUBLE THE SPEEDS FOR FREE!! That should shut Comcast out...

fifty nine

join:2002-09-25
Sussex, NJ
kudos:2

1 recommendation

said by houkouonchi:

The reason VZ is so low on that has to be because its including both FIOS and DSL users... too bad they couldn't separate them.

Or maybe Verizon needs to get off its ass and deploy more FiOS.

MonkeyLick78

join:2002-01-27
Hixson, TN

??

Surely the don't include power boost in this, right? That would make the whole thing pretty worthless.

Mediar46

@mycingular.net

Mediacon honors last place

Along with outrageous service and prices. I get during peak hours 1mb/s of 12mb. Even att 3g is faster than this s***t.
corinthos

join:2007-10-09

Nice to see WOW on there

Nice to see WOW on there. most comparasions I never see them listed.
Wilsdom

join:2009-08-06

Might as well rank them alphabetically

Comparing capped and uncapped services is ridiculous. In terms of the value and real functionality of the service to subscribers, Potemkin ISPs like Cox and Comcast can't even be ranked.
zackvlion

join:2010-06-13

not helpful at all...

This is really more about how fast the average plan speed is...If you get the cheap-o 1mbps plan, you'll have slow internet. if you pay the $100/month and get the 50mpbs you'll have faster...

What would be more helpful is how well the ISPs deliver closest to advertised speeds. If you pay for 15megs down, do you get it?

jmn1207
Premium
join:2000-07-19
Ashburn, VA
kudos:1

Re: not helpful at all...

It's not supposed to be helpful, as much as it is controversial. I sincerely wish this was just left as a link.

BHNtechXpert
BHN Staff
Premium,VIP
join:2006-02-16
Saint Petersburg, FL
kudos:151

1 recommendation

PC Mag had it right...Oookla had it wrong...

I'll catch all sorts of hell for this but I know I'm right and it needs to be said. PC Mag had the general idea right to begin with, Ookla was wrong and they have always been wrong.

First of all the online user experience should NOT be measured by web browsing alone because being online is far more than just surfing nowadays.

There are two major types of measurements that should be made when you are benchmarking the quality of your internet connection.

1) Application Speed and Quality - The absolute maximum speed and quality of service obtainable through a single socket test. This is important because streaming, voip and similar applications in most cases are NOT multi-socket but are in fact single socket. That said this form of testing is critical in evaluating the real quality of an internet connection because for example if your subscribed tier is 40/5 and your single socket application test results rates you at 5/1 with a QoS of 37% you are going to have serious problems with certain applications.

This happens a lot with certain providers and yet very few benchmarking sites help users diagnose these issues when this is what people should be looking at. For most windows based systems the above referenced connection will top out somewhere around 12-15/4.8 with a QoS of not less than 90% when running a single socket TCP test to be considered a good connection. There are ways to improve this depending on your operating system, distance between test site and obviously the quality of your network connection.

2) Capacity - This is more like the traditional speed tests which first calibrate based on connection speed, determining at what datarate the connections starts to drop packets and then open a specific number of sockets based on the calibration speed for the duration of the test. This test should ideally also measure QoS (or the connections ability to maintain a steady datarate for the duration of the test). This is the test that essentially behaves like modern browsers would.

It's critically important to know the results of both tests before making an evaluation of the connection quality and Ookla doesn't offer you both of these tests so any results they provided PC Mag would have been only marginally useful.

PC Mag was on the right track to begin with...they just did an awful job of explaining it to the public and press who apparently hassled them over it and apparently caved this year to the pressure...sad really because they had an opportunity to properly educate the public on internet connection benchmarking and blew it.

cork1958
Cork
Premium
join:2000-02-26

Charter?

Holy cow!!

Charter and speed mentioned in the same sentence!!

Actually,
My Charter connection is and always has been quite good.
On the 12/1 plan:


--
The Firefox alternative.
»www.mozilla.org/projects/seamonkey/

RR

@fiu.edu

Re: Charter?

Its not bad here at my uni




techlte

join:2003-01-01
US

Re: Charter?

It isn't bad here at my home either





--
WRTSL54GS v2 + WRT54G v2
ArizonaSteve

join:2004-01-31
Apache Junction, AZ
Reviews:
·voip.ms
·CenturyLink

2 recommendations

Ookla is a bad choice.

Ookla's Speedtest.net is not accurate and tends to give extra high speed readings which is why a lot of companies like Qwest use it. It would give me a speed of 1600k on my1500k DSL line that was only running about 640k. It gives insanely high readings on Virgin Mobile too. Speed.io is more accurate.

BHNtechXpert
BHN Staff
Premium,VIP
join:2006-02-16
Saint Petersburg, FL
kudos:151

Re: Ookla is a bad choice.

said by ArizonaSteve:

Ookla's Speedtest.net is not accurate and tends to give extra high speed readings which is why a lot of companies like Qwest use it. It would give me a speed of 1600k on my1500k DSL line that was only running about 640k. It gives insanely high readings on Virgin Mobile too. Speed.io is more accurate.

I agree with your Oookla statement however speed.io is far from being an accurate or reliable test. Yea its looks fancy but in my tests with multiple connections and providers Speed.io was an epic fail....sorry!

Speed.io is an excellent example of the ad farm speed test proliferation that has poluted the net recently adding further confusion to consumers. Lots of ads...worthless test.
--
~All truth goes through three phases. First, it is ridiculed. Second, it is violently opposed. Third, it is accepted as self-evident. - Einstein~

sniperboyz

join:2009-10-30
Portsmouth, VA
Reviews:
·Cox HSI

Here at Rochester Institute of Tech College.

Internet came from Time Warner Fiber Optics business with 400mbps maximum.

At Dorm:


At Classroom: