Topic 'Re: The test is inaccurate...' in forum '' - dslreports.com http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23939433 en Sat, 25 May 2013 11:41:28 EDT Sat, 25 May 2013 11:41:28 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23949735 http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23949735 Sun, 14 Mar 2010 16:40:24 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23949433 said by houkouonchi:


»www.la.bbb.org/Business-Report/N···13131294]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23949433 Sun, 14 Mar 2010 14:55:10 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23949402


I get much better results to various servers on speedtest.net:








--
20/2mb Charter, 18/1.5mb U-verse, 6mb/768k DSL Extreme load balanced for a total of 44/4.2]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23949402 Sun, 14 Mar 2010 14:43:03 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23948614 said by funchords:

said by AMDUSER:

I tested my cable internet on it, and it said I had about 30 Meg (even though I only have 7 Meg Road Runner, but with Powerboost(tm).

If they want accurate, they are going to have to do something to mitigate the Powerboost to get the rated service speed.
If it measured 30 meg, you want it to dismiss it?

The goal here is to see whether you're getting what you paid for.

If it says 3 Mbps and you're paying for 7 Mbps, then you're not.
If it says 7 Mbps or 30 Mbps and you're paying for 7 Mbps, then you are.

Powerboost is real speed. So some credit should be given to that fact. But it is really temporarily and not assured, and I agree that something should also measure that. But what we don't want measurement tools to do is to discard actual performance.
What I would like is for the speedtest to test for actual internet access speed.. as 30 Meg is not available locally.
[At least not without paying for a business class data circuit [ie fractional T3 / OC3 for $$$$.]]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23948614 Sun, 14 Mar 2010 10:15:00 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23947303 http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23947303 Sat, 13 Mar 2010 20:47:31 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23943152 said by Bill Dollar:

The FCC should take a cue from Offcom, and do a SamKnows style testing, which corrects for selection bias as well as all sorts of other intervening factors.
I agree that something like this is necessary to get an accurate picture. Whether we'll get there sooner or later is an open question. In either case, what we did yesterday is a start.
--
Robb Topolski -= funchords.com =- District of Columbia -- KJ7RL
Tweet! Tweet! -- »twitter.com/funchords]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23943152 Fri, 12 Mar 2010 19:13:44 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23942330 http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23942330 Fri, 12 Mar 2010 16:43:02 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23942141
From the office the Ookla test returns: 35M/3M
and M-Lab: 21M/4M
--
JL
Comcast]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23942141 Fri, 12 Mar 2010 16:12:00 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23941592
Speedtest.net is more along the line of what I'd expect, whereas broadband.gov's test showed about one third of our total download speed and one fortieth of our total upload speed.]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23941592 Fri, 12 Mar 2010 14:38:19 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23941478 said by Ericthorn:

If it's the same then why are the results so skewed? Ookla had inaccurate d/l data while speedtest was accurate.
Are they? I get similar results:

[att=1]
[att=2]

Not identical, but I wouldn't expect them to be. Which speedtest server did FCC choose for ookla test? It may not be the same. And how long the test runs for the FCC vs native speedtest.net may not be the same.
  
]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23941478 Fri, 12 Mar 2010 14:23:41 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23941242 http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23941242 Fri, 12 Mar 2010 13:50:26 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23941117 said by jlivingood:

Robb - You are the resident M-Labs expert... Do you know how many TCP connections that the M-Labs tool uses?

Also - do you have any idea how the test decides to use one test or the other (M-Labs or Ookla)?
The FCC web site says it picks it at random. But when the test ends, you can then run the other test.

»www.broadband.gov/qualitytest/about/
Users will be assigned to one of the two chosen testing tools: Ookla or M-Lab, or they can directly choose one of these tools at the links at the bottom of this page.
But you can pick which one you want from the above link or use the links I found below:

Ookla:
»www.broadband.gov/qualitytest/?sCode=O

M-Lab:
»www.broadband.gov/qualitytest/?sCode=M]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23941117 Fri, 12 Mar 2010 13:24:30 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23941044
Also - do you have any idea how the test decides to use one test or the other (M-Labs or Ookla)?
--
JL
Comcast]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23941044 Fri, 12 Mar 2010 13:12:36 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940969 said by fifty nine:

The problem is that the mlab test doesn't work with Firefox on Linux, which is what I'm using.

It also doesn't work with chrome on windows 7 which is what I also used.

If this test is IE only they're going to get a lot of skewed results. Not everyone wants to use IE.
From FCC:
Note: the M-Lab application currently does not work with Safari, Chrome, and Opera web browsers.

The ookla test should work:
Ookla also provides free services at their public web site Speedtest.net.
It is the same test as used at speedtest.net]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940969 Fri, 12 Mar 2010 13:00:36 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940927
It also doesn't work with chrome on windows 7 which is what I also used.

If this test is IE only they're going to get a lot of skewed results. Not everyone wants to use IE.]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940927 Fri, 12 Mar 2010 12:54:50 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940891
With all due respect, any real honest statictician would have a hard time using any of this data, because it violates basic rules of sample selection bias. There are ways to correct for that, but nothing that could render anything worthwhile from these self-selected tests.

The FCC should take a cue from Offcom, and do a SamKnows style testing, which corrects for selection bias as well as all sorts of other intervening factors.]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940891 Fri, 12 Mar 2010 12:49:42 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940623 Which test you see first is randomized, everyone should be sure to take BOTH tests, each time they visit the site, so any data collected is closer to reality.
That said it is a beta, and the location data is probably more important than the speed data at this point (if you don't feel comfortable using your own address, please use one nearby, so as not to skew the location HAVE/HAVE NOT part. It's not like you really are hidding using a fake/distant location, they can trace your IP if the really want too anyway.)]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940623 Fri, 12 Mar 2010 12:09:09 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940582 said by funchords:



The goal here is to see whether you're getting what you paid for.

I have to disagree. The goal for the Gov't speed test is to sample what speeds are available where.

If powerboost causes a test to show 30Mbps then it is a false result.]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940582 Fri, 12 Mar 2010 12:02:52 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940398
The speedtest.net site seems to do the best of all, though, since it looks for the best server for your location.]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940398 Fri, 12 Mar 2010 11:42:18 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940320 said by tschmidt:

I ran the test twice; the first time jitter and latency were the same, which is impossible. Transfer speed measurement seemed reasonable accurate.

Overall I think having FCC do its own research is laudable but there are several serious caveats:

1) This is by no means a scientifically credible poll of US broadband.

2) We, like many others, have home network. To be an accurate determination of ISP performance no other computer should be using the Internet for the duration of the test. I did not see any mention of that on the test page.

3) Likewise applications on the test PC should not access the Internet for the duration of the test.

4) Performance testing is end-to-end. Issues outside the control of the ISP can degrade performance.

5) As already mentioned, temporary speed increases, will artificially inflate test performance.

/Tom
All true. There are several more issues. That said, these are data points in a set of gathered data. Once analyzed, some test results will be statistical outliers. In the end, analysts should be able to refine and clarify the picture by filtering out much of the noise.
--
Robb Topolski -= funchords.com =- District of Columbia -- KJ7RL
Tweet! Tweet! -- »twitter.com/funchords]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940320 Fri, 12 Mar 2010 11:27:58 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940302
Overall I think having FCC do its own research is laudable but there are several serious caveats:

1) This is by no means a scientifically credible poll of US broadband.

2) We, like many others, have home network. To be an accurate determination of ISP performance no other computer should be using the Internet for the duration of the test. I did not see any mention of that on the test page.

3) Likewise applications on the test PC should not access the Internet for the duration of the test.

4) Performance testing is end-to-end. Issues outside the control of the ISP can degrade performance.

5) As already mentioned, temporary speed increases, will artificially inflate test performance.

/Tom]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940302 Fri, 12 Mar 2010 11:25:28 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940233 said by funchords:

Powerboost is real speed. So some credit should be given to that fact. But it is really temporarily and not assured, and I agree that something should also measure that. But what we don't want measurement tools to do is to discard actual performance.
If you want a test that handles distortions caused by powerboost, use this one:
»www.measurementlab.net/measureme···iffprobe]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940233 Fri, 12 Mar 2010 11:15:48 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940197 said by AMDUSER:

I tested my cable internet on it, and it said I had about 30 Meg (even though I only have 7 Meg Road Runner, but with Powerboost(tm).

If they want accurate, they are going to have to do something to mitigate the Powerboost to get the rated service speed.
If it measured 30 meg, you want it to dismiss it?

The goal here is to see whether you're getting what you paid for.

If it says 3 Mbps and you're paying for 7 Mbps, then you're not.
If it says 7 Mbps or 30 Mbps and you're paying for 7 Mbps, then you are.

Powerboost is real speed. So some credit should be given to that fact. But it is really temporarily and not assured, and I agree that something should also measure that. But what we don't want measurement tools to do is to discard actual performance.
--
Robb Topolski -= funchords.com =- District of Columbia -- KJ7RL
Tweet! Tweet! -- »twitter.com/funchords]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940197 Fri, 12 Mar 2010 11:09:52 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940004 said by Ericthorn:

Agreed - it's giving me about 2.7m down and any other test anywhere I go gives me less than 2m, usually in the 1.8m range which is accurate.

It wouldn't surprise me a bit if the carriers have some hand in this...
I agree. It is pretty inaccurate. It doesn't even match what regular speedtest says:

Here is my result from broadband.gov speedtest:



And here is regular speedtest:


There is no powerboost on my connection. ]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23940004 Fri, 12 Mar 2010 10:39:44 EDT Re: The test is inaccurate... http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23939779
It wouldn't surprise me a bit if the carriers have some hand in this...
--
Ever try stuffing a melted marshmallow up a wildcat's ass? It can be done, but you have to like your job. - This Is The Way The World Ends by James Morrow - Join a DC club, it can't hurt you!]]> http://www.dslreports.com/forum/Re-The-test-is-inaccurate-23939779 Fri, 12 Mar 2010 10:11:11 EDT The test is inaccurate... http://www.dslreports.com/forum/The-test-is-inaccurate-23939433
If they want accurate, they are going to have to do something to mitigate the Powerboost to get the rated service speed.]]> http://www.dslreports.com/forum/The-test-is-inaccurate-23939433 Fri, 12 Mar 2010 08:54:53 EDT