dslreports logo
 
    All Forums Hot Topics Gallery
spc
uniqs
43

funchords
Hello
MVM
join:2001-03-11
Yarmouth Port, MA

1 recommendation

funchords to tschmidt

MVM

to tschmidt

Re: The test is inaccurate...

said by tschmidt:

I ran the test twice; the first time jitter and latency were the same, which is impossible. Transfer speed measurement seemed reasonable accurate.

Overall I think having FCC do its own research is laudable but there are several serious caveats:

1) This is by no means a scientifically credible poll of US broadband.

2) We, like many others, have home network. To be an accurate determination of ISP performance no other computer should be using the Internet for the duration of the test. I did not see any mention of that on the test page.

3) Likewise applications on the test PC should not access the Internet for the duration of the test.

4) Performance testing is end-to-end. Issues outside the control of the ISP can degrade performance.

5) As already mentioned, temporary speed increases, will artificially inflate test performance.

/Tom
All true. There are several more issues. That said, these are data points in a set of gathered data. Once analyzed, some test results will be statistical outliers. In the end, analysts should be able to refine and clarify the picture by filtering out much of the noise.

Bill Dollar
join:2009-02-20
New York, NY

1 recommendation

Bill Dollar

Member

Rob,

With all due respect, any real honest statictician would have a hard time using any of this data, because it violates basic rules of sample selection bias. There are ways to correct for that, but nothing that could render anything worthwhile from these self-selected tests.

The FCC should take a cue from Offcom, and do a SamKnows style testing, which corrects for selection bias as well as all sorts of other intervening factors.

funchords
Hello
MVM
join:2001-03-11
Yarmouth Port, MA

funchords

MVM

said by Bill Dollar:

The FCC should take a cue from Offcom, and do a SamKnows style testing, which corrects for selection bias as well as all sorts of other intervening factors.
I agree that something like this is necessary to get an accurate picture. Whether we'll get there sooner or later is an open question. In either case, what we did yesterday is a start.