how-to block ads
|
NetAdmin
CCNA
join:2008-05-22
| reply to Dogfather
Re: Old data and biased source - the CWA who hate telecom mgt
said by Dogfather  :However, the results of Speedtest.net REFUTE absolutely the claims made in the CWA report. Sorry, but that is not the case. The measurements and results generated by Speedtest are neither rigorously controlled or reproducible with any reliability. So say that poorly controlled and processed data absolutely refutes another poor dataset is like saying one crazy man saying that aliens are green absolutely refutes the claims of another than aliens are purple.
And no I didn't miss the effect of IP changes on Speedtest.net's results. I clearly explained why your point is irrelevant in this instance and why IP changes make the result we would be trying to find (max line speed) more accurate. My mathematics skills are just fine. Please learn to read. My reading skills are fine... Your experimental design, measurement accuracy and statistics skills are in need of a refresher.
I have clearly explained why focusing on max speed not only creates an overly rosy picture of the state of broadband, but also affects the value of the data. Additionally, your methodology is deeply flaw because relevant data is excluded arbitrary on the flawed assumption of how IP addresses are issued.
Huh? Finding out what speeds are available is exactly how you represent the state of broadband. If everyone is just buying 1Mb plans but everyone can get 100Mb, median speed test results aren't an accurate indicator as to the state of anything other than the frugality of users. That requires the assumption that broadband speeds are uniform over a given area. Not true.
The IP changes of Speedtest.net lend themselves to skewing toward the maximum line speeds available in that geographic area for that particular ISP. I'll rehash this fact since you ignored it and it counters your assertion that using IP is a suitable way to determine location. That assumes that all providers hand out IP addresses into discrete geographic areas, not always true, especially with larger providers. I've seen numerous instances of end users on PPPoE pulling IPs from a POP three states over because of network conditions. Additionally, even in cable networks, it is possible for users in one town to pull an IP that was in use in another town/zip code/etc. due to the design of the network. So only using the highest speedtest result from an IP creates an inaccurate picture of broadband purchased or available. Period.
People don't choose their income. People choose what tier they buy. Missed the point again...
Your reading comprehension is lacking. Look at what the line before that says. said by Speedtest.net FAQ :
We first calculate the 95th percentile speed in each direction for every unique IP address that has tested at Speedtest.net. These numbers are then averaged together for each geographic level (both overall and per ISP).
They take the 95th percentile of everyone's top speed, and THEN average them. They don't average every result as you originally claimed. They only take the top speed result from every IP for this computation.
Given your poor reading comprehension I give little weight to what you claimed to have read. See below...
Wow, and you accuse someone else of having poor reading comprehension.
I stated that the raw data has some use and that was all. I never defended the study, the conclusions or the analysis, but thanks for making stuff up that is not there.
Before accusing others of having poor reading skills, you should make sure that yours are up to snuff and that you are not making crap up. Distorting statements to make your point, that's a new low.
My reading comprehension is just fine and I have in depth knowledge in data collection, the scientific method, P&S and how error rates affect conclusions. As evidenced by what you have posted, the evidence casts serious doubt on that statement.
The fact that you rigorously defend Speedtest, despite its data being trash due to poor control and test uniformity and only attack that Speedmatters data, which is also trash, betrays your real motive - the CWA disagrees with your viewpoint.
It is sad... I would figure that you might actually be interested in providing an accurate picture of the status of broadband in this country. As evidenced by your posts, that is not the case.
Your mind was made up before the discussion started and there is no changing it, no matter what, regardless of the data or problems with it. I'm not going to waste any more time on this, especially if you are going to resort to distortion...
Good day.
--
---
Over ten plus years of carrying The Clue Bat... | |
Dogfather
Premium
join:2007-12-26
Laguna Hills, CA
 ·Cox HSI
 ·Verizon FIOS
 ·Cox VOIP
·ViaTalk
·RoadRunner Cable
 ·MegaPath
·Verizon west (ex G..
·Time Warner VOIP
3 edits | reply to NetAdmin
said by netAdmin :
What the CWA reports says has NO BEARING on the accuracy of the data presented by Speedtest.net. PERIOD. It is totally and completely unrelated to the quality of the Speedtest.net data.
Two, you completely and totally missed the point of IP changes and the affect they have on Speedtest.net's data. Your response illustrates that math and science education in this country are in SERIOUS need of help because you response COMPLETELY misses the point I made and fails to even address it.
Huh? No one is saying that the CWA report had bearing on the accuratcy of Speedtest.net. However, the results of Speedtest.net REFUTE absolutely the claims made in the CWA report.
And no I didn't miss the effect of IP changes on Speedtest.net's results. I clearly explained why your point is irrelevant in this instance and why IP changes make the result we would be trying to find (max line speed) more accurate. My mathematics skills are just fine. Please learn to read.
said by netAdmin :
Yes, and every state is reported as MEDIAN download speed... Do you even know what the median is or how to compute it? Do you even know what the limitations and uses of the median are? Do you know why and when the median should be used? Do you even know what the other measures of central tendency are?
You are completely missing the point AGAIN. What the CWA report says has no bearing on the legitimacy of their data or the data presented at Speedtest.net.
Median DL speed is irrelevant. First, their data isn't accurate. And 2, their data only reflects what people buy, not what is available to them. Yes, I know what median is, I have a B.A. in Mathematics.
said by NetAdmin :
No, the issue is about accurately representing the state of broadband in America. If you rely purely on the maximum line speed, you do not get an accurate representation.
Huh? Finding out what speeds are available is exactly how you represent the state of broadband. If everyone is just buying 1Mb plans but everyone can get 100Mb, median speed test results aren't an accurate indicator as to the state of anything other than the frugality of users. The IP changes of Speedtest.net lend themselves to skewing toward the maximum line speeds available in that geographic area for that particular ISP.
said by NetAdmin :
That's like taking the highest incomes in a county and saying that represents the state of the American household's income...
People don't choose their income. People choose what tier they buy.
said by netAdmin :
It says right on their site that they do averaging...
Your reading comprehension is lacking. Look at what the line before that says.
said by Speedtest.net FAQ :
We first calculate the 95th percentile speed in each direction for every unique IP address that has tested at Speedtest.net. These numbers are then averaged together for each geographic level (both overall and per ISP).
They use the 95th percentile because they assume that the maximum speed received isn't going to be received all the time. 95th percentile calculations for maximum throughput are common in the networking industry. The calculation is used to set a point at which 95% of results will be lower. Again, totally accurate for comparing maximum line speeds. They DO NOT average all of the speeds together as you originally claimed.
said by Speedtest.net :
We first calculate where your download speed ranks, and then we do the same for upload. The overall rank is the average of the two. This is done at both the global and country levels. Only your best test scores are used for the ranking. Note that ranks done per IP address. So if you IP address changes, your rank will change as well. You can look at results for past IP addresses though to see the rank of each.
They take the 95th percentile of everyone's top speed, and THEN average them. They don't average every result as you originally claimed. They only take the top speed result from every IP for this computation.
said by NetAdmin :
From what I've read, you don't have the necessary training or qualifications to make the determination of which groups methodology is more thorough and accurate.
Given your poor reading comprehension I give little weight to what you claimed to have read.
said by NetAdmin :
Actually, you won't find a single statement where I defend the outcome and conclusions of the CWA, not one. I do not, nor have ever, been a fan of the CWA.
No, you just defended the study. »Re: Old data and biased source - the CWA who hate telecom mgt
said by NetAdmin :
In my case, one test was accurate, out of the FIVE I tried... Meaning, for other people who don't try five different tests, their first and only test may not be accurate.
Additionally, 20% accuracy is crap. If I took more tests, the accuracy would fall even further.
Your statement operates on the assumption that everyone will test their connection on multiple sites, trying to get the highest speed. Simply not the case.
I already explained once why this error doesn't matter when we're looking for maximum line speed. But I'll explain it again. READ THE SPEEDTEST.NET FAQ. They only take the TOP SPEED RESULT from an IP. So in your example they would have only used your TOP SPEED RESULT in their calculations. Thus in your example, if Speedtest data was used for this study, your contribution would have been 100% accurate. And those users only doing one test are mitigated by others doing multiple tests, because they take the 95th percentile average top speed. And even if what you say is true, it only means that the state of broadband is BETTER than what the Speedtest.net data indicates.
said by NetAdmin :
Did you even understand what their collection process is? Are you even aware of the issues with their collection process? Do you even understand the limitations of the tools and processes that they use? If you don't, they how can you defend them?
I do. It is you who can't comprehend their FAQ.
said by NetAdmin :
Actually, I have read the CWA report. Like I said, and I will say it again, their basic design is good, their execution of that design sucked. While I may agree with the statement that US broadband is not up to par with other nations, it was not at all founded on the data in the CWA report, but rather, real-world, in the trenchs experience.
And, no, I have never been a member of the CWA.
Their basic design is horrid for a number of reasons.
1. Their test servers sucked ass and weren't collecting accurate data.
2. They draw assumptions about dial up users while ignoring the inability of their test servers to deliver accurate results.
3. Speeds recorded do not reflect the "state of broadband in America". Even with 100% accuracy in data collection, it only reports what people are BUYING, not what they can get. Again, if people CHOOSE to buy 1Mb service while 100Mb service is available, a bunch of 1Mb results do not reflect the state of broadband.
Remember, the CWA's conclusion is that broadband performance is lacking. And even with perfect data collections that conclusion one way or another couldn't be reached by their collection and computational methods.
said by NetAdmin :
Right, because you say it is, despite illustrating that you have no understanding of the methods used by either group and are completely oblivious to the flaws and issues in each method, especially the flaws in the Speedtest.net methodology. Before you claim that others lack objectivity, perhaps you need to step back and realize that you have illustrated a lack of objectivity and a lack of knowledge of basic research methods, testing methods and data analysis.
Again, it's you who doesn't understand the Speedtest.net FAQ or how they're calculating data. You also don't get how changing IPs would improve statistics about the state of broadband using the Speedtest data collection methodology.
My reading comprehension is just fine and I have in depth knowledge in data collection, the scientific method, P&S and how error rates affect conclusions.
The CWA report is nothing but propaganda and Speedtest.net is a far superior source of this data.
With that, I'm finished feeding you. You'll have to troll elsewhere. | |
NetAdmin
CCNA
join:2008-05-22
| reply to Dogfather
said by Dogfather :OMFG, you're kidding right? Did you bother to read the CWA report and what their case was? They're making the case that US broadband sucks and needs improvement. Again, you outsmart yourself. Swapping IPs and getting better results only serves to PROVE that fast broadband is available. IP issuances are regional which is how they know your location (Go google GeoIP). Maximum throughput results aren't skewed by a bunch of cheapskates who don't want to pony up for the fastest speeds available to them. The more IP swaps that happen, the MORE accurate the results. What the CWA reports says has NO BEARING on the accuracy of the data presented by Speedtest.net. PERIOD. It is totally and completely unrelated to the quality of the Speedtest.net data.
Two, you completely and totally missed the point of IP changes and the affect they have on Speedtest.net's data. Your response illustrates that math and science education in this country are in SERIOUS need of help because you response COMPLETELY misses the point I made and fails to even address it.
Again, did you even bother to read the CWA report? This ain't about test server averages or what they buy. Yes, and every state is reported as MEDIAN download speed... Do you even know what the median is or how to compute it? Do you even know what the limitations and uses of the median are? Do you know why and when the median should be used? Do you even know what the other measures of central tendency are?
You are completely missing the point AGAIN. What the CWA report says has no bearing on the legitimacy of their data or the data presented at Speedtest.net.
No, the issue is about accurately representing the state of broadband in America. If you rely purely on the maximum line speed, you do not get an accurate representation.
That's like taking the highest incomes in a county and saying that represents the state of the American household's income...
More data points is ALWAYS better, especially when you're looking to compute maximum line speed. You've missed the point again...
quote:
This coming from the guy who thought Speedtest.net's averaged all results. Nice try genius.Perhaps you should read Speedtest.net AGAIN:
How are the Global Stats averages calculated?
We first calculate the 95th percentile speed in each direction for every unique IP address that has tested at Speedtest.net. These numbers are then averaged together for each geographic level (both overall and per ISP). It says right on their site that they do averaging...
If you think that the CWA methodology is ANYWHERE NEAR as thorough, accurate or complete as speedtest.net you're simply a shill. From what I've read, you don't have the necessary training or qualifications to make the determination of which groups methodology is more thorough and accurate.
Again, in your haste to defend the CWA's finding, you forget what the question was. What is the max speed of these lines. And in that measurement, Speedtest.net is comprehensive, and accurate. Actually, you won't find a single statement where I defend the outcome and conclusions of the CWA, not one. I do not, nor have ever, been a fan of the CWA.
The data collection was dead accurate in your case. And you can't draw any comparison between Speedtest and the CWA. Speedtests collection methods are hugely better, far more accurate and given they don't have a vested interest in their results, far more trust worthy. In my case, one test was accurate, out of the FIVE I tried... Meaning, for other people who don't try five different tests, their first and only test may not be accurate.
Additionally, 20% accuracy is crap. If I took more tests, the accuracy would fall even further.
Your statement operates on the assumption that everyone will test their connection on multiple sites, trying to get the highest speed. Simply not the case.
Speedtest.net's results appear to make sense because their results are accurate simply because the collection methods are more involved and more accurate. And yes, they're absolutely idiots. If you had read their report, you would know that. Did you even understand what their collection process is? Are you even aware of the issues with their collection process? Do you even understand the limitations of the tools and processes that they use? If you don't, they how can you defend them?
You need to relax, reread the article, reread the CWA report and then come to an unbiased conclusion as to who is more accurate, and more importantly, if the conclusion in the CWA report is well founded. Actually, I have read the CWA report. Like I said, and I will say it again, their basic design is good, their execution of that design sucked. While I may agree with the statement that US broadband is not up to par with other nations, it was not at all founded on the data in the CWA report, but rather, real-world, in the trenchs experience.
And, no, I have never been a member of the CWA.
Do I believe that the CWA report is accurate? For their dataset it is, but their dataset is junk.
Do I believe that the Speedtest.net numbers are accurate? Their data collection is problematic and so are their computational methods. The dataset is very expansive, but it is processed poorly.
Are either worth anything? Strictly speaking, if you want to draw a scientific conclusion from either dataset, no. Both the Speedtest and Speedmatters data are JUNK, TRASH, CRAP, etc. And I have said that since square one.
You lack objectivity and now you're letting your ego argue on your behalf.
In summary, the CWA report was total bullshit propaganda. And Speedtest.net is a far better indicator as to the state of US broadband. Right, because you say it is, despite illustrating that you have no understanding of the methods used by either group and are completely oblivious to the flaws and issues in each method, especially the flaws in the Speedtest.net methodology. Before you claim that others lack objectivity, perhaps you need to step back and realize that you have illustrated a lack of objectivity and a lack of knowledge of basic research methods, testing methods and data analysis. | |
Dogfather
Premium
join:2007-12-26
Laguna Hills, CA
·Cox HSI
·Verizon FIOS
·Cox VOIP
·ViaTalk
·RoadRunner Cable
·MegaPath
·Verizon west (ex G..
·Time Warner VOIP
4 edits | reply to NetAdmin
said by NetAdmin :
Wrong... Again. And with the new information, we cn see again that their methodology is deeply flawed in that it only works if EVERY host that use Speedtest.net has a static IP... IF they are doing what their FAQ states, what happens when an IP address used by a person with a 8/768 Comcast connection is issued to another Comcast subscriber with a lower speed connection?
Here's the answer: The person with the lower speed service's numbers may be tossed out, despite being a valid test. Ergo, their methodology is seriously flawed.
OMFG, you're kidding right?
Did you bother to read the CWA report and what their case was? They're making the case that US broadband sucks and needs improvement. Again, you outsmart yourself. Swapping IPs and getting better results only serves to PROVE that fast broadband is available. IP issuances are regional which is how they know your location (Go google GeoIP). Maximum throughput results aren't skewed by a bunch of cheapskates who don't want to pony up for the fastest speeds available to them. The more IP swaps that happen, the MORE accurate the results.
said by NetAdmin :
"Truncating the slowest results" = eliminating valid data points. In other words, their numbers are actually higher than they should be.
Again, did you even bother to read the CWA report? This ain't about test server averages or what they buy.
said by NetAdmin :
Seriously, statistics and science need to be stressed more in this country... More datapoints are not always better, especially when you message the numbers in a method that can throw out valid data points. Now that you've posted the excerpt from the Speedtest.net site, their numbers are even MORE suspect because their methodology is not statistically OR scientifically sound. They are arbitrarily tossing out data points without, it appears, understanding the consequence for doing so or fully understanding the reasons for the variability within the data.
More data points is ALWAYS better, especially when you're looking to compute maximum line speed.
said by NetAdmin :
Certainly not as laughable as believing that the Speedtest.net numbers are any better... What is laughable is that you are trying to argue on a topic that don't seem to have very much expertise on, otherwise you would have spotted the problem with Speedtest.net's methods.
This coming from the guy who thought Speedtest.net's averaged all results. Nice try genius. If you think that the CWA methodology is ANYWHERE NEAR as thorough, accurate or complete as speedtest.net you're simply a shill.
said by NetAdmin :
And Speedtest.net uses non-uniform setups that can affect the raw data compounded by seriously flawed data collection, aggregation and processing to arrive at numbers that in actuality do not accurately represent the sample due to the arbitrary removal of low performing tests based.
The claim that Speedtest.net is more accurate and more representative doesn't hold water if you understand why their methods are flawed.
Again, in your haste to defend the CWA's finding, you forget what the question was. What is the max speed of these lines. And in that measurement, Speedtest.net is comprehensive, and accurate.
said by NetAdmin :
Only if you trust Speedtest.net's flawed dataset and analysis methods... Based on how both Speedtest and Speedmatters get their results, statistically speaking, neither is trustworthy enough to draw that conclusion or, in reality, any conclusions that are sound.
The data collection was dead accurate in your case. And you can't draw any comparison between Speedtest and the CWA. Speedtests collection methods are hugely better, far more accurate and given they don't have a vested interest in their results, far more trust worthy.
said by NetAdmin :
You've shown that you don't have a grasp or the qualifications to understand why the data at both sites is flawed. For you to call someone else an idiot when you seem to be illustrating the same lack of understanding is astounding. Just because Speedtest.net's processing of test data is more involved and appears to make sense, does not mean that it is that way.
Speedtest.net's results appear to make sense because their results are accurate simply because the collection methods are more involved and more accurate. And yes, they're absolutely idiots. If you had read their report, you would know that.
You need to relax, reread the article, reread the CWA report and then come to an unbiased conclusion as to who is more accurate, and more importantly, if the conclusion in the CWA report is well founded.
said by NetAdmin :
Anything else?
Yeah, you've dug yourself such a huge hole defending the CWA and attacking Speedtest.net (while obviously not knowing anything about their collection methods and computational methods until I told you), my guess is you will continue struggling to defend the CWA with more guesses and conjecture. You lack objectivity and now you're letting your ego argue on your behalf.
In summary, the CWA report was total bullshit propaganda. And Speedtest.net is a far better indicator as to the state of US broadband. | |
NetAdmin
CCNA
join:2008-05-22
| reply to Dogfather
said by Dogfather : said by Speedtest.net :
How are global and country ranks calculated?
We first calculate where your download speed ranks, and then we do the same for upload. The overall rank is the average of the two. This is done at both the global and country levels. Only your best test scores are used for the ranking. Note that ranks done per IP address. So if you IP address changes, your rank will change as well. You can look at results for past IP addresses though to see the rank of each.
So Speedtest.net would have reported PERFECT accuracy in your case. Wrong... Again. And with the new information, we cn see again that their methodology is deeply flawed in that it only works if EVERY host that use Speedtest.net has a static IP... IF they are doing what their FAQ states, what happens when an IP address used by a person with a 8/768 Comcast connection is issued to another Comcast subscriber with a lower speed connection?
Here's the answer: The person with the lower speed service's numbers may be tossed out, despite being a valid test. Ergo, their methodology is seriously flawed.
They only look at the top speeds of each IP, ISP, etc. to generate their max throughput data. They're mitigating errors by truncating the slowest results. "Truncating the slowest results" = eliminating valid data points. In other words, their numbers are actually higher than they should be.
Not perfect, but the more samples they collect, the more accurate their maximum line speed data estimates. And they collect a LOT of samples, over 410 MILLION so far and it's increasing by more than 10 samples per second. Seriously, statistics and science need to be stressed more in this country... More datapoints are not always better, especially when you message the numbers in a method that can throw out valid data points. Now that you've posted the excerpt from the Speedtest.net site, their numbers are even MORE suspect because their methodology is not statistically OR scientifically sound. They are arbitrarily tossing out data points without, it appears, understanding the consequence for doing so or fully understanding the reasons for the variability within the data.
So you still think that the CWA methods are just as accurate and comprehensive as Speedtest.net's in collecting data about maximum line speeds? That would be utterly laughable. Certainly not as laughable as believing that the Speedtest.net numbers are any better... What is laughable is that you are trying to argue on a topic that don't seem to have very much expertise on, otherwise you would have spotted the problem with Speedtest.net's methods.
The CWA uses flawed collection methods, flawed data, unsupported assumptions, compounded by flawed logic to draw the conclusion that US broadband sucks and someone needs to spend money (with them) to fix it. Their conclusions aren't worth the paper it's printed on. It is simply propaganda. And Speedtest.net uses non-uniform setups that can affect the raw data compounded by seriously flawed data collection, aggregation and processing to arrive at numbers that in actuality do not accurately represent the sample due to the arbitrary removal of low performing tests based.
The claim that Speedtest.net is more accurate and more representative doesn't hold water if you understand why their methods are flawed.
US broadband, much to the disappointment of the CWA, holds it own when compared to other nations. Only if you trust Speedtest.net's flawed dataset and analysis methods... Based on how both Speedtest and Speedmatters get their results, statistically speaking, neither is trustworthy enough to draw that conclusion or, in reality, any conclusions that are sound.
The only thing the CWA showed is that they're idiots when it comes to scientific method and that their test servers suck ass. You've shown that you don't have a grasp or the qualifications to understand why the data at both sites is flawed. For you to call someone else an idiot when you seem to be illustrating the same lack of understanding is astounding. Just because Speedtest.net's processing of test data is more involved and appears to make sense, does not mean that it is that way.
Anything else? | |
Dogfather
Premium
join:2007-12-26
Laguna Hills, CA
·Cox HSI
·Verizon FIOS
·Cox VOIP
·ViaTalk
·RoadRunner Cable
·MegaPath
·Verizon west (ex G..
·Time Warner VOIP
4 edits | reply to NetAdmin
said by NetAdmin :
And then, here is the kicker, each of those results is going to be used by Speedtest.net to generate its numbers.
WRONG! Read their FAQ. If that is what you thought, then I can see why you would have questioned it.
said by Speedtest.net :
How are global and country ranks calculated?
We first calculate where your download speed ranks, and then we do the same for upload. The overall rank is the average of the two. This is done at both the global and country levels. Only your best test scores are used for the ranking. Note that ranks done per IP address. So if you IP address changes, your rank will change as well. You can look at results for past IP addresses though to see the rank of each.
So Speedtest.net would have reported PERFECT accuracy in your case.
Your measured line speed of 9742/546 is more than your advertised speed of 9000/512 and likely close to your maximum throughput. And they do the same for me.
Then you may ask, what if you didn't do the 2nd test? Good question. Wish I had thought of it 
said by Speedtest.net :
How are the Global Stats averages calculated?
We first calculate the 95th percentile speed in each direction for every unique IP address that has tested at Speedtest.net. These numbers are then averaged together for each geographic level (both overall and per ISP).
They only look at the top speeds of each IP, ISP, etc. to generate their max throughput data. They're mitigating errors by truncating the slowest results and by taking the 95th percentile, mitigate nonsense highest results. Not perfect, but the more samples they collect, the more accurate their maximum line speed data estimates (because in your case, more samples gave them the correct data). And they collect a LOT of samples, over 410 MILLION so far and it's increasing by more than 10 samples per second. Also of important note that you can look at subsets of all this calculated data if you weren't interested in "Global" averages. You can go all the way down to ISP data on a city basis levels if you want.
So you still think that the CWA methods are just as accurate and comprehensive as Speedtest.net's in collecting data about maximum line speeds? That would be utterly laughable.
--
The CWA uses flawed collection methods, flawed data, unsupported assumptions, compounded by flawed logic to draw the conclusion that US broadband sucks and someone needs to spend money (with them) to fix it. Their conclusions aren't worth the paper it's printed on. It is simply propaganda.
US broadband, much to the disappointment of the CWA, holds it own when compared to other nations.
The only thing the CWA showed is that they're idiots when it comes to scientific method and that their test servers suck ass. | |
NetAdmin
CCNA
join:2008-05-22
| reply to Dogfather
Because I and many others here did testing there and the results were horribly inaccurate. Their speed claims are based on results from their own test servers. Then, in reality, what your statement should be is not that it is "comprehensively inaccurate", simply that based on what you've seen, its results are suspect. You can't draw the conclusion of total inaccuracy from a few tests.
said by Dogfather :Then they applied THEIR speed test capacity to be the speed of those HSI subscribers lines. Their testing methods and server capacity doesn't allow for this translation. Actually, their testing method was a sound design, their execution was poor. They simply required more testing servers of a uniform setup, something that sites like Speedtest.net can not say they provide.
For example, for me, they reported a DL speed of 10-15Mb. However I have 30Mb service and routinely get 30Mb service from other test sites (like those on speedtest.net). So in their report they would say that my speed is 15Mb, when it's certainly not. And so you are going to trust the Speedtest.net because it gave you a better result, because basically that is your premise. What is interesting is that the Speedtest.net results I get from different sites vary so widely, that you can't trust the data they can give you (the aggregate performance numbers they provide).
On my 9Mbps/512kbp line, I got the following results from Speedtest sites:
From the server "closest" to me at a whopping 169ms:2286/540
From a server "further" away, but actually close: 9742/546
From a server near the second test: 3226/528
1511/483
3676/521
4844/514
Tell me again how Speedtest.net is more accurate? That's not even mentioning the variability of the results.
And then, here is the kicker, each of those results is going to be used by Speedtest.net to generate its numbers.
As well, there is no filtering of large corporate or University connections from the dataset, so that further poison's the data they provide. For example, look at:
»www.speedtest.net/global.php?con···ountry=1
I would also say it reflects a need for them to get educated in scientific method, particularly, observation, uncertainly and most importantly correlation from ANOTHER SOURCE OF DATA. There are not other sources of data that can be trusted to correlate their data to.
Understand, I'm not taking issue with your statement that their data can't be trust, what I am taking exception with is the method used by DSLR members to come to that conclusion and the belief that somehow Speedtest.net provides better data (which I've demostrated above, does not). Speedmatters basic design was sound - you have one setup so that testing conditions are uniform - however, they executed poorly, very poorly - the server should have located in one of the four or five major peering locations in the US. Additionally, I'm taking exception with the conclusion that their data sucks based on comparisons to even more problematic data, as provided by Speedtest.net. | |
NetAdmin
CCNA
join:2008-05-22
| reply to iansltx
said by iansltx :Oh, and Data393 DOES have multiple gigabits of connectivity to the internet. Their website says so. Seeing as how they're a colocation\server facility I wouldn't be one to doubt that. So no, that "assumption" is not borne out. Actually, yes there are still plenty of assumptions... What connectivity to their providers core network does the server have? What types of tweaks have been performed to Apache to improve performance that affect the validity of speedtests have been made? And those are just two you have to know before you can trust the reliability of the data you are presented with.
You can believe that the Speedtest.net tests are somehow more scientific or better than Speedmatters, but that is anything but true. The fact is, Speedtest.net data is based on non-uniform setups that aren't controlled by one group of people, ergo are not better than a server that may be overloaded. Period. | |
NetAdmin
CCNA
join:2008-05-22
| reply to iansltx
said by iansltx :Go flame somewhere else. Maybe take a look at an XKCD comic or two. Clearly you need to learn what a flame is...  | |
iansltx
join:2007-02-19
Golden, CO
·Comcast
·Qwest.net
·magicjack.com
 ·BeeCreek Communica..
·Sprint Mobile Broa..
| reply to NetAdmin
Oh, and Data393 DOES have multiple gigabits of connectivity to the internet. Their website says so. Seeing as how they're a colocation\server facility I wouldn't be one to doubt that. So no, that "assumption" is not borne out. On several occasions (using a college link) I've gotten 75 Mbit/s both ways on the Data393 Speedtest.net server, an order of manitude more than SpeedMatters would likely support. Don't worry though,Speedtest.net filters university results out of their equation when it comes t internet speeds. | |
iansltx
join:2007-02-19
Golden, CO
·Comcast
·Qwest.net
·magicjack.com
·BeeCreek Communica..
·Sprint Mobile Broa..
| reply to NetAdmin
Okay, here's the reason I won't take SpeedMatters' results, and this has been discussed at length in other posts: server overcrowding, such that an accurate speed test cannot be taken. Any connection faster than a few megabits is capped off artificially by the fact that the server can only handle a few hundred megabits of data pouring in at the same time...or 45 in the case of the west coast server.
Also, even if the SpeedMatters results WERE accurate, they were from a year-plus ago. I know my ISP has increased speeds by a third since then, as have a lot of other providers.
Go flame somewhere else. Maybe take a look at an XKCD comic or two. | |
Dogfather
Premium
join:2007-12-26
Laguna Hills, CA
·Cox HSI
·Verizon FIOS
·Cox VOIP
·ViaTalk
·RoadRunner Cable
·MegaPath
·Verizon west (ex G..
·Time Warner VOIP
4 edits | reply to NetAdmin
quote:
And you know this how? Do you have documentation or articles or anything that show how their data is "comprehensively inaccurate"?
Because I and many others here did testing there and the results were horribly inaccurate. Their speed claims are based on results from their own test servers.
quote:
I've learned that when people say that on this site, it means that the topic is question don't reflect reality as you see it. Unless they purposely fabricated data, those results are valid for the tests that were conducted.
Uh, no. There is only 1 reality. Their test was to find out the speed of those individual connections for the purpose of reporting average user speeds for these states. And if those numbers are low they indicated need for improvements in US broadband infrastructure.
But the singular reality was their results don't reflect the speed of those connections, only the speed at which their test servers were able to serve those connections. Quite 2 different things.
Then they applied THEIR speed test capacity to be the speed of those HSI subscribers lines. Their testing methods and server capacity doesn't allow for this translation.
For example, for me, they reported a DL speed of 10-15Mb. However I have 30Mb service and routinely get 30Mb service from other test sites (like those on speedtest.net). So in their report they would say that my speed is 15Mb, when it's certainly not.
Their testing methods were seriously flawed thus all the conclusions made from this erroneous data, bogus.
Their results indicate not a need for national infrastructure improvement, only a desperate need of improvement in their own server capacity.
I would also say it reflects a need for them to get educated in scientific method, particularly, observation, uncertainly and most importantly correlation from ANOTHER SOURCE OF DATA.
But alas, they aren't interested in actual research or reaching scientific truth. They're a biased propaganda outfit and pick and choose data to further that agenda. Unbiased sources like speedtest.net who have partner servers all over the world indicate the US is in a very favorable position in terms of average reported speed. | |
NetAdmin
CCNA
join:2008-05-22
| reply to iansltx
said by iansltx :Customers' IPs wouldn't change while they're shopping servers for a better result. You are basing that statement on what? When an IP lease or PPPoE session expires, it expires.
About SpeedMatters, 300 Mbit to one server and 45 to the other are NOT enough to give accurate speed test results, particularly if the servers are only homed into one network, and I'm betting that the 45 Mbit server was. Considering you don't know the Speedmatters setup, trying to make arguments like this one is pointless.
Compare this with places like Data393 where they have multi-gigabit links available and you'll see that Speedmatters will have an obvious bias to the lower end on speeds. Actually, you can't make that assumption based on the information you have. There are far too many assumptions that have to be made.
Also, if Speedmatters had only two servers for speed tests overall, versus a few dozen that speedtest.net has (through various providers) then if they had enough tests to get "accurate" results the servers woul be too overloaded to give accurate results. How many is enough tests? You assume that you have to have enough tests that Speedmatters servers would be overloaded. Statistics says otherwise.
Also, are you saying that no trends can be taken from 400 MILLION tests, a lot of which are in the US, and that Speedmatters' number of tests is valuable for trending information though the same number of tests is taken by Speedtest.net every 1.5 hours or so? I don't buy it. You don't buy it because you don't know how to digest, process and interpret the data that is being present by either Speedmatters OR Speedtest.net. Neither dataset it very good, period. Trends can be taken, but because of the lack of control that Speedtest.net has over test servers, sites and the connectivity (there is no standard setup), their data has more issues. The Speedmatters setup is uniform and is less affected by site variability. If that concept escapes you, there is no reason to proceed since you've already made your mind up and won't change it, regardless of the data and reasons presented.
How about using BBR's speed test results, does that work better for ya? Even that data is suspect since the connectivity and setup for each server varies. | |
iansltx
join:2007-02-19
Golden, CO
·Comcast
·Qwest.net
·magicjack.com
·BeeCreek Communica..
·Sprint Mobile Broa..
| reply to NetAdmin
Customers' IPs wouldn't change while they're shopping servers for a better result.
Also, I said *some* normalization, not *total* normalization.
About SpeedMatters, 300 Mbit to one server and 45 to the other are NOT enough to give accurate speed test results, particularly if the servers are only homed into one network, and I'm betting that the 45 Mbit server was. Compare this with places like Data393 where they have multi-gigabit links available and you'll see that Speedmatters will have an obvious bias to the lower end on speeds.
Also, if Speedmatters had only two servers for speed tests overall, versus a few dozen that speedtest.net has (through various providers) then if they had enough tests to get "accurate" results the servers woul be too overloaded to give accurate results.
Also, are you saying that no trends can be taken from 400 MILLION tests, a lot of which are in the US, and that Speedmatters' number of tests is valuable for trending information though the same number of tests is taken by Speedtest.net every 1.5 hours or so? I don't buy it.
How about using BBR's speed test results, does that work better for ya? | |
NetAdmin
CCNA
join:2008-05-22
| reply to iansltx
said by iansltx :Interesting, because they do a decent job of locating me: in the city I live near. And? The point is that the method used for getting that information does not accurately determine position - zip code, etc. IP location services that attempt to guess your location like that have serious issues. All of those services that guess your location based on IP do a decent job estimating your location, but they are nowhere near accurate enough to be relied upon for zip, street, etc. information.
Also, shopping servers is fine. To get a true speed test result ou pick the server with the most capacity possible. I'm guessing that's why Speedtest.net doesn't put latency in the graphs
They also keep track of the IP doing the speed test, so they can do some normalization. That would be fine if all the ISPs gave their customers static IPs... As it is, most do not just hand out statics to residential customers, including the one I work for, so normalization is useless.
Additionally, is there really a better alternative than Speedtest.net's stats? Theirs is a TON more scientific than SpeedMatters. Not even close. Neither is scientific. Speedmatters, however, controls their test servers. Speedtest does not control all of the servers used for speed testing.
On the one hand you have 400+ million tests, and on the other 40,000, a number that isn't even statistically significant given the number of people using the internet in the US. Granted, Speedtest.net isn't US-only but I'm betting 100+ million of its results are. So yes, it is much mroe scientific to look at their results than Speedmatters'. Sorry, more data does not necessarily mean better data, especially when no quality control is performed on that data.
Responding to another post, Norway probably can't get ultra high speed access everywhere, and if it does, I'd hate to see how much is going from taxpayers' pockets to subsidize the access. And that is different than the US how? I will say that, at least in countries like Norway, people are willing to invest tax money into infrastructure. I can think of a dozen or so infrastructure projects in this country that need to be funded but aren't for whatever reason (the money probably went to something stupid like tax breaks for some special group or a museum on the preparation of pigs' feet in BFE). | |
NetAdmin
CCNA
join:2008-05-22
| reply to Dogfather
said by Dogfather :Comprehensively inaccurate. Their data testing methods are seriously flawed and data worthless. And you know this how? Do you have documentation or articles or anything that show how their data is "comprehensively inaccurate"?
The testing results do not reflect reality. I've learned that when people say that on this site, it means that the topic is question don't reflect reality as you see it. Unless they purposely fabricated data, those results are valid for the tests that were conducted. | |
Dogfather
Premium
join:2007-12-26
Laguna Hills, CA | reply to NetAdmin
Comprehensively inaccurate. Their data testing methods are seriously flawed and data worthless. The testing results do not reflect reality. | |
iansltx
join:2007-02-19
Golden, CO
·Comcast
·Qwest.net
·magicjack.com
·BeeCreek Communica..
·Sprint Mobile Broa..
| reply to NetAdmin
Interesting, because they do a decent job of locating me: in the city I live near.
Also, shopping servers is fine. To get a true speed test result ou pick the server with the most capacity possible. I'm guessing that's why Speedtest.net doesn't put latency in the graphs
They also keep track of the IP doing the speed test, so they can do some normalization.
Additionally, is there really a better alternative than Speedtest.net's stats? Theirs is a TON more scientific than SpeedMatters. On the one hand you have 400+ million tests, and on the other 40,000, a number that isn't even statistically significant given the number of people using the internet in the US. Granted, Speedtest.net isn't US-only but I'm betting 100+ million of its results are. So yes, it is much mroe scientific to look at their results than Speedmatters'.
About RCN vs. other providers, I noticed when I looked at Europe that they had a lot of ISPs over there testing in the (real world) 40 Mbit range. That's cool. FiOS can do that too.
Responding to another post, Norway probably can't get ultra high speed access everywhere, and if it does, I'd hate to see how much is going from taxpayers' pockets to subsidize the access. | |
iansltx
join:2007-02-19
Golden, CO
·Comcast
·Qwest.net
·magicjack.com
·BeeCreek Communica..
·Sprint Mobile Broa..
| reply to Rob_
Call me hypercapitalist but I don't want MY tax dollars paying for a system that may well fail, not be upgraded to support demand or supply, etc.
We've seen what has happened to muni wireless. The only "success story" cost taxpayers $5 million and serves only city employees. These employees could likely have been better served by a private entity...pay the entity $40 a month per wireless line and you give that company enough money tuo upgrade their infrastructure. With such a large customer, the wireless companies would compete and thus have a better network for everyone.
Last I checked, competition is good for the economy, and all that. The problem is that there is no competition, or very little int the internet market, not that there isn't a government-run network. I'd rather pay $50 a month for internet access to a private company than pay $55 a month extra in taxes to the government, who would then contract a private ompany to build a network. Uh, no thanks.
Also, the whole "one network" thing doesn't work. Fiber will only reach so far before the cost becomes prohibitive, or at least more prohibitive than a wireless setup. By the way, WiMax is here right now and LTE isn't, and WiMax can support 70 Mbps symmetric, so what's the big push to kill it off? LTE I'm sure can adapt their standard to be compatible...
If you want government-subsidized internet, move to a country that has it. Then realize that you're paying extra in taxes for this subsidy. Disconnect, anyone?
Though on a slightly different angle the government would do well to foster any competitors, especially small ones, that look to have a working product as far as 'net access goes, to keep the megacorps in check. Sort of like how Grande Communications provides cable competition in areas near (60 miles away or more) me.
Also, I have no problem with calling 768 kbps internet broadband. Everyone can agree that it's not great speed and all but you can watch online video, use VoIP and do gaming over such a conection, provided it has decent latency. Oh, and 256 kbps upload speed is broadband too, I'd say. Or if you want to be picky make it 384. Though everyone agrees that 200 kbps is NOT.
Another problem with the 2/1 broadband requirement is that it would mean a T1 connection isn't broadband. Which is a load of baloney.
Also, the US didn't buy ANY wireless infrastructures that you spoke of, Rob_. That was the wireless carriers. At the beginning of things there was actually one standard, AMPS. Then features were added and CDMA came out. GSM followed later and had more features, but at the same time CDMA got some upgrades. Now we're down to two networks. Sure they're incompatible, but so are cable and DSL. People can compete with proprietary technologies as long as those technologies allow access to the same types of (unfettered) service: voice, video and internet data.
So please by all means let the government come in and stir up competition by fostering new internet service providers opening up for inexpensive license wireless bands to take backhaul out of the hands of the LECs. But I don't want to pay extra taxes just so my internet *might* be faster.
Then again, I plan to solve the internet problem in my area, so I guess that's why I don't like armchair debate. | |
NetAdmin
CCNA
join:2008-05-22
| reply to iansltx
said by iansltx :At any rate, check out speedtest.net for a more accurate picture of internet speeds by country, continent, state and city. Their location data is suspect because they use GeoIP-like methods for guessing where a given host is located.
These guys' results are sort of bunk. Especially since RCN's speeds on Speedtest.net exceed anything on Asia's top 10, ISP wise. Speedtest users can shop servers, going from server to server getting the best results. So Speedtest.net is not even close to scientific or much better than the Speedmatters tests.
Most providers, including RCN, are nowhere near the speed of some of the other providers in other nations. | |
|
