Network gear manufacturer Sandvine apparently isn't a big fan of both Netflix's and YouTube's new ISP streaming performance rankings, insisting that the data collected by both is unreliable and conflicting. In a blog post, Sandvine points out that ISPs deemed "HD Verified" by Google's new ISP ranking (discussed by us here) are sometimes categorized as under-performers in Netflix's rankings, and vice-versa:
quote:
Google is essentially saying Rogers’ customers who use YouTube are capable of regularly experiencing HD streams, while Netflix is saying Rogers’ subscribers are experiencing the worst quality of Netflix streaming in the country. At the same time Netflix is saying Bell Canada’s DSL subscribers are capable of experiencing HD streams when using Netflix, but Google is saying Bell non-Fibe (DSL) are not YouTube HD-verified and “should be able to watch YouTube videos in Standard Definition (at least 360p) with moderate load times."
I've already pointed out
numerous times that Netflix's streaming rankings are made less valuable by the fact that Netflix CDN partners do better in the rankings than those who refuse to participate in the free service. Similarly I've noted Google and YouTube's new ISP rankings need some work as well, as navigation is clunky and some
ISPs are listed as HD verified even in markets they don't even provide service.
With that said, it's worth noting that the lion's share of Sandvine's money comes courtesy of the nation's largest ISPs, and as such Sandvine has traditionally been reluctant to admit fault on the ISP end of the equation. The company recently proclaimed that YouTube streaming issues that have plagued most broadband users are entirely YouTube's fault, even if data tends to suggest there's plenty of blame to go around.
It also seems obvious that ranking YouTube and Netflix streaming performance would be "conflicting," given those rankings are testing entirely different content services, taking entirely different routes to the end user. Routes can be even more different now that companies like Netflix are (begrudgingly) striking direct interconnection deals with ISPs like Comcast and Verizon.
While both Google and Netflix efforts need work (and will hopefully improve), they're at least attempting to shed a little more light on the buffering problems that have plagued the industry the last few years. That's more than can be said for most ISPs, which have long hidden performance data (unless it's to cherry pick it to justify policies like usage caps, deregulation, etc.). If ISPs want to contest YouTube and Netflix's findings, they could simply release raw performance data of their own.
At a time when transit companies are claiming these slowdowns are
intentional to make an extra buck, it seems like every shred of additional performance data is helpful if we're interested in the truth.