dslreports logo
 
    All Forums Hot Topics Gallery
spc
Search similar:


uniqs
25609
PJL
join:2008-07-24
Long Beach, CA

PJL to qwert9999

Member

to qwert9999

Re: Netflix unable to stream HD in Northeast?

I just fired up a Netflix session and determined I was connected to ipv4_1.lagg0.c013.lax004.ix.nflxvideo.net [198.38.96.142]. I did a trace route to it (1st hop was my router):
2 4 ms 4 ms 4 ms L100.LSANCA-VFTTP-128.verizon-gni.net [96.229.59.1]
3 12 ms 10 ms 11 ms G1-6-0-0.LSANCA-LCR-21.verizon-gni.net [100.41.196.42]
4 7 ms 9 ms 8 ms ae9-0.LAX01-BB-RTR1.verizon-gni.net [130.81.163.240]
5 * * * Request timed out.
6 8 ms 8 ms 7 ms 0.ae2.XL3.LAX15.ALTER.NET [140.222.227.19]
7 10 ms 13 ms 11 ms TenGigE0-6-0-0.GW4.LAX15.ALTER.NET [152.63.114.205]
8 12 ms 11 ms 11 ms teliasonera-gw.customer.alter.net [152.179.21.42]
9 29 ms 27 ms 27 ms netflix-ic-300871-las-bb1.c.telia.net [213.248.95.34]
0 28 ms 28 ms 28 ms ipv4_1.lagg0.c013.lax004.ix.nflxvideo.net [198.38.96.142]

Note the hop after alert.net. The last altern.net node (teliasonera-gw.customer.alter.net [152.179.21.42]
) is in Ashburn Virginia. The next node (netflix-ic-300871-las-bb1.c.telia.net [213.248.95.34]) is in Europe! The last node ipv4_1.lagg0.c013.lax004.ix.nflxvideo.net [198.38.96.142] is back in Wilmington Delaware.

Now correct me if I'm wrong, but that means there's a hop to Europe and then one back, both outside of the Verizon network -- the one controlled by telia.net. Why would this be so? Isn't this routing NOT controlled by Verizon but by whoever Netflix is using? Doesn't this mean Netflix is manipulating routing in a very stupid way that is likely saturating something on their own ISP's network between here and Europe?

Please help me understand who controls this routing.

BioGeek
Quantum-VHO7
Premium Member
join:2007-08-25
West Orange, NJ

BioGeek

Premium Member

I actually find it more interesting that the very first hop past Verizon timed out.

birdfeedr
MVM
join:2001-08-11
Warwick, RI

1 edit

birdfeedr to PJL

MVM

to PJL
Timing not possible via europe. Round trip would need to add 50 to 75 ms. Lookup is not accurate.

[Added] your hops on the west coast up to 12ms before transit to east coast up to 30ms is very good for cross country.
telia address really only shows ownership location. Telia is in sweden, but is worldwide network.
last two hops with nearly same time says the telia gateway into the netflix server farm is probably a colocation in same facility.
PJL
join:2008-07-24
Long Beach, CA

PJL

Member

said by birdfeedr:

Timing not possible via europe. Round trip would need to add 50 to 75 ms. Lookup is not accurate.

[Added] your hops on the west coast up to 12ms before transit to east coast up to 30ms is very good for cross country.
telia address really only shows ownership location. Telia is in sweden, but is worldwide network.
last two hops with nearly same time says the telia gateway into the netflix server farm is probably a colocation in same facility.

Interesting information on the location. I use »www.ip-adress.com/ip_tracer/ which actually plots the IP address on a map.

Could you post a trace to the same Netflix server?
PJL

PJL to BioGeek

Member

to BioGeek
said by BioGeek:

I actually find it more interesting that the very first hop past Verizon timed out.

I'm actually not surprised at this. I think it's likely the edge router so wouldn't they turn off ping to prevent DOS attacks?

BioGeek
Quantum-VHO7
Premium Member
join:2007-08-25
West Orange, NJ

BioGeek

Premium Member

Possible. I didn't see that when I did a trace route for that IP though. I did notice one of the later hops timed out twice and them made it through. The big pings seem to be outside of Verizon.
betatester4
join:2014-06-09
united state

betatester4 to PJL

Member

to PJL
said by PJL:

Interesting information on the location. I use »www.ip-adress.com/ip_tracer/ which actually plots the IP address on a map.

6 50 ms 20 ms 18 ms 0.xe-3-0-1.XL4.IAD8.ALTER.NET [152.63.3.69]
7 25 ms 19 ms 23 ms TenGigE0-7-4-0.GW1.IAD8.ALTER.NET [152.63.35.157]
8 18 ms 26 ms 22 ms teliasonera-gw.customer.alter.net [152.179.50.234]
9 18 ms 19 ms 17 ms ash-bb4-link.telia.net [213.155.133.224]
10 87 ms 79 ms 94 ms las-bb1-link.telia.net [213.155.135.153]
11 * 100 ms 102 ms netflix-ic-300871-las-bb1.c.telia.net [213.248.95.34]
12 96 ms 99 ms 98 ms ipv4_1.lagg0.c013.lax004.ix.nflxvideo.net [198.38.96.142]

I checked ISP TeliaSonera International Carrier which is a European Union company, Interconnection Facilities located at Manhattan NY. Facility Management Telx, they have private peering agreement with netflix. I confirmed by »www.peeringdb.com which have information related to Peering.

Telx New York (111 8th)
Facility Management Telx
Website »www.telx.com/
Address 1 111 8th Avenue
Address 2 3rd, 8th, 14th and 15th Floors
City / State / Postal New York NY 10011
Country Code US
CLLI Code NYCMNY
NPA-NXX 212-905

SeattleMatt
Streaming Tech Director
Premium Member
join:2001-12-28
Seattle, WA

1 recommendation

SeattleMatt

Premium Member

You guys do know how to read the naming conventions, right?

In PJL's trace - you're staying in Los Angeles the entire time.

In Beta's trace - you go down to DC - over to Ashburn, VA on Telia, then out to Los Angeles.

There's no Europe... LOL
betatester4
join:2014-06-09
united state

betatester4

Member

said by SeattleMatt:

You guys do know how to read the naming conventions, right?

I didn't know that. When you mentioned name conventions its look more easy to identify the location for every node (LOL). I did the search with the IP address and the company that handle the nodes.

I just was trying to clarify that the trace stay all the time in U.S.A. never cross the Atlantic. My case north east.

Packeteers
Premium Member
join:2005-06-18
Forest Hills, NY
Asus RT-AC3100
(Software) Asuswrt-Merlin

Packeteers to Morris0

Premium Member

to Morris0
said by Morris0:

Works fine in Queens, New York

same here. i netflix on my roku3 at 720p on a 15/1 docsis3 twc isp only service. it's a shame verizon can shake streamers down for more money - thanks to our useless FCC.
PJL
join:2008-07-24
Long Beach, CA

3 edits

3 recommendations

PJL

Member

said by Packeteers:

said by Morris0:

Works fine in Queens, New York

same here. i netflix on my roku3 at 720p on a 15/1 docsis3 twc isp only service. it's a shame verizon can shake streamers down for more money - thanks to our useless FCC.

...or a steamer can purchase transit through an inferior ISP that overloads it's ports (as defined in the peering agreement) to the Verizon network (or in your case the TWC network since with a 15/1 connection you should be getting 1080p streams unless the Roku3 doesn't support that resolution) and then blames it on Verizon or the other end-user ISP. Which is it really?

I just checked Netflix Sunday morning at 6:45 am Pacific time (not a peak Netflix viewing time) and didn't get over a 720p stream on my 50/25 connection (which is performing fine based on speed test results). Netflix controls the quality of service of their own stream (the streaming rate) with the client -- not Verizon. So it looks to me like Netflix is limiting their streams at this point rather than it actually hitting a capacity issue to the Verizon network.
Expand your moderator at work
navyson
join:2011-07-15
Upper Marlboro, MD

navyson to tk421storm

Member

to tk421storm

Re: Netflix unable to stream HD in Northeast?

Currently in Jamaica on vacation. We brought our Roku 3 with us for the kids to watch when we are back in the hotel room. Netflix plays in beautiful HD with no buffering. Also, we got to watch on Netflix the Showtime blockbuster series Homeland which is not available for streaming In the US.

penske1
join:2002-10-19
Fishkill, NY

1 recommendation

penske1 to PJL

Member

to PJL
said by PJL:

Netflix controls the quality of service of their own stream (the streaming rate) with the client -- not Verizon. So it looks to me like Netflix is limiting their streams at this point rather than it actually hitting a capacity issue to the Verizon network.

That makes absolutely no sense. Stream rate is set depending on available bandwidth at the time.

BioGeek
Quantum-VHO7
Premium Member
join:2007-08-25
West Orange, NJ

1 recommendation

BioGeek

Premium Member

That bandwidth is determined all along the transit path though. The data can only go as fast as the slowest step. If that step is TeliaSonera, then it isn't Verizon's fault. If that limiting step is Alter.net or Verizon then it is Verizon's fault. (note that my previous comment had me forgetting that Alter.net was the name for the MCI/Verizon Business Tier 1 Backbone) The point being that Netflix's choice of carrier to get the data to Verizon could be just as much at fault.

I have been watching Orange is the New Black the past few days/nights and have always gotten at least 480 resolution. I often got 720 and would get 1080 towards the later parts of the night and sometimes on the weekend late mornings. There is some randomness though. Sometimes I would have 480, pause, do something for a few minutes and then come back only to find 1080 when I hit play.

Some of it seems random and some seems to be driven by peak times.

FOR PJL's TRACE:
Note that Verizon's Alter.net got that ping to the LA area and did so in the same amount of time it took Telia and Netflix to play around on their servers in the LAX area and get that ping dealt with.

Verizon's last hop averages about 11ms (seems good to me) and then it takes an average of another 17ms for Netflix's server to handle the ping through a Telia handoff. That whole 17ms is all happening in LAX area...

Verizon's Alter.net got that data/ping across just fine. From this ping, somewhere around Telia/Netflix needs to handle quicker.

FOR BETA's TRACE:

That one just looks plain unhappy compared to PJL's. The cross continent part of that (and the big ping) is on Telia. If Telia is doing the long haul to get to the nearest Verizon/Alter.net peering node, then it seems far more likely that Telia is hitting a limit of bandwidth at peak times.

Verizon isn't doing the long haul on the video data. That means that the impact is spread out on their network. (let's not pretend like they aren't a first class member of the Tier 1 club) Telia however, is taking the big hit carrying all of the episodes of OITNB across the country in 480, 720 and 1080 resolution.

I still think they are playing a game of brinksmanship here. Other ISPs do just fine during some times (although I hear of coworkers on Optimum and Comcast complaining of low res during peak). At some point I think Telia AND Verizon need to improve the peering connection between them. Either that or Netflix needs to get on the Verizon train.

Let's not forget that the Tier 1 "agreement" means that the network impacts should be neutral. That means that Alter.net should be dumping just as much data onto Telia that Telia dumps onto Alter.net. When the balance shifts severely to one side.... well....the big kids fight.

Them fighting only means a loss for us. Why...why? Why can't they all just get along!!?!?
PJL
join:2008-07-24
Long Beach, CA

PJL to penske1

Member

to penske1
said by penske1:

said by PJL:

Netflix controls the quality of service of their own stream (the streaming rate) with the client -- not Verizon. So it looks to me like Netflix is limiting their streams at this point rather than it actually hitting a capacity issue to the Verizon network.

That makes absolutely no sense. Stream rate is set depending on available bandwidth at the time.

Stream rate is not set solely on available bandwidth at the time. Stream rate can also be dependent on other factors that are use to set the quality of service parameters (such as determining where the request is coming from and limiting the stream based on that determination). This is the point I was implying.

BioGeek
Quantum-VHO7
Premium Member
join:2007-08-25
West Orange, NJ

BioGeek

Premium Member

I am convinced that pausing and restarting a stream sometimes works out. I have gone from 480 to 1080 and right back to 480 when pausing. Yet maintaining the stream maintains the resolution. Somewhere there is an active control over what resolution makes it through. At some point it just settles in and stays as whatever. Then pausing seems to create a new point to evaluate the stream bandwidth.
PJL
join:2008-07-24
Long Beach, CA

PJL to BioGeek

Member

to BioGeek
said by BioGeek:

That bandwidth is determined all along the transit path though. The data can only go as fast as the slowest step. If that step is TeliaSonera, then it isn't Verizon's fault. If that limiting step is Alter.net or Verizon then it is Verizon's fault. (note that my previous comment had me forgetting that Alter.net was the name for the MCI/Verizon Business Tier 1 Backbone) The point being that Netflix's choice of carrier to get the data to Verizon could be just as much at fault.
...

Some of it seems random and some seems to be driven by peak times.

FOR PJL's TRACE:
Note that Verizon's Alter.net got that ping to the LA area and did so in the same amount of time it took Telia and Netflix to play around on their servers in the LAX area and get that ping dealt with.

Verizon's last hop averages about 11ms (seems good to me) and then it takes an average of another 17ms for Netflix's server to handle the ping through a Telia handoff. That whole 17ms is all happening in LAX area...

Verizon's Alter.net got that data/ping across just fine. From this ping, somewhere around Telia/Netflix needs to handle quicker.

FOR BETA's TRACE:

That one just looks plain unhappy compared to PJL's. The cross continent part of that (and the big ping) is on Telia. If Telia is doing the long haul to get to the nearest Verizon/Alter.net peering node, then it seems far more likely that Telia is hitting a limit of bandwidth at peak times.

Verizon isn't doing the long haul on the video data. That means that the impact is spread out on their network. (let's not pretend like they aren't a first class member of the Tier 1 club) Telia however, is taking the big hit carrying all of the episodes of OITNB across the country in 480, 720 and 1080 resolution.

I still think they are playing a game of brinksmanship here. Other ISPs do just fine during some times (although I hear of coworkers on Optimum and Comcast complaining of low res during peak). At some point I think Telia AND Verizon need to improve the peering connection between them. Either that or Netflix needs to get on the Verizon train.

Let's not forget that the Tier 1 "agreement" means that the network impacts should be neutral. That means that Alter.net should be dumping just as much data onto Telia that Telia dumps onto Alter.net. When the balance shifts severely to one side.... well....the big kids fight.

Them fighting only means a loss for us. Why...why? Why can't they all just get along!!?!?

"The point being that Netflix's choice of carrier to get the data to Verizon could be just as much at fault. "
I think this is the basic issue -- as impacted by a peering situation that is not neutral.

I'm off the train on this issue in this thread.

BioGeek
Quantum-VHO7
Premium Member
join:2007-08-25
West Orange, NJ

BioGeek

Premium Member

Sadly, I think we all are. We can't control or fix this. The big ones with the pipes need to work out this little spat so we can all get back to watching the daily goings on of a women's prison.

Yeah...that sounded wrong. Congrats OITNB, you ruined us all and broke the internet!
qwert9999
join:2001-06-27
Port Richey, FL

qwert9999 to tk421storm

Member

to tk421storm
there is a trick that use to work sometimes. it hasn't worked for me for the past month. but might work for people in other areas. right now every night i get rebuffering or connection lost from 9:30pm past 1am when i go to sleep.

if netflix is running at a low image quality. close nexflit and reconnect.

i was doing a test one night at 3 am. i connected and had 1080p right off the bat. i shut it down and i restarted it 4 times. 3 times i got 1080p and 2 times i was stuck at low image quality around 480bps.

the 3 times i got 1080p i got a spike of 40+mbps and was at 1080 within 10 seconds. when i was stuck at 480 i didn't get a spike and speed was around 340kbps.

but it did work for me quite a few times when fios netflix speeds started to drop last year


guppy_fish
Premium Member
join:2003-12-09
Palm Harbor, FL

guppy_fish to tk421storm

Premium Member

to tk421storm
I vote this the most useless thread of 2014 ....
serge87
join:2009-11-29
New York

serge87

Member

said by guppy_fish:

I vote this the most useless thread of 2014 ....

For as much complaining that happens, pretty much every single discussion on this entire forum about private business deals between ISPs has no influence on these matters either way. Though they're good for a morning chuckle with coffee.

biggbrother
Premium Member
join:2001-11-07
Providence, RI

1 recommendation

biggbrother to tk421storm

Premium Member

to tk421storm
Wow nearly 8 months later and this is clearly still an issue. I've been trying to watch "Sons of Anarchy" on Netflix and the experience has been horrendous. I have a 25/25 connection. The past dozen episodes (including last night), have literally taken 10 minutes just to begin playing on my Xbox One, which is wired directly via Ethernet.

Even when it starts playing, the quality is horrendous. It literally looks worse than VHS quality (for those of you old enough to remember VHS), and still occassionally buffers for 2 minutes. I am not sure what is considered "peak time" for Netflix but this was last night around 10PM EST.

Frustrated, I finally switched over to Amazon Prime IInstant Video on the Xbox One and watched the first episode of Under the Dome, which streamed quickly in glorious 1080p with not more than 5 seconds of buffering. After suffering through SoA in VHS quality on Neflix it was like putting on a good pair of prescription lenses.

I don't know whose fault this is but it needs to get fixed. I like Amazon Prime but I like Netflix better because they have more selection. If it comes out that Verizon is not investing in adequate peering then I may need to look into switching to Cox.

What the heck is the point in offering all these "Quantum" speeds to its customers if Verizon won't support their infrastructure to handle reliable HD streaming. WTF do I need 25/25 or 50/25 speeds to browse web pages for? If that's all I did, I'd save $20-$20n a month and downgrade to 5MB per month or something.

If practically all traffic on the internet is monitored, why haven't any experts done a through analysis of this problem to determine who is at fault?
serge87
join:2009-11-29
New York

serge87

Member

said by biggbrother:

I don't know whose fault this is but it needs to get fixed.

»Re: [Networking] 1st Proof of Verizon / Netflix Direct Peering????
Go out and enjoy summer
PJL
join:2008-07-24
Long Beach, CA

1 recommendation

PJL to penske1

Member

to penske1
said by penske1:

said by PJL:

Netflix controls the quality of service of their own stream (the streaming rate) with the client -- not Verizon. So it looks to me like Netflix is limiting their streams at this point rather than it actually hitting a capacity issue to the Verizon network.

That makes absolutely no sense. Stream rate is set depending on available bandwidth at the time.

I think you just agreed with me. It's Netflix that sets the streaming rate, not Verizon. And since I'm seeing low rates at very off-peak times (like 6am) it's hard to believe that there is an actual bandwidth limit at that time. Further, during these off-peak times the stream pegs at a certain level (like 1750kbps) and stay at that level -- never higher or lower -- even though if the path to Verizon was saturated it should drop lower or buffer, or if there's available bandwidth it should go higher. I think this is evidence of arbitrary stream management limits, not the automatic and dynamic stream management you would expect when the stream adjusts (on the fly) to available bandwidth.

guppy_fish
Premium Member
join:2003-12-09
Palm Harbor, FL

1 recommendation

guppy_fish

Premium Member

Were probably seeing the while Netflix whines like a 2 year old, its all the telcos fault, once the peering is addressed and there is still quality / bandwidth issues persist its at Netflix's end. They don't have the server and network capacity to supply all HiDef the customers want

biggbrother
Premium Member
join:2001-11-07
Providence, RI

biggbrother to tk421storm

Premium Member

to tk421storm
Verizon denies again that they are doing anything to negativley affect Netflix traffic. Says they are working with Netflix to improve performance in a seperate effort apart from the peering agreement.

»www.slashgear.com/netfli ··· 1337049/

nothing00
join:2001-06-10
Centereach, NY

2 recommendations

nothing00 to tk421storm

Member

to tk421storm
The interesting thing (to me) in the graphic that Verizon published recently about the network utilization was just how high their peak usage rates were in their internal network. Those are pretty darned high numbers. Pretty close to the edge of failing.

If Verizon did fix their peering issues they may find that their internal network can't handle the Netflix traffic. Even without Netflix, natural bandwidth utilization increases will shortly overcome what they've got.

I hope they're upgrading their internal network like mad.

SeattleMatt
Streaming Tech Director
Premium Member
join:2001-12-28
Seattle, WA

1 recommendation

SeattleMatt

Premium Member

Just tried my weekly random test-
Could barely pull 720p in the middle of the day.

Accessing via L3 in Atlanta.

I really wish that it could determine where you are - and connect you to the nearest most efficient IP. For me, that'd be in NYC on the direct peer.
PJL
join:2008-07-24
Long Beach, CA

PJL

Member

So I was up early this morning (4am PDT) and fired up Netflix using the Windows 8 app (which supports 1080p SuperHD streaming). I was only able to get to 3000kpbs -- 720p quality. The stream ramped up to that in just a few seconds, but then it pegged at that level. Obviously there was no buffering.

So I did a traceroute to look to see what the ping times on the route were. Here it is:
1 1 ms 1 ms 1 ms Wireless_Broadband_Router.home [192.168.1.1]
2 4 ms 4 ms 4 ms L100.LSANCA-VFTTP-128.verizon-gni.net [96.229.59.1]
3 8 ms 7 ms 11 ms G0-14-1-5.LSANCA-LCR-22.verizon-gni.net [130.81.191.120]
4 8 ms 7 ms 7 ms ae14-0.LAX01-BB-RTR2.verizon-gni.net [130.81.163.248]
5 * * * Request timed out.
6 * 8 ms 7 ms 0.ae5.BR1.LAX15.ALTER.NET [140.222.225.135]
7 9 ms 8 ms 9 ms ae-7.r05.lsanca03.us.bb.gin.ntt.net [129.250.8.85]
8 7 ms 7 ms 7 ms ae-5.r04.lsanca03.us.bb.gin.ntt.net [129.250.2.220]
9 9 ms 8 ms 9 ms 129.250.199.50
10 6 ms 7 ms 7 ms ipv4_1.lagg0.c047.lax001.ix.nflxvideo.net [108.175.46.92]

So I fired up IE and accessed the site via the web client. The stream quickly ramped up to 5800kbps, SuperHD quality, and stayed there. The route was to a different Netflix server using L3 this time instead of NTT:
1 1 ms 1 ms 1 ms Wireless_Broadband_Router.home [192.168.1.1]
2 4 ms 4 ms 4 ms L100.LSANCA-VFTTP-128.verizon-gni.net [96.229.59.1]
3 11 ms 7 ms 7 ms G0-5-1-4.LSANCA-LCR-22.verizon-gni.net [130.81.55.179]
4 8 ms 8 ms 8 ms so-3-1-0-0.LAX01-BB-RTR2.verizon-gni.net [130.81.199.40]
5 * * * Request timed out.
6 * 9 ms 7 ms 0.ae5.BR1.LAX15.ALTER.NET [140.222.225.135]
7 9 ms 7 ms 7 ms ae6.edge1.LosAngeles9.level3.net [4.68.62.169]
8 8 ms 7 ms 7 ms 4.53.228.170
9 7 ms 7 ms 7 ms ipv4_1.lagg0.c039.lax004.ix.nflxvideo.net [198.38.97.148]

During peak hours the streams never reach these levels and the ping times (including the external nodes) are in the 20s (below for the Verizon nodes), still good. So this shows that Netflix is tailoring their streams (and the ISP they use to get it to me) depending on the client.

Now it doesn't look like network congestion between the Verizon network and ntt.net. It looks like Netflix is limiting their streams depending on that client and the server reached. And this doesn't appear necessary during the off-peak time I used to do this testing (4am PDT).