dslreports logo
site
    All Forums Hot Topics Gallery
spc
Search Topic:
uniqs
146
share rss forum feed


koitsu
Premium,MVM
join:2002-07-16
Mountain View, CA
kudos:23
reply to isamu99

Re: Heads up guys....new trick to eliminate Youtube throttling!

Note: I am not a TWC customer.

What's being proposed in that Reddit thread, and in the reference materials that thread cites (some random blog, and that blog cites some random Youtube video), is dangerous on numerous levels. There's a very strong indication that the authors involved do not understand networking, and/or the implications of their ""solution"".

The proposed ""solution"", at the firewall level, does the following:

1. Drop any TCP packet destined to an IP within the 173.194.55.0/24 netblock, with a TCP destination port of 80
2. Drop any TCP packet destined to an IP within the 206.111.0.0/16 netblock, with a TCP destination port of 80

To date, not a single person has provided actual packet captures providing that TWC is doing some kind of HTTP redirection which causes your client to fetch content from alternate locations (e.g. content hosted within the 173.194.55.0/24 or 206.111.0.0/16 netblocks -- BTW, those are, respectively, Google and XO Communications -- and in the case of the former, Google advertises 173.194.0.0/16 via BGP on the Internet).

TWC can't be doing reverse-proxying because otherwise said ""solution"" wouldn't work; they would have to be, for example, looking for very specific HTTP headers and GET requests and injecting an HTTP Location: header into the response, to force the browser to fetch content from an alternate location. If it was done via reverse-proxying (e.g. transparently), the above ""solution"" would not work, since it's being done on the workstation itself.

I suppose an alternate model/method they could be using is wildcard DNS matching (e.g. return IPs within the above netblocks when folks look up very specific FQDNs for c.youtube.com content -- normally they're very strange FQDNs such as r9---sn-nwj7knel.c.youtube.com), in which case using an alternate DNS provider (e.g. OpenDNS) should, unless they're doing transparent DNS response injection, accomplish the same thing. However, in the below referenced video, the author claims "changing DNS providers did nothing".

I can talk at extreme length about this whole ordeal and what it actually amounts to at a networking level, but it's all speculative because nobody has provided actual evidence of what's going on; the Youtube video that mitchribar.com mentions as "proof" is not a proper analysis of any sort.

Packet captures of all HTTP and DNS traffic when visiting Youtube and watching a video need to be provided -- and preferably with ipconfig /flushdns being done beforehand.

--
Making life hard for others since 1977.
I speak for myself and not my employer/affiliates of my employer.


ke4pym
Premium
join:2004-07-24
Charlotte, NC
Reviews:
·VOIPO
·ooma
·Verizon Broadban..
·Northland Cable ..
·Time Warner Cable

said by koitsu:

Note: I am not a TWC customer.

What's being proposed in that Reddit thread, and in the reference materials that thread cites (some random blog, and that blog cites some random Youtube video), is dangerous on numerous levels. There's a very strong indication that the authors involved do not understand networking, and/or the implications of their ""solution"".

The proposed ""solution"", at the firewall level, does the following:

1. Drop any TCP packet destined to an IP within the 173.194.55.0/24 netblock, with a TCP destination port of 80
2. Drop any TCP packet destined to an IP within the 206.111.0.0/16 netblock, with a TCP destination port of 80

To date, not a single person has provided actual packet captures providing that TWC is doing some kind of HTTP redirection which causes your client to fetch content from alternate locations (e.g. content hosted within the 173.194.55.0/24 or 206.111.0.0/16 netblocks -- BTW, those are, respectively, Google and XO Communications -- and in the case of the former, Google advertises 173.194.0.0/16 via BGP on the Internet).

TWC can't be doing reverse-proxying because otherwise said ""solution"" wouldn't work; they would have to be, for example, looking for very specific HTTP headers and GET requests and injecting an HTTP Location: header into the response, to force the browser to fetch content from an alternate location. If it was done via reverse-proxying (e.g. transparently), the above ""solution"" would not work, since it's being done on the workstation itself.

I suppose an alternate model/method they could be using is wildcard DNS matching (e.g. return IPs within the above netblocks when folks look up very specific FQDNs for c.youtube.com content -- normally they're very strange FQDNs such as r9---sn-nwj7knel.c.youtube.com), in which case using an alternate DNS provider (e.g. OpenDNS) should, unless they're doing transparent DNS response injection, accomplish the same thing. However, in the below referenced video, the author claims "changing DNS providers did nothing".

I can talk at extreme length about this whole ordeal and what it actually amounts to at a networking level, but it's all speculative because nobody has provided actual evidence of what's going on; the that mitchribar.com mentions as "proof" is not a proper analysis of any sort.

Packet captures of all HTTP and DNS traffic when visiting Youtube and watching a video need to be provided -- and preferably with ipconfig /flushdns being done beforehand.

I'd be happy to pull this off. But I have issues with Youtube and Maps from all over the place.

I have access to the following ISPs:

Verizon Wireless (3G & 4G)
AT&T via metro ethernet
Time Warner Cable via cable modem and metro ethernet
Time Warner Telecom via metro ethernet
Charter cable via cable modem

On all of those connections, with various hardware, Youtube and Maps S U C K 99.8% of the time.

If you want me to send you a packet capture from TWC via cable modem let me know. I'll be happy to send you something.


koitsu
Premium,MVM
join:2002-07-16
Mountain View, CA
kudos:23

I don't want "something" -- I want a proper packet capture of all traffic from someone on TWC who can confirm that the aforementioned ""solution"" works for them (and I want the capture to be when such ""solutions"" are not in place). I want an actual raw capture, not "logging lines", because every packet and its headers (particularly the payload portion) need to be examined.

The pcap filter rule I'd recommend using is (tcp and port 80) or (udp and port 53). This will capture HTTP and DNS traffic only. I would suggest closing down all other applications (including anything running in the systray) before doing this.

I would also appreciate this be done by someone who is not behind a router (unless their router is running a Linux equivalent and they can use tcpdump to capture traffic out the WAN interface), i.e. directly connected.

Please keep a clock running on your computer doing the captures, and make note of what times/etc. you do certain things -- it might be easier to just click an existing Youtube video URL that starts with http://.

Finally remember that visiting http://www.youtube.com/ will redirect you to https://www.youtube.com/ (Google does that), and subsequent videos shown via that would be streamed via HTTPS (SSL) and not HTTP (the above pcap filter will not match HTTPS traffic); I don't see how TWC could be doing any kind of redirection or injection if HTTPS is used, as that would indicate a MITM attack and certificate verifications would fail.

--
Making life hard for others since 1977.
I speak for myself and not my employer/affiliates of my employer.



swintec
Premium,VIP
join:2003-12-19
Alfred, ME
kudos:5
Reviews:
·Time Warner Cable
·VoicePulse
·Sprint Mobile Br..
·RapidVPS

said by koitsu:

Finally remember that visiting http://www.youtube.com/ will redirect you to https://www.youtube.com/ (Google does that), and subsequent videos shown via that would be streamed via HTTPS (SSL) and not HTTP

Are you sure? »[OOL] is punishing encrypted YouTube straems at prime time?

Poster on the 4th page:
"YouTube streams are not encrypted. If you go to https://www.youtube.com/watch?v=YIBx0PoSqSg you are only viewing the webpage using SSL. The actual video stream still comes via a non SSL connection from the CDN. This can be verified using WireShark or a similar packet capture software."

--
Usenet Block Accounts | Unlimited Accounts


koitsu
Premium,MVM
join:2002-07-16
Mountain View, CA
kudos:23

said by swintec:

said by koitsu:

Finally remember that visiting http://www.youtube.com/ will redirect you to https://www.youtube.com/ (Google does that), and subsequent videos shown via that would be streamed via HTTPS (SSL) and not HTTP

Are you sure? »[OOL] is punishing encrypted YouTube straems at prime time?

Poster on the 4th page:
"YouTube streams are not encrypted. If you go to https://www.youtube.com/watch?v=YIBx0PoSqSg you are only viewing the webpage using SSL. The actual video stream still comes via a non SSL connection from the CDN. This can be verified using WireShark or a similar packet capture software."

Wow, interesting -- you're right. I just did a packet capture and verified this claim.

I guess this is one thing that, somehow, streaming video models let you get away with: mix-matched URI schemes. Normally with HTTPS you can't have mix-matched content (i.e. everything has to use HTTPS else risk the browser throwing nastygrams at you). It isn't a "Flash thing" because I enabled HTML5 and found a Youtube video that was available via HTML5 and despite the URL using an HTTPS URI scheme, the video was still transmitted via HTTP.
--
Making life hard for others since 1977.
I speak for myself and not my employer/affiliates of my employer.

ke4pym
Premium
join:2004-07-24
Charlotte, NC
Reviews:
·VOIPO
·ooma
·Verizon Broadban..
·Northland Cable ..
·Time Warner Cable
reply to koitsu

said by koitsu:

I don't want "something" -- I want a proper packet capture of all traffic from someone on TWC who can confirm that the aforementioned ""solution"" works for them (and I want the capture to be when such ""solutions"" are not in place). I want an actual raw capture, not "logging lines", because every packet and its headers (particularly the payload portion) need to be examined.

The pcap filter rule I'd recommend using is (tcp and port 80) or (udp and port 53). This will capture HTTP and DNS traffic only. I would suggest closing down all other applications (including anything running in the systray) before doing this.

I would also appreciate this be done by someone who is not behind a router (unless their router is running a Linux equivalent and they can use tcpdump to capture traffic out the WAN interface), i.e. directly connected.

I'm down with doing this. However, as far best as I can tell, m0n0wall won't do a packet capture. And there's noooo way I'm hooking up directly to the cable modem.

I'm very familiar with taking packet captures with WireShark. I can do that, or if you'll help with command line syntax, I can use tcpdump on my Mac.

--edit to correct pcap for tcpdump.


koitsu
Premium,MVM
join:2002-07-16
Mountain View, CA
kudos:23

m0n0wall is based on FreeBSD; if it has a shell interface, has tcpdump, and has bpf device support enabled in the kernel or as a module (this is default), it should be able to do the capturing.

From this thread it appears that the kernel has bpf support, but that the tcpdump binary and associated libpcap library (and possibly others) are missing -- installing them by unpacking some standard FreeBSD package tarballs and putting both libpcap.so* and tcpdump in the same directory should suffice. Do not follow the advice of the fellow who download libpcap.so.1.1.1 and renamed it to libpcap.so.1 -- do not do this.

If you want to provide me with a Wireshark capture from a Windows workstation (if that's easiest for you), be my guest.

On OS X it's just as easy. As root: tcpdump -p -i {iface} -l -n -v -s 0 -w capture.pcap '(tcp and port 80) or (udp and port 53)' where {iface} should be the interface name of your Ethernet connection (i.e. en0, etc.). To end the capture, press Ctrl-C, and then copy the capture.pcap file somewhere where you can open it in Wireshark, or hand it over to me.

Be aware: cookies and other things will be visible in these captures, as well as any other TCP port 80 and DNS traffic that occurred during that time, so if you're concerned about privacy (which I would be in this particular circumstance), you can PM me with a link to the capture and I'll look at it privately -- see my profile for kudos/etc. from folks here on DSLR/BBR as justification that I'm not going to violate your privacy or distribute the capture.

--
Making life hard for others since 1977.
I speak for myself and not my employer/affiliates of my employer.


ke4pym
Premium
join:2004-07-24
Charlotte, NC
Reviews:
·VOIPO
·ooma
·Verizon Broadban..
·Northland Cable ..
·Time Warner Cable

said by koitsu:

m0n0wall is based on FreeBSD; if it has a shell interface, has tcpdump, and has bpf device support enabled in the kernel or as a module (this is default), it should be able to do the capturing.

Yeah, I did look into that and saw where the guy wasn't getting the library stuff correct.

Let's reserve that option in case it is only very very necessary.

I'm keenly aware of the data I'll be sending you. I have to capture network data quite often when I troubleshoot the load balancers and web app firewalls at work - a medical facility.

Soon as a get a few free minutes, I'll whip everything up and document what the captures are for.

Do you want the capture of the entire video stream? Or just a few minutes?

Do you want me to stream a video for a bit, then switch to 1080p?

Then rinse and repeat with the claimed miracle fix?


koitsu
Premium,MVM
join:2002-07-16
Mountain View, CA
kudos:23

I'd like a capture starting prior to Youtube even being loaded/visited in the web browser. Start the capture, visit a Youtube video link in your browser, and let the capture go for about 5-10 seconds.

It would also be useful if the exact same could be repeated but with the supposed ""solution"" in place (assuming it works for you). Please make sure you visit the same Youtube URL in both captures, and please make sure to completely shut down/exit + restart your browser between captures.

I don't need the whole duration of the video stream captured -- what's being claimed is that TWC is in effect redirecting browsers to an alternate destination (specifically 173.194.55.0/24 or 206.111.0.0/16 netblocks) that is rate-limited; the claim seems to be that by blocking outbound packets to those destinations, the browser attempts to get the same content but from a different destination (i.e. a netblock not in either of those ranges) which is not rate-limited.

Thanks.
--
Making life hard for others since 1977.
I speak for myself and not my employer/affiliates of my employer.