dslreports logo
site
 
    All Forums Hot Topics Gallery
spc

spacer




how-to block ads


Search Topic:
uniqs
5202
share rss forum feed

NiteOwl

join:2001-09-15
K1L 9G9

How to block IPs of ISPs caching servers.

Hi I'm with Start.ca and I'm using and linksys wrt54gl router with DD-WRT custom firmware.

Is response to this youtube videos (

youtu.be/YJOLdpR447o?t=37s
) that talks about how ISPs throttling the speed of specific services by redirecting you to their caching server (I think that's how it works) I wanted to know, fisrt since the article that the youtube video links to talk about Time Warner Cable, I wanted to know how do I find out if and to what IPs my services like youtube and netflix were being rerouted to and how to block them on my router.


rocca
Start.ca
Premium
join:2008-11-16
London, ON
kudos:23
For what it's worth, we're not doing any redirection or throttling of services. That of course doesn't answer your question how to avoid it, will leave that to someone else with experience, but just wanted to give you a heads up that we're not doing any redirection or rate limiting on any services.


BliZZardX
Premium
join:2002-08-18
Toronto, ON

2 edits
reply to NiteOwl
That's not how it works. Your ISP does not run the caching server. Google does.

You should not try to bypass video cache servers AKA CDNs. If everyone did that the internet would break.

Seriously. The reason those caches exist is to get video traffic off the internet as quickly as possible at the edge (where your ISPs router meets the start of the internet) and not congest the backbones the internet actually needs to transport non-static content.

If your video is slow to load the cache probably hasn't fully loaded yet, this happens often on NEWLY POSTED videos or videos that aren't played very often (because they just aren't popular). It's only after a certain number of plays and high demand that YouTube mirrors the videos to Google cache's around the world.

Other possible bottlenecks are the connection between your ISP and Bell/Rogers is saturated, the connection between your ISP and Google is saturated, or DNS geomapping for your IP address prefix is broken and sending you to a cache server that is not local.


somrandomguy

@distributel.net
reply to NiteOwl
Simply put, caching servers are a good thing. It moves the content you want to watch/download closer to you, giving you the fastest possible access to the content.

Google is a great example, and so are Microsoft Updates... and STEAM games too, and most huge websites too.

This happens outside the ISP's network generally, and affects 10s of thousands of customers.

Is there a reason you want to or think you should avoid them?

HeadSpinning
MNSi Internet

join:2005-05-29
Windsor, ON
kudos:5
said by somrandomguy :

Is there a reason you want to or think you should avoid them?

I suspect because sometimes the CDNs might have not provided for enough capacity at the ISPs edge, or something is broken. I've seen situations for example where Akamai has something wrong on their servers at a particular ISP, affecting only those customers.

In theory, it shouldn't happen, but does.
--
MNSi Internet - »www.mnsi.net


pstewart
Premium,VIP
join:2005-10-12
Peterborough, ON
kudos:1
With Youtube it's hard to avoid their caches (meaning specifically Google Global Cache) and for good reasons - they want to deliver the best possible delivery to you ... not saying it's a perfect science by any means.

You can change your delivery with some CDN networks (Google/Youtube not included) based on which DNS servers you use. This is something you may wish to double check because if you are not using your ISP's DNS servers then CDN performance is "typically" not ideal (this is getting better but has quite a bit of work to go yet). For example if you are using OpenDNS your results "typically" will not be as good as using your ISP's own DNS servers.

lawrenson

join:2012-02-22
reply to BliZZardX
said by BliZZardX:

If your video is slow to load the cache probably hasn't fully loaded yet, this happens often on NEWLY POSTED videos or videos that aren't played very often (because they just aren't popular). It's only after a certain number of plays and high demand that YouTube mirrors the videos to Google cache's around the world.

Google will cache all content (and removes it after a certain period of time if it's isn't requested anymore), some others like Netflix will specifically cache popular content.
But it's the same idea really, brand new content may be slightly slower than it would be from the cache.

Here's some documentation from a few years ago, it still works more or less the same though:

quote:
The explosion of broadband access and rich multimedia content continually increases the demand on service provider networks. Google Global Cache (GGC) allows you to serve Google content, primarily video, from the edge of your own network. This eases congestion on your network and lessens traffic on peering and transit links. GGC saves you money while improving the experience of your users.

System Overview

Without GGC, every user request for the latest YouTube video causes a unique copy of that video to transit your network, from Google to your user. With GGC, only the first copy of the video makes the transit. When another user requests that same video, Google serves it from your GGC node.

GGC Features

- Reduced traffic through your network

Cache hit rates vary with the usage pattern of your users, but typical performance is close to 75%.

- Faster response, transparent to users

Google transparently serves your users’ requests from caches inside your network.

- Easy to set up

Installation requires a rack, a laptop, a copy of our CD, and a connection to your network. Once the servers have been initially configured and are reachable, Google will do the rest of the work and monitoring remotely.

- Robust

The node has multiple levels of redundancy. If the GGC node is unavailable for any reason, user requests will be sent transparently to Google.

How GGC Works

When a user requests a piece of content – for example, a video, web page, or image – Google systems determine if that content can be served from the GGC node inside your network and if the user is authorized to access the GGC node.

If the GGC node already has the requested content in its local cache, it will serve the content directly to the end user, improving the user experience and saving transit expense.

If the content is not stored on the GGC node, the node will retrieve it from Google, serve it to the user, and store it for future requests.

Request Flow Diagram

Diagram Key

1. A user follows a link to a Google-hosted video or other content. The computer generates a DNS request for the IP address of the content host.

2. Your DNS resolver queries Google DNS for the IP address of the content host.

3. Google DNS knows that you have Google Global Cache, so it replies with the IP address of the node. It knows this because you have advertised the IP addresses of your DNS resolver to the node (via BGP) and Google has loaded that information into its DNS system.

4. Your DNS returns the IP address of the cache node to the user.

5. The user’s computer now sends the content request to the received IP address, which routes to your GGC node.

6. The node validates that the user should be served from this node. It does this by comparing the user’s IP address to the list of IP blocks advertised to the node via BGP. If the address is not valid for the requested content, the user is redirected to a cache on the Google network.

7. If the content is not already on the GGC node, the node requests the content from Google and caches it.

8. Once the GGC node has the content, it serves it to the user. The content is retained on the node so that the next request can be served without pulling the content from Google.

Maintenance and Support

The GGC system is designed with multiple levels of redundancy. Content and user requests are spread across all available servers, so if a server failure occurs, another server in the GGC node can immediately respond to the user request. If a server is unable to respond to a request, it will redirect the user back to Google.

Google monitoring will detect failures and attempt to resolve issues remotely. In the event that the failure cannot be repaired remotely, we will contact your technical contact to schedule the next step. This next step could include hardware diagnosis, hardware replacement, or software reinstallation. If a hardware swap is required, Google will arrange for the RMA.

Frequently Asked Questions

Q: What is the expected hit rate of the video cache

A: Cache hit rate will vary based on the traffic and usage pattern of your users. Typical cache hit rate is between 70% and 90%.

Q: Will the users be required to make any changes to take advantage of GGC? Could this generate additional call volume on our technical support lines?

A: The system will be transparent to users. If the GGC node is unavailable for any reason, user requests will be sent directly to Google as they are today.

Q: How will Google send content to the GGC Node?

A: The local cache is filled in a read-through basis when content is requested by the end-user. No content is pre-loaded.

Q: What Google services will be supported by the GGC node?

A: Typically, most of the traffic served from the GGC node is large content such as video streams and file downloads. Other web services, such as Google search and maps are proxied and cached as well. Web services are dynamically added and removed from cache nodes based on capacity and end-user performance improvement.

Q: Who owns the GGC node?

A: Google will retain ownership of the hardware and software that makes up the node. Google will be responsible for all maintenance, support, and shipping costs related to the server equipment.

Q: Will other ISPs’ customers be sent to the GGC node on our network?

A: Requests from any user who can access your DNS resolvers may be sent to the node. The node will redirect requests from users outside of the prefixes you are advertising. For this reason, access control lists limiting IP ranges that can reach the cache are not permitted.

If you provide service to downstream ASNs, please ensure their prefixes and resolvers are provided via the BGP feed.

Q: We provide transit services for other ISPs. Will their end users use the GGC node?

A: They can, and probably should, if your network is their primary path to Google. Ensure their user and DNS resolver IP ranges are included in the BGP feed.


Guspaz
Guspaz
Premium,MVM
join:2001-11-05
Montreal, QC
kudos:23
reply to NiteOwl
And yet Youtube is notorious for having poor performance, where often 1080p videos won't play without lots of start-and-stop buffering, both on ISPs that do have edge caches and those that don't.
--
Developer: Tomato/MLPPP, Linux/MLPPP, etc »fixppp.org


pstewart
Premium,VIP
join:2005-10-12
Peterborough, ON
kudos:1
said by Guspaz:

And yet Youtube is notorious for having poor performance, where often 1080p videos won't play without lots of start-and-stop buffering, both on ISPs that do have edge caches and those that don't.

We rarely run across this issue - get virtually no complaints from customers in that regard.


pstewart
Premium,VIP
join:2005-10-12
Peterborough, ON
kudos:1
reply to lawrenson
said by lawrenson:

said by BliZZardX:

If your video is slow to load the cache probably hasn't fully loaded yet, this happens often on NEWLY POSTED videos or videos that aren't played very often (because they just aren't popular). It's only after a certain number of plays and high demand that YouTube mirrors the videos to Google cache's around the world.

Google will cache all content (and removes it after a certain period of time if it's isn't requested anymore), some others like Netflix will specifically cache popular content.

Netflix caches are pre-loaded based on popularity and dynamically update daily... typically off-hours (defined by the ISP).


jmck
formerly 'shaded'

join:2010-10-02
Ottawa, ON
Reviews:
·TekSavvy DSL
·Start Communicat..
reply to Guspaz
said by Guspaz:

And yet Youtube is notorious for having poor performance, where often 1080p videos won't play without lots of start-and-stop buffering, both on ISPs that do have edge caches and those that don't.

aren't you on TSI? I was previously on TSI and Youtube would constantly buffer at 720p and 1080p. I switched providers and haven't had a single Youtube issue since.


YoutubeCache

@distributel.net
How to stop ISPs sucking at Youtube

»mitchribar.com/2013/02/time-warn···witchtv/