dslreports logo
 
    All Forums Hot Topics Gallery
spc
Search similar:


uniqs
950

Inssomniak
The Glitch
Premium Member
join:2005-04-06
Cayuga, ON

Inssomniak

Premium Member

Reliving http caching/proxy

voxframe and Semaphore brought it up first in another thread so I opened a new one, but Ive been thinking of it again.

Im re-thinking the idea of a squid caching proxy as my user count is higher now and my internet pipe is getting nearly full.
Also it might improve the user experience having common stuff cached right on the network.

Anyone doing it anymore? What kind of hit rates?
raytaylor
join:2009-07-28

1 recommendation

raytaylor

Member

Click for full size
I use an appliansys cachebox. Works really well - especially for youtube. Things like music videos that subscribers have in their favorite lists, or popular songs dont need to be downloaded through the upstream pipe for every view.

It also brings major parts of websites closer to the user - like reducing the ping to the website for half the content. Many of my customers think that most everyday websites such as the NZ herald (major newspaper) and trademe (our equivalant of ebay) actually are faster and more responsive on their 2mbit rural connection through my network, than at their office on a 20mbit ADSL2 connection with a major ISP.

I also run a dns cache too which helps a bit.

Many people are opposed to caching because of the fear of stale pages - i have no such problem with the appliansys cachebox system. I know its based on squid but whatever witchcraft they do inside the box just seems to work, and works well.

I measure my cachebox performance based on the megabytes it saves me each day and on total traffic, its saving me around about 4 to 8 gigabytes - average number in percentage i see is about 12-15%
For example, when psy gangnam released his new gentleman song, I saw an 11 gigabyte saving that day.
I also remember when lady gaga released her telephone video - in high definition on youtube, i had alot less customers at the time and was trialing the cachebox with a demo unit, it was what made me decide to buy it.

Customers seeing better performing websites was just a side effect.
And again i have never had one complaint of stale web pages being served up.

So I am pro-caching
My upstream is saturated most of the evening - with the cachebox, my customers dont realise - facebook and everything else still appears to run nice and smooth

Attached is an image which is from 1-2 years ago - when I was buying an 8mbit upstream which could burst up to 10mbits - we almost always got the burst avaliable 100% of the time.
Anyhow, you can see what the cachebox was doing in the peak times with the extra throughput on the upstream connection to this router.

Semaphore
Premium Member
join:2003-11-18
101010

Semaphore to Inssomniak

Premium Member

to Inssomniak
Yup. Sill have a cascading cache setup for HTTP and DNS, but as I mentioned before it's what I used to do for a living so this level of caching is probably not for everyone. It's not as effective as it used to be in cache hits and BW saved vs. the big picture anymore. Mostly because the Web has changed a helluva lot in the past 8 years... but it's still stupidly fast and I never have to touch any of it unless a board blows.

Inssomniak
The Glitch
Premium Member
join:2005-04-06
Cayuga, ON

Inssomniak

Premium Member

I set up a test squid server 3.1 and just ran myself and my partner thru it for a while, and already hit a site that wouldnt load via the proxy I can only imagine once I send a thousand people thru it it will be bad.

Thats my biggest fear is setting up one, and it breaking a shit ton of sites and get calls like crazy.

warwick
join:2009-06-05
Hollywood, FL

warwick

Member

said by Inssomniak:

I set up a test squid server 3.1 and just ran myself and my partner thru it for a while, and already hit a site that wouldnt load via the proxy I can only imagine once I send a thousand people thru it it will be bad.

Thats my biggest fear is setting up one, and it breaking a shit ton of sites and get calls like crazy.

hmm, you may have to go under the hood and tinker with things a bit to find a right balance, or to get things working the way you want in general.

This is much easier done through webmin...
raytaylor
join:2009-07-28

raytaylor to Inssomniak

Member

to Inssomniak
I have never had that problem
Although i do use ip-spoofing
This is where if the proxy needs to get the content from the real website (uncached) it will say it is coming from the client ip address instead of the proxy. This means that websites can tell clients apart when they are behind the same proxy.
Porch
join:2005-06-19

Porch to Inssomniak

Member

to Inssomniak
I use squid a bunch and have not run into any site not loading problems. Back in the early days of the internet I had it happen, but not in a long time since then.

Inssomniak
The Glitch
Premium Member
join:2005-04-06
Cayuga, ON

Inssomniak

Premium Member

Anyone running squid 3.1 want to try this site for me?
It partially loads the controls but there should be a map of the county load up as well but its just blank. We use this site for computer based site surveys in our county and its invaluable to us.

»maps.niagararegion.ca/Na ··· aldimand

Semaphore
Premium Member
join:2003-11-18
101010

Semaphore

Premium Member

Does not load for me with intercept caching. Weird - my township uses a similar tool and that works. You can always add the domain to the ACL with direct-connect to whitelist it if that happens, or you can go the other way and only intercept the sites you actually want to cache which is a lot safer I suppose.
raytaylor
join:2009-07-28

raytaylor to Inssomniak

Member

to Inssomniak
Although i dont know what version of squid is in the appliansys box, it works from here too.

It appears to be a full screen flash player app
+1 above - i would add the addresses to the bypass list

Inssomniak
The Glitch
Premium Member
join:2005-04-06
Cayuga, ON

Inssomniak to Semaphore

Premium Member

to Semaphore
said by Semaphore:

Does not load for me with intercept caching. Weird - my township uses a similar tool and that works. You can always add the domain to the ACL with direct-connect to whitelist it if that happens, or you can go the other way and only intercept the sites you actually want to cache which is a lot safer I suppose.

I did try initially adding the obvious domain but I think I have to add all the sites it connects to, as doing that didnt work, I can see in the logs it access several different sites.

Semaphore
Premium Member
join:2003-11-18
101010

Semaphore

Premium Member

I'm running a vanilla 3.1.16... upgraded from 3.0.