dslreports logo
site
 
    All Forums Hot Topics Gallery
spc

spacer




how-to block ads


Search Topic:
uniqs
9
share rss forum feed


Squiddy

@pnap.net
reply to maxtor

Re: Bandwidth shaping only on port 80. Penalize heavy http.

Hey, great you've made some very good progress. Another thing to consider is the Squid box is going to be performing a heavy amount of DNS lookups, by installing a local caching resolver, like Bind9, pointed to roothints you'll substantially improve speeds since you'll be caching and recursive. You'd want to point /etc/resolv.conf to localhost after doing this.

You will probably want to mount noatime or relatime for the /var partition or optionally / if you're not using separate partitions.

Lots of TCP_MISS may be related to max cache object size or that the cache simply hasn't filled yet.

Here are snippets of my Squid 2.x setup, for the most part it should apply to the Squid 3 syntax:

#Bandwidth Control
delay_pools 1
delay_class 1 1
delay_parameters 1 660000/660000
delay_access 1 allow localnet
 
#Bind IP:Port Mode
visible_hostname proximo
http_port 192.168.1.3:3128 transparent
icp_port 0
dns_nameservers 127.0.0.1
 
#CGI caching
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY
 
#Cache
cache_mem 128 MB
maximum_object_size 65535 KB
minimum_object_size 0 KB
maximum_object_size_in_memory 200 KB
cache_replacement_policy lru
memory_replacement_policy lru
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern .  0  20% 4320
 
#Directories
cache_dir aufs /external/squid 8096 16 256
cache_access_log /external/squid/access.log
cache_log /external/squid/cache.log
cache_store_log /external/squid/store.log
coredump_dir /external/squid
 

maxtor

join:2012-12-21
122002
Thanks a lot Squiddy, I am indeed running local bind for queries. I am attaching my squidclient report:

# squidclient -h 10.1.1.1 mgr:info

HTTP/1.0 200 OK
Server: squid/3.1.10
Mime-Version: 1.0
Date: Fri, 04 Jan 2013 07:58:27 GMT
Content-Type: text/plain
Expires: Fri, 04 Jan 2013 07:58:27 GMT
Last-Modified: Fri, 04 Jan 2013 07:58:27 GMT
X-Cache: MISS from ggn-10-1-1-1
X-Cache-Lookup: MISS from ggn-10-1-1-1:3128
Via: 1.0 ggn-10-1-1-1 (squid/3.1.10)
Connection: close

Squid Object Cache: Version 3.1.10
Start Time: Thu, 03 Jan 2013 07:55:23 GMT
Current Time: Fri, 04 Jan 2013 07:58:27 GMT
Connection information for squid:
Number of clients accessing cache: 122
Number of HTTP requests received: 332119
Number of ICP messages received: 0
Number of ICP messages sent: 0
Number of queued ICP replies: 0
Number of HTCP messages received: 0
Number of HTCP messages sent: 0
Request failure ratio: 0.00
Average HTTP requests per minute since start: 230.1
Average ICP messages per minute since start: 0.0
Select loop called: 61015593 times, 1.419 ms avg
Cache information for squid:
Hits as % of all requests: 5min: 4.9%, 60min: 8.6%
Hits as % of bytes sent: 5min: 4.9%, 60min: 9.4%
Memory hits as % of hit requests: 5min: 26.1%, 60min: 43.2%
Disk hits as % of hit requests: 5min: 8.0%, 60min: 5.2%
Storage Swap size: 458408 KB
Storage Swap capacity: 89.5% used, 10.5% free
Storage Mem size: 343052 KB
Storage Mem capacity: 67.4% used, 32.6% free
Mean Object Size: 17.21 KB
Requests given to unlinkd: 77431
Median Service Times (seconds) 5 min 60 min:
HTTP Requests (All): 0.33943 0.27332
Cache Misses: 0.35832 0.28853
Cache Hits: 0.00000 0.00000
Near Hits: 0.20843 0.23230
Not-Modified Replies: 0.00000 0.00000
DNS Lookups: 0.14261 0.13638
ICP Queries: 0.00000 0.00000
Resource usage for squid:
UP Time: 86583.403 seconds
CPU Time: 368.918 seconds
CPU Usage: 0.43%
CPU Usage, 5 minute avg: 0.57%
CPU Usage, 60 minute avg: 0.78%
Process Data Segment Size via sbrk(): 435808 KB
Maximum Resident Size: 1772304 KB
Page faults with physical i/o: 0
Memory usage for squid via mallinfo():
Total space in arena: 435940 KB
Ordinary blocks: 419312 KB 2101 blks
Small blocks: 0 KB 0 blks
Holding blocks: 1012 KB 4 blks
Free Small blocks: 0 KB
Free Ordinary blocks: 16627 KB
Total in use: 420324 KB 96%
Total free: 16627 KB 4%
Total size: 436952 KB
Memory accounted for:
Total accounted: 387557 KB 89%
memPool accounted: 387557 KB 89%
memPool unaccounted: 49394 KB 11%
memPoolAlloc calls: 81220376
memPoolFree calls: 82767989
File descriptor usage for squid:
Maximum number of file descriptors: 1024
Largest file desc currently in use: 413
Number of file desc currently in use: 197
Files queued for open: 0
Available number of file descriptors: 827
Reserved number of file descriptors: 100
Store Disk files open: 1
Internal Data Structures:
26681 StoreEntries
26299 StoreEntries with MemObjects
26280 Hot Object Cache Items
26636 on-disk objects


Squiddy

@pnap.net
Our values look pretty close and mine is snappy, though I have a fraction of the clients you do. Could it be that you need to ratchet down your delay pool size to account for other circuit usage such as RTMP and streaming media or other usages? I have pretty good cache hit ratios as compared to you but that could be due to reduced number of clients.

What are you using for these arguments, mine are below:
cache_mem 128 MB
maximum_object_size 65535 KB
minimum_object_size 0 KB
maximum_object_size_in_memory 200 KB
cache_replacement_policy lru
memory_replacement_policy lru
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern .  0  20% 4320
 

Oh yeah, can't believe I missed this. You need to make sure you're not applying your delay_pool to your cache either or your dest net so you might want to acl your local RFC 1918 range and deny it to your delay pool.

My stuff

# squidclient -h 192.168.1.3 mgr:info
HTTP/1.0 200 OK
Server: squid/2.6.STABLE21
Date: Fri, 04 Jan 2013 14:28:09 GMT
Content-Type: text/plain
Expires: Fri, 04 Jan 2013 14:28:09 GMT
Last-Modified: Fri, 04 Jan 2013 14:28:09 GMT
X-Cache: MISS from proximo
X-Cache-Lookup: MISS from proximo:3128
Proxy-Connection: close
 
Squid Object Cache: Version 2.6.STABLE21
Start Time:     Sun, 16 Dec 2012 19:12:39 GMT
Current Time:     Fri, 04 Jan 2013 14:28:09 GMT
Connection information for squid:
     Number of clients accessing cache:     9
     Number of HTTP requests received:     158813
     Number of ICP messages received:     0
     Number of ICP messages sent:     0
     Number of queued ICP replies:     0
     Request failure ratio:      0.00
     Average HTTP requests per minute since start:     5.9
     Average ICP messages per minute since start:     0.0
     Select loop called: 25095446 times, 64.734 ms avg
Cache information for squid:
     Request Hit Ratios:     5min: 23.6%, 60min: 31.9%
     Byte Hit Ratios:     5min: 6.4%, 60min: 24.4%
     Request Memory Hit Ratios:     5min: 0.0%, 60min: 27.3%
     Request Disk Hit Ratios:     5min: 68.6%, 60min: 45.7%
     Storage Swap size:     7459924 KB
     Storage Mem size:     131048 KB
     Mean Object Size:     53.72 KB
     Requests given to unlinkd:     0
Median Service Times (seconds)  5 min    60 min:
     HTTP Requests (All):   0.33943  0.33943
     Cache Misses:          0.49576  0.68577
     Cache Hits:            0.04776  0.01955
     Near Hits:             0.22004  0.23230
     Not-Modified Replies:  0.00194  0.00194
     DNS Lookups:           0.18639  0.10428
     ICP Queries:           0.00000  0.00000
Resource usage for squid:
     UP Time:     1624529.862 seconds
     CPU Time:     3056.093 seconds
     CPU Usage:     0.19%
     CPU Usage, 5 minute avg:     0.28%
     CPU Usage, 60 minute avg:     0.11%
     Process Data Segment Size via sbrk(): 170024 KB
     Maximum Resident Size: 681872 KB
     Page faults with physical i/o: 1
Memory usage for squid via mallinfo():
     Total space in arena:  170156 KB
     Ordinary blocks:       163846 KB    919 blks
     Small blocks:               0 KB      0 blks
     Holding blocks:           376 KB      2 blks
     Free Small blocks:          0 KB
     Free Ordinary blocks:    6309 KB
     Total in use:          164222 KB 96%
     Total free:              6309 KB 4%
     Total size:            170532 KB
Memory accounted for:
     Total accounted:       153903 KB
     memPoolAlloc calls: 30136285
     memPoolFree calls: 29472771
File descriptor usage for squid:
     Maximum number of file descriptors:   1024
     Largest file desc currently in use:     46
     Number of file desc currently in use:   19
     Files queued for open:                   0
     Available number of file descriptors: 1005
     Reserved number of file descriptors:   100
     Store Disk files open:                   0
     IO loop method:                     epoll
Internal Data Structures:
     138947 StoreEntries
       8873 StoreEntries with MemObjects
       8872 Hot Object Cache Items
     138862 on-disk objects