dslreports logo
 
    All Forums Hot Topics Gallery
spc
Search similar:


uniqs
15324
Cloneman
join:2002-08-29
Montreal

Cloneman to mikefxu

Member

to mikefxu

Re: Cloneman's Tomato QoS Tips for adsl, vdsl2, and cable

updated all my screens. Does the new shibby have min & max for downstream?
mikefxu
join:2004-10-05
Titusville, FL

mikefxu

Member

Click for full size
Click for full size
Click for full size
Cloneman
join:2002-08-29
Montreal

Cloneman

Member

looks like shibby put in toastman QoS and the tvlz TC-atm patch.
quote:
- Dodano wsparcie dla TC-ATM overhead w zakadce QoS – dziki Tvlz
- Odwieone reguki QoS – dziki Toastman

mikefxu
join:2004-10-05
Titusville, FL

mikefxu

Member

Tomato Firmware 1.28.0000 MIPSR2-105 K26 USB AIO

USB support integration and GUI,
IPv6 support,
Linux kernel 2.6.22.19 and Broadcom Wireless Driver 5.10.147.0 updates,
support for additional router models, dual-band and Wireless-N mode.
Copyright (C) 2008-2011 Fedor Kozhevnikov, Ray Van Tassle, Wes Campaigne
»www.tomatousb.org

OpenVPN integration and GUI,
Copyright (C) 2010 Keith Moyer,
tomatovpn@keithmoyer.com

"Shibby" features:
- Transmission 2.76 integration
- GUI for Transmission
- NFS utils integration and GUI
- Custom log file path
- SD-idle tool integration for kernel 2.6
- 3G Modem support (big thanks for @LDevil)
- SNMP integration and GUI
- APCUPSD integration and GUI (implemented by @arrmo)
- DNScrypt-proxy 1.0 integration and GUI
- TOR Project integration and GUI
- TomatoAnon project integration and GUI
- TomatoThemeBase project imtegration and GUI
Copyright (C) 2011-2013 Micha Rupental
»openlinksys.info

"JYAvenard" Features:
- OpenVPN enhancements & username/password only authentication
- PPTP VPN Client integration and GUI
Copyright (C) 2010-2012 Jean-Yves Avenard
jean-yves@avenard.org

"Victek" features:
- Extended Sysinfo
- Captive Portal. (Based in NocatSplash)
- HFS / HFS+ filesystem integration
Copyright (C) 2007-2011 Ofer Chen & Vicente Soriano
»victek.is-a-geek.com

"Teaman" features:
- QOS-detailed & ctrate filters
- Realtime bandwidth monitoring of LAN clients
- Static ARP binding
- VLAN administration GUI
- Multiple LAN support integration and GUI
- Multiple/virtual SSID support (experimental)
- UDPxy integration and GUI
- PPTP Server integration and GUI
Copyright (C) 2011 Augusto Bott
Tomato-sdhc-vlan Homepage

"Toastman" features:
- Configurable QOS class names
- Comprehensive QOS rule examples set by default
- TC-ATM overhead calculation - patch by tvlz
- GPT support for HDD by Yaniv Hamo
Copyright (C) 2011 Toastman
Using QoS - Tutorial and discussion

"Tiomo" Features:
- IMQ based QOS Ingress
- Incoming Class Bandwidth pie chart
Copyright (C) 2012 Tiomo

"Victek/PrinceAMD/Phykris/Shibby" feature:
- Revised IP/MAC Bandwidth Limiter

Based on Tomato Firmware v1.28
Copyright (C) 2006-2010 Jonathan Zarate
»www.polarcloud.com/tomato/

Built on czw, 17 sty 2013 22:15:38 +0100 by Shibby, »openlinksys.info

Davesnothere
Change is NOT Necessarily Progress
Premium Member
join:2009-06-15
Canada

4 edits

Davesnothere to Cloneman

Premium Member

to Cloneman
said by Cloneman:

looks like shibby put in toastman QoS and the tvlz TC-atm patch.

 
Thank you, folks, for the updated info !

So does this mean that I ought to install the latest Toastman build which I downloaded around when the current thread began, and which had been discussed just before that in more detail here »Tomato QoS major bug (resolved - normal behavior) , or should I choose the latest Shibby build which you are discussing today in the current thread ?

MY currently installed Shibby build (v93, from sometime last year) has some stuff from Toastman in it, but as discovered in the other thread, NOT an Inbound Global Maximum Bandwidth limiting control, which looks to be a Tiomo feature rather than one of Toastman's, if I read the earlier replies about that properly.

As I see things, it's just that Toastman seems to have included this Tiomo tweak in HIS builds before Shibby did.

Bear in mind again that I have a Cisco-Linksis WRT54GL router, and it would depend upon whether or not Shibby has done a new version of his for that, using an older kernel as he did last time (v.2.4), and as I believe that my downloaded and ready Toastman (also built last year) is based too.
Cloneman
join:2002-08-29
Montreal

Cloneman

Member

I guess either one would be fine? I don't know how shibby runs on old hardware, I haven't tried the new shibby either.

I would get the latest toastman , maybe it's these for wrt54gl:
»www.4shared.com/dir/v1Bu ··· NH6DiVQV

Davesnothere
Change is NOT Necessarily Progress
Premium Member
join:2009-06-15
Canada

4 edits

Davesnothere

Premium Member

said by Cloneman:

I guess....

 
I believe that it depends upon whether or not a build has been done based on the Linux 2.4 kernel with 'ND' (new driver), as that is supposed to be OK with the electronics of my mature series of router, IIUC.

I'll check the your link and the other download site I mentioned in the other thread, and then post update(s) here of whatever I find.

Thanks.

= = = = = = =

UPDATE 1 :

Toastman's Builds :

Your above link (4share) = LOADING..... LOADING..... LOADING.....

Going to »www.linksysinfo.org/inde ··· s.36106/ (see 2nd post) gives these links :

»www.4shared.com/dir/v1Bu ··· v1BuINP3 - This link seems active, and browses fine to what looks like YOUR link. [shrug]

»www.mediafire.com/?88t1vzzcgrphx - Seems OK - That's where I got my files on Jan 3rd.

Both of these have the same Toastman files that I already recently downloaded, and are from a build of March of last year (tomato-WRT54G_WRT54GL-1.28.7633.3-Toastman-IPT-ND series).

So the only things left to wonder/investigate now are whether Shibby made a newer build which is compatible with my WRT54 beast, and if yes, whether it has any OTHER advantages than only the Tiomo QoS tweak which itself is also in that Toastman build.

UPDATE 2 :

Shibby's site »tomato.groov.pl/ seems to have a current (3 days old) group of builds (v105) for the WRT54 series, at »tomato.groov.pl/download/K24/ , (as well as others for current routers), so I will download some of these and read the changelog, not necessarily in that order.

SHIBBY's Changelog »tomato.groov.pl/?page_id=78
Davesnothere

4 edits

Davesnothere

Premium Member

 
Conclusions/Questions :

It looks like Toastman updated his K24-ND builds (for old routers like mine) last March, but not since.

That build includes the Tiomo tweak for adding an Inbound Global Maximum Bandwidth limiting control.

He apparently HAS updated his K26 builds since then (mostly intended for newer routers), but I've not investigated what changes are in those builds.

According to Shibby's changelog, the Tiomo control appears for the first time only in his newest (rev 105) builds, and there DOES seem to be a 105 version for his K24-ND branch, so I have downloaded the flavours of that which are designated for an earlier Non-USB router like mine.

The DSL tweak is irrelevant to me as I am on Cable, but also seems to be for the first time only in Shibby's 105 builds.

So should I install the latest Shibby, or last March's Toastman ?

Are there OTHER reasons for me to choose only one of them, besides the Tiomo control tweak which they both seem to have ?

I currently have Shibby's K24-ND-093 build, installed since last summer.
Cloneman
join:2002-08-29
Montreal

Cloneman

Member

hrm, not sure, I'm very confused myself with build numbers.

You could try latest shibby or Tvlz's toastman mod for K24 which I think uses a fairly recent toastman as a base, but im not sure.

»www.box.com/s/45a8764123 ··· 38575632

Davesnothere
Change is NOT Necessarily Progress
Premium Member
join:2009-06-15
Canada

3 edits

Davesnothere

Premium Member

said by Cloneman:

....You could try latest shibby or Tvlz's toastman mod for K24 which I think uses a fairly recent toastman as a base, but im not sure....

 
Thanks.

Hmmmm....

Everyone seems to be releasing builds which include some or all of the other folks' earlier contributions.

TVLZ is the author credited with the mod called 'TC-ATM overhead calculation for DSL', which is included in the latest Shibby build #105 from Jan 23rd.

UPDATE :

I just checked and your linked TVLZ build's changelog does not say whether or not it contains the TIOMO control for Inbound QoS - nor anything else from any other authors, for that matter.

So I guess I'm back to the 2 choices mentioned in my last post.
mikefxu
join:2004-10-05
Titusville, FL

mikefxu to Cloneman

Member

to Cloneman
I screwed with this for many hours this weekend and never could get QoS working during upload stressing. Even going as far to carve out a static amount for VoIP versus everything else caused VoIP quality to drop per Visualware's online VoIP test.

Davesnothere
Change is NOT Necessarily Progress
Premium Member
join:2009-06-15
Canada

4 edits

Davesnothere to Cloneman

Premium Member

to Cloneman
said by Cloneman:

looks like shibby put in toastman QoS and the tvlz TC-atm patch.

quote:
- Dodano wsparcie dla TC-ATM overhead w zakadce QoS – dziki Tvlz
- Odwieone reguki QoS – dziki Toastman

 
Also in Shibby build #105 (for the first time) is the Tiomo Inbound QoS control (which is the part which might matter to ME), and bless my soul, do I see the kitchen sink in there too ?!

BTW, methinks that Shibby had already included Toastman's own mods in MY currently installed build for Cisco-Linksys WRT54GL.

My 'About' page : Shibby K24-ND build #093 from 2012-06-01

Screenshot : »Re: Tomato QoS major bug (resolved - normal behavior)
Cloneman
join:2002-08-29
Montreal

Cloneman to mikefxu

Member

to mikefxu
said by mikefxu:

I screwed with this for many hours this weekend and never could get QoS working during upload stressing. Even going as far to carve out a static amount for VoIP versus everything else caused VoIP quality to drop per Visualware's online VoIP test.

It's not easy to configure QoS, I lost a lot of sleep over the last few years.

If you want to keep it simple for testing, set all UDP traffic to high and everything else to normal. You should be able to maintain less than 8 ms average jitter and less than 0.4% packet loss. Don't use visualware's guidelines, they are overly strict.
mikefxu
join:2004-10-05
Titusville, FL

mikefxu

Member

said by Cloneman :

It's not easy to configure QoS, I lost a lot of sleep over the last few years.

If you want to keep it simple for testing, set all UDP traffic to high and everything else to normal. You should be able to maintain less than 8 ms average jitter and less than 0.4% packet loss. Don't use visualware's guidelines, they are overly strict.

Correct me if I am wrong but my understand was Highest or Lowest were only labels and did not correlate to priority but are merely for categorizing data into groups and then you establish QoS by setting your groups limit.
Cloneman
join:2002-08-29
Montreal

2 edits

Cloneman

Member

said by mikefxu:

Correct me if I am wrong but my understand was Highest or Lowest were only labels and did not correlate to priority but are merely for categorizing data into groups and then you establish QoS by setting your groups limit.

You are indeed incorrect. They are 2 seperate entities. Items that have higher priority on the list will always be prioritized. The bandwith resitricts % is a "bonus" secondary system that should be used mostly to prevent high classes from using _too_much bandwith.

Here's an example setup for upload QoS:

Classifi. type MIN MAX
Priority 1 VoIP 10% 40%

Priority 2 ICMP 2% 10%

Priority 3 HTTP 1% 100%

In this setup, VoiP and ICMP will _ALWAYS_ push http out of the way, even if http is using 100%. For instance when we give ICMP a 10% max, we are saying:

"although we want ICMP to always have priority over http, we don't want the total amount of ICMP traffic to ever exceed 10%"

The purpose of this is that, we assuming that if ever ICMP or VoIP starts using a bunch of bandwith, we're going to assume there's an application misbehaving and we dont want it using up all the bandwith.

If you wanted to, you could set everything to 100% MAX and your QoS would still work, because the most important thing is the order on the list from highest to lowest. My recommendation however is to set some high priorty things to a lower MAX. For example, you could decide that UDP should have a higher priority than http but never use too much bandwith on its own.

I actually made the some of the same assumptions as you which is what inspired making this thread.
rdcanuck
join:2010-09-22
Nepean, ON

rdcanuck

Member

OMG...the lights in my brain just came on! Thanks Cloneman for that lucid explanation. I had been trying to get my head around it for a while and was also making the wrong assumptions.

Now, can you, or anyone, back up those assumptions with a reference? It's not that I doubt you but it is counter-intuitive like you said.

Btw, this thread inspired me and I went and grabbed the latest shibby for my Asus RT-N16, (2.6...105.1 AIO). Upgraded without a hitch from an older beta 2.6 Tomato USB-VPN build. I am blown away by how much stuff is in there.

Davesnothere
Change is NOT Necessarily Progress
Premium Member
join:2009-06-15
Canada

1 edit

Davesnothere

Premium Member

said by rdcanuck:

....Thanks Cloneman for that lucid explanation....

 
+1

Yes indeed !

Well stated and understandable.

Now you said in your example that it applied to the Upload (Outbound) QoS, for which most alternate firmwares do contain support to some degree (for example, I noticed something of QoS in DD-WRT when I used it as a step-up to get my existing Tomato installed).

But what about settings for the Inbound (Download) QoS, such as this Tiomo contribution which has myself and some others looking at moving to one of either the newest Shibby or Toastman builds (which both seem to contain it), what would you suggest doing about setting THAT ?

spock8
join:2012-07-08

spock8 to Cloneman

Member

to Cloneman
Great thread
. Thx

Davesnothere
Change is NOT Necessarily Progress
Premium Member
join:2009-06-15
Canada

4 edits

Davesnothere

Premium Member

 
Well, I did her !

Below (next post) is my 'About' screen, showing Shibby K24-ND build #105 of SD-VPN flavour, for Non-USB routers such as my Linksys WRT54GL.

I upgraded just over 3 hours ago, using the internal function of my existing Shibby K24-ND build #093 of SD-VPN.

BTW, I chose Shibby again this time, mainly because I did not wish to reset my NVRAM and figured that staying with the same author would give me the best chance of getting away with being that cocky (and lazy), as I have quite a few custom settings, including a non-standard subnet range and some port fwds.

Also, the Shibby build I have been running since last summer has been flawless - that is, within the range of things which it was inteneded to do.

Also BTW, the credits are getting so long now that I had to zoom to 90%, even on my 1280x1024 older screen, and set the browser to full screen mode, just to grab it all in one piece.

Thanks to Cloneman and the other contributors, of BOTH threads, for the info and psychological support to get me to do this.

Now, I just need to see how the new QoS behaves....
Davesnothere

Davesnothere

Premium Member

Click for full size
Here it is : ^^^
Cloneman
join:2002-08-29
Montreal

2 edits

Cloneman

Member

for those of you who asked for a starting point, here is mine:

0) Set max upload and download to 85% of Real ISP speed. (you can be more aggressive later - especially on the download side of things w/ ADSL overhead settings)

1) Delete all the default rules, and Rename rubbish class names to something like

1_Highest
2_
3_UDP
4_
5_
6_
7_default
8_
9_Lowest

2) for testing - only 1 rule, priority for UDP traffic (class 3). Everything else default (class 7)



basic settings as follows:



3) 40 bytes overhead for ADSL, 'none' for cable or vdsl

4) try visualware voip testing while hammering your connection. Be sure to click advanced at the end to confirm jitter is less than 8ms and less than 0.5% packet loss, for both upload and download. This is a synthetic test, later you'll use other rules to actually prioritize your real voip connections (although UDP is a catch-all)

Visualware's simulations use UDP so they will have priority.

5) if you want to hammer your connection with torrents, UDP priority will obviously not work because of UTP. In that case, you'll need to prioritize visualware by ports (mine was set to class 2)



This is the starting point from which you work your way up with more complex setups - such as the one I posted on page one of this thread (my current setup). »Re: Cloneman's Tomato QoS Tips for adsl, vdsl2, and cable
neilio
join:2000-08-17
Toronto

neilio

Member

Thanks so much for this! I haven't had a chance to test this yet but it's great to have a clear starting point.

I have 25/10 fibre, so I'm not sure if I should implement #3 (40 bytes overhead), and where I'd set that if I should. Otherwise this is a very clear overview.

Edit: Oh, I see, this is the new build with that patch applied. I'm trying to find the latest Toastman build with that added in; may switch to Shibby, though.

Davesnothere
Change is NOT Necessarily Progress
Premium Member
join:2009-06-15
Canada

3 edits

Davesnothere

Premium Member

said by neilio:

....I have 25/10 fibre, so I'm not sure if I should implement #3 (40 bytes overhead), and where I'd set that if I should.

Otherwise this is a very clear overview.

Edit: Oh, I see, this is the new build with that patch applied.

I'm trying to find the latest Toastman build with that added in.

May switch to Shibby, though.

 
The very recently written DSL patch is in Shibby's latest build #105.

I just installed it over Shibby build #093 from last June and all seems well so far - see posts on previous page of this thread.

Not sure that Toastman has the DSL patch in his yet, even though I do not need it for my Cable service - He did not add it for my older modem's branch (K24 series), last March, but DID include the Tiomo patch (see below) at that time.

Shibby is supporting both older and newer routers and has that patch in all branches, as well as Tiomo's QoS for Inbound mod in all of them - which is part of why I chose his - also that I already had his earlier build and it was stable for me.

Toastman has had the Tiomo patch in all of his builds since early 2012.
Cloneman
join:2002-08-29
Montreal

Cloneman

Member

according to my research, the overhead setting is not needed for Bell Fibe 25/10, because this is VDSL. If you router has it, set to to "none". The overhead setting is used to accomodate overhead problems with ADSL because it uses the ATM somewhere down the line, which VDSL does not.

The overhead setting is availble on versions that implement tvlz interface for the tc-atm patch, which has apparently now expanded to shibby.
MaverickHL
join:2006-01-20

MaverickHL to Guspaz

Member

to Guspaz
With regards to VPN since we are in the topic of QoS. I posted in the generic hardware forums for the router I setup but got no response. Hopefully someone can help explain or give a solution to my situation. So just a bit of context:

- Subscribed to Teksavvy Cable (not that it is too much importance)
- The QoS I had setup only involves outbound related traffic and not inbound
- The router is an Asus RT-N66U which should have plenty of horsepower to run an OpenVPN client.

I subscribed with PIA (privateinternetaccess) and setup the OpenVPN client. After googling around and finding a fancy way to setup certain internal IP ranges to have full VPN access I was pleased with the setup. The issue comes with the Tomato QoS where it can not classify the connections that are using VPN from my assigned range of IP addresses.

I tried setting classification for the IP address range and all the traffic from the VPN IP addresses are not classified and ends up killing those who use it the Internet for local use. So if anyone has any input it would be great.

- MaverickHL
Cloneman
join:2002-08-29
Montreal

Cloneman

Member

As your first order of business you'll need a firmware that correctly supports the global maximum for inbound traffic. The latest tomato toastman is what I use (available on his 4shared page), I'm told that shibby has also included this in their (very) recent release)

if something is "unclassified" my assumption is that it falls under "default". Your objective is to classify it somehow, and assign it to a class that is below default (perhaps using the ports that your vpn server privateinterneaccess uses?)

When you click on "connection details" or graphs you should see some information that will help you classify that vpn connection (protocol, port, ip address)

as a bonus You can also prioritize www traffic above default (TCP DEST 80,443 ), in case some unwanted traffic ends up in default

I'm curious to see the results of your findings. If your VPN provider uses UDP, some members here have suggested that you cannot control inbound rates effectively. I would hypothesize that if your router the dropping the hell out of inbound UDP packets, hopefully there's an application level congestion control mechanism (in the absence of a protocol -level one)
MaverickHL
join:2006-01-20

2 edits

MaverickHL

Member

Hi Cloneman,

Currently I am using Shibby's firmware at the moment. The last update I did was the December one, I just noticed today that it was updated on the 23rd of January. Not sure if that version will have the support of the global maximum connection for inbound traffic fix.

Just some observations from the "unclassified" side of the VPN. So far what I see is mainly the assigned IP addresses I assigned pointing to the sites it is currently looking at, as well as Google's DNS in UDP (8.8.8.8) and a GRE address. All of these have already been put as a classification of Download in the Tomato USB QoS setup.

What made it difficult for me to actually nail down ports was the fact that after I change settings in the router, the custom WAN IP script I am using that handles the assignment of VPN addresses gets wiped out and I would have to reboot the router (not sure if that is a bug in the firmware).

I can give the latest Shibby a go and see on the weekend as I will have to backup the settings before updating the firmware. With regards to the global maximum connection for inbound traffic, are you referring to the Advanced > Conntrack/Netfilter section of the router?

- MaverickHL
Cloneman
join:2002-08-29
Montreal

Cloneman

Member

I'm referring to the max bandwidth limit under inbound, on QoS basic settings page.

Older versions of shibby explicitly say "this is NOT a global maximum!"

As for ports I was hoping you connected VPN provider always connected on the same DESTination port, that would be an easy way to classify.

Davesnothere
Change is NOT Necessarily Progress
Premium Member
join:2009-06-15
Canada

3 edits

Davesnothere to MaverickHL

Premium Member

to MaverickHL
said by MaverickHL:

Currently I am using Shibby's firmware at the moment. The last update I did was the December one, I just noticed today that it was updated on the 23rd of January.

Not sure if that version will have the support of the global maximum connection for inbound traffic fix....

....I can give the latest Shibby a go and see on the weekend as I will have to backup the settings before updating the firmware.

With regards to the global maximum connection for inbound traffic, are you referring to the Advanced > Conntrack/Netfilter section of the router?

 
I can confirm that the latest Shibby build #105 DOES have what you are calling the 'global maximum connection for inbound traffic' fix.

That is also known as the Tiomo patch, and was first introduced to us by Toastman, early during 2012.

Shibby did not include it at the time, but now does in all of his #105 builds, including the K24 series for my older WRT54GL router.

Go here »tomato.groov.pl/?page_id=12

Click on 'Changelog', note the build and date (105 and Jan 23, 2013), then click on 'English' for details.

I was successful in my upgrade from an earlier Shibby (#093) without clearing the NVRAM - so far OK, and all of my settings were continued (though some folks say I took a foolish chance on that part, as it sometimes can 'brick' a router).

QoS > Basic Settings > Inbound Rates/Limits > Max Bandwidth Limit is where you set this, and the term 'Global' is not mentioned there, but you will see that the small-print advice on the right of the field looks different at that spot than in your existing build.

I have posted other details earlier in the current thread.

Cheers !

EDIT : OOOOOPS ! - I said Outbound in the bolded path, when I meant to say IN-bound. - I've now changed that.
MaverickHL
join:2006-01-20

MaverickHL to Cloneman

Member

to Cloneman
said by Cloneman:

I'm referring to the max bandwidth limit under inbound, on QoS basic settings page.

Older versions of shibby explicitly say "this is NOT a global maximum!"

As for ports I was hoping you connected VPN provider always connected on the same DESTination port, that would be an easy way to classify.

The only port provided on the VPN is the one used to connect to the VPN which is just 1194. I could give that a try and map the destination IP address and port which I used in the OpenVPN client and classify that, but I was not sure if that actually would work.

@Davesnothere

I will do an update this weekend then and give it a shot. Would I need to adjust my QoS for inbound? I was under the impression that there is not much control over inbound traffic anyways, so I just let it go as None for everything.