Equipment Support > Hardware By Brand > Linksys > Benchmarking WRT Firmware... Some Surprises!

Benchmarking WRT Firmware... Some Surprises!

What if you overclocked the WRT54G*? I'd like to see if that would make a difference.

Not too much of a big deal really, for internet this router still owns, and I put a gigabit switch after it hooking up my other computers so LAN transfers are fast anyways.

reply to Bill_MI
Interested to know if the router under test is doing NAT (with connection tracking) in Linksys Gateway mode or just routing.

Also how complicated are the iptables rules, its a long time since I used stock Linksys firmware but I recall they were pretty simple, Tomato etc create lots of extra tables/chains and there must be an overhead if every packet has to be checked against everything - and also scope for optimisation.

Hi mstombs and TomatoUser, good questions.

I'm not sure I'll have a chance to get it going again for awhile but I'd not raise much above 200MHz. I do run 216MHz in production routers for that tiny 8% CPU gain But these are ALL at default 200MHz unless (and I didn't check) any set faster by default.

I know iptables/conntrack is running on OpenWrt and assume it is on all the others. Whatever their defaults are is how it ran. This is same as "factory defaults" or just plain erasing NVRAM which forces NVRAM initialization.

Yes, they all did NAT/PAT. The WAN pulled an address from my main network and my main box was on the LAN (static IP 192.168.1.2). Interesting is how every one of these firmwares needs ZERO config to work like this.

I'm taking note of what to try next... perhaps over the holidays.

I'm definitely going to capture iptables and nvram for each condition... at least.

reply to Bill_MI
In the Tarifa firmware I made a memory optimization,
at startup it detects the amount of RAM installed and calculates the size of the hash-tables for TCP-ip routing/nat-ing/tcp-ing.

Standard Linux, the hash tables size set by size of RAM-memory and will be calculated as a desktop pc. These sizes should be increased, if used as a router.

The standard Linksys firmware is indeed fast, but it can be made faster. The modulo-logic hash table, can be replaced by an and-logic hash table, this will increase speed.
I made a firmware called Tarifa_034_basic (not released), this is simply the Linksys 4.30.12 firmware, with all packets updated and telnet activated on default. This firmware was actually only a little bit faster.

This is also the reason why the standard firmware slows down on high amount of connections.
The (is default) Hash implementation is very small 128 (16Mb) or 256 (32Mb) places (buckets), but due to this stupid size, a huge amount of hash collision will easily occur, this making the router slow down.

All the third party firmwares do a better job under stress, you should test 1,2,3, etc connections....even between 1 and 2 connections you will see strange differences.

More: the protocol can make a difference, test http, ftp and you will see speed difference.

An other strange thing I found, that some firmwares are fast and do 90% processor-load and others are a bit slower and do around 50% pr-load. Till now, I have not found out why the processor doesn't do nothing during the spare time.

Greetings,
jchuit

Thanks for the testing done so far though, really interesting to know.

reply to jchuit
Hi jchuit! Great to hear from the source. I haven't played with Tarifa but sure am aware. Just an old EE learning bits and pieces on the Linux world both desktop and embedded at the same time that latched onto OpenWrt at about RC4.

Something auto-adjusting to memory would very much be affected by this GSv3. I'd love to get your opinion if it may affect any of my results above.

And if I get to doing this again, would Tarifa034RC1 be a good candidate to try? I see the changelog has what you are referring. Perhaps I could change some of those memory controller NVRAM variables to limit memory. I'm pretty sure, based on the actual board, this GSv3 is otherwise the same.

EDIT: And if you have any measurements or intuition, do my results look what you'd expect? are they sane?

reply to scubascythan

reply to Bill_MI
I didn't buy a WRT for wired access.

reply to Bill_MI
To Bill

The values you found are very sane, I tested hyperwrt, tomato, Linksys and Tarifa.
I also concluded that the original is the fastest, but also the most unstable.
10% increase in clock freq will give 10 % increase in speed, 200->216MHz.

Here is an old test with one HTTP connection:
»ftp://krumdeel.dyndns.org/Public/wrt54···peed.xls
As you can see the Linksys firmware is the fastest, and the wrt54gl is a bit faster then the old wrt54g hardware. These firmwares use the obsolete but fast Modulo hashing.

Later, I made an Tomato test, and here I tested with aida32:
»ftp://krumdeel.dyndns.org/Public/wrt54···mato.bmp

In this test the tomato was a bit faster, but all other processes were very slow on tomato. It looks like, the processor is the main slow down, but what it makes slowdown is unknown by me. Maybe it is some kind of processor setting in Linux somewhere, also strange that the processor sometimes run on 50%.

The strong point of Tarifa034RC1 is the Lookup3 hash implementation, this should be in the standard Linux distros.

This is also the weak point. Lookup3 does the best job on processor with an Rotate instruction, but huge stupid the MIPSs processor family doesn't have a Rotate instruction, meaning an workaround of 3 extra processor cycles is needed.

By the way, lookup3 is still better then the lookup2 implementation. The main advantage is that it makes a lot of simultaneous connections possible, without making the router slowdown.
see: »burtleburtle.net/bob/c/lookup3.c

reply to SpeedBooster

reply to Bill_MI
Bill, could you try flashing the router again with a current DD-WRT build since v24 is rather old now. I'd be interested in seeing how build 11100+ performs with your setup.

Hi Swinn, I can find no valid "STD 2.4 SP1" build if that's what you mean (see original post). I haven't tried mini, vpn, voip, etc.

EDIT: I may have found the problem. Some or all "BIN" files are not standard Linksys flash images. They seem to already be striped of the Linksys header and are in fact TRX files already. Can anyone confirm this? I'm not getting any warm fuzzy feeling when every model directory takes you to the same set of files.

I'm not sure. I've always tftped the bin to an openwrt router in order to flash it. I've never messed with a trx file.

This is the latest Eko build as of today: »www.dd-wrt.com/dd-wrtv2/download···_std.bin

I'll see how it goes when I setup again.... probably over the holidays.

Bill, thanks for the test, the results are interesting. Perhaps you can do a test with various number of concurrent streams, similar to Tim Higgins' in this review: »www.smallnetbuilder.com/content/···6843/51/ (tests on pages 4 & 5). Not sure though if it's possible without some kind of commercial software like IxChariot...

reply to Bill_MI

New Results

LAN-to-LAN                             85.20
Tomato 1.23                            58.42
Linksys 4.71.4.001                     50.12
Tomato 1.22                            46.84
Tarifa 034RC1                          46.40
OpenWrt Kamikaze 7.09                  35.46
OpenWrt White Russian 0.9              35.30
DD-WRT 2.4 STD                         31.78
DD-WRT 2.4-SP1 STD                     31.32
DD-WRT EKO v24-11218_NEWD_std          30.72
DD-WRT EKO v24-11100_NEWD_std          30.72
OpenWrt Kamikaze 8.09RC1 (2.6 kernel)  24.24

Another interesting thing you may want to consider might be the latest Tomato build 1.23_ND with TCP Vegas enabled, toss that into your benchmarks and see what the ND (New Driver) builds do.

I've been running 1.23_ND with TCP Vegas on and have been curious how LAN-WAN and WLAN-WAN performance stacks up against the normal as well as other 3rd party builds. I think there might be something interesting there.

Great stuff!
--
"Some people never see the light till it shines thru bullet holes." -Bruce Cockburn

I'm testing Gmail's spam filters: Broadbandreports1@gmail.com
Spam: 12900+ messages currently using 406 MB.

Hi NT, After reviewing and repeating a strange "hole" around 50mbps I decided to do these all over again to the same server as before.

The "ND" version does not seem to be for this platform. However, something significant happened in Tomato 1.23. Could this "new driver" be involved? Or could it be as simple as a default setting change?

For anyone interested, I got better results to my laptop as the iperf server on a direct LAN-to-LAN connection - averaging well over 90mbps. But something happened to the speed to Linksys and Tomato firmware (I call it a 50mbps "hole"). After repeating the same strange results I decided to return to using my main server and repeating all the tests. Now they compare to the original results (and Tomato 1.23 scores even higher!). I've edited the results to the newer data.