Brano
I hate Vogons
Premium,MVM
join:2002-06-25
Burlington, ON
 ·TekSavvy Solutions..
| USG200 speed tests #2
With the release of 2.12 firmware and as follow-up to »USG200 speed tests I've done some additional tests.
USG 200 2.12 netperf tests
Testing environment:
Server on LAN: openSUSE 11.1, gigabit NIC
Client on WAN2: openSUSE 11.1, gigabit NIC
All tests are done netperf TCP_STREAM as per below (default test):
Setup notes:
- BWM and CF was always Off
- There were some other firewall rules on the USG defined and active, but the Internet WAN link was Off
- AppPatrol all default, forward
- AV ZyXel engine, all protocols on
- IDP default, WAN to any
- ADP default, WAN to any
- AS default, WAN to any
Performed tests:
First I've done 3 subsequent 10 seconds tests. Then I've started over and done one 30 seconds test.
Notes on results:
Services such as AV, AS are supposed to be scanning only specific protocols i.e. HTTP FTP SMTP POP3 IMAP4 thus not sure how good are these tests since I was using netperf on some random port non-specific protocol. Interestingly AV was slowing down significantly!
AppPatrol speed is significantly improved from 2.11 firmware. Actually the results seem too high, I'm not sue if the packet inspection engine was kicking in since I was not using any specific messenger or other monitored app for testing.
IDP however seems to be corrupted in this version, speeds were constantly below the advertised! Additionally IDP speeds were significantly jumping up and down from test to test.
Firewall throughput is also below advertised.
Advertised speeds:
Firewall Throughput: 150 Mbps
UTM Throughput (AV+IDP+Firewall): 40 Mbps
Final thought
I'm quite confused about these results. Maybe every specific engine needs to be tested with specific protocol. Also the fact that AV/IDP results were up/down from test to test makes me feel uneasy about scanning engine(s) implementation.
I would definitely not make any decisions and/or conclusions based on these tests. Just FYI for those who are interested ;)
--
openSUSE 11.1, KDE 4.2 |
|
ba142
join:2007-06-10 | Well done.
Would you please test the BMW Throughput under AppPatrol? |
|
bbarrera
Premium,MVM
join:2000-10-23
Sacramento, CA
clubs:
·SureWest Internet
| said by ba142  :BMW Throughput on the autobahn?  |
|
Brano
I hate Vogons
Premium,MVM
join:2002-06-25
Burlington, ON
·TekSavvy Solutions..
| I couldn't digest the results so I ran another test this time using ftp protocol. I was downloading exactly the same 330MB zipped iso as in my first 2.11 set of tests »USG200 speed tests
FTP server was vsftpd and I was downloading with wget redirecting output to /dev/null to get rid of possible disk write delays.
Here is an example from just the firewall test
with following results
USG200 2.12 ftp test
Comments on results:
Tests on AppPatrol, AV, IDP, ADP should be valid as all these are scanning FTP protocol.
The AS test is invalid since AS looks at SMTP and POP3 only.
Interestingly the AV and IDP tests speeds were varying from 35KB/s to 8MB/s during the one file download. Now I'm convinced that AV and IDP have serious issues!
Overall the device seems to be not delivering to advertised specs with firmware 2.12(AQU.0) :(
--
openSUSE 11.1, KDE 4.2 |
|
style4me32
@com.tw | For IDP/AV performance, you can try to test it with multiple connections. The aggregate multiple session performance will be much better than single session performance.
BR,
style4me32 |
|
Brano
I hate Vogons
Premium,MVM
join:2002-06-25
Burlington, ON | It probably would, but that's not the point of my test.
DUT should perform to advertised specs for single session.
--
openSUSE 11.1, KDE 4.2 |
|
dslpartner
join:2005-02-18
| said by Brano :It probably would, but that's not the point of my test. DUT should perform to advertised specs for single session. Grey area 
--
"Perl is executable line noise, Python is executable pseudo-code."
|
|
Brano
I hate Vogons
Premium,MVM
join:2002-06-25
Burlington, ON
·TekSavvy Solutions..
| See my #1 speed test with 2.11 firmware »USG200 speed tests
The performance then was mostly up to specs (excluding AppPatrol).
Something seems to be wrong in 2.12.
--
openSUSE 11.1, KDE 4.2 |
|
ba142
join:2007-06-10 | reply to Brano
Under the ftp test, does the USG200 has high CPU usage issue? What is the USG200 CPU usage when download the 330MB iso file? |
|
Wintel
join:2009-02-19
| I have a 30Mbps Internet connection (PPPoE) and using the ZyWALL USG200, when downloading using FTP the USG200 CPU usage is quite high 60%~80%.
When downloading a 1300MB ISO file using FlashGet and set to 8 simultaneous connections in microsoft.com, the USG200 CPU usage is >85%, does the USG200 has a PPPoE or CPU usage high issue?
Testing ISO File:
»www.microsoft.com/downloads/deta···3d06d6cb |
|
Brano
I hate Vogons
Premium,MVM
join:2002-06-25
Burlington, ON
·TekSavvy Solutions..
| reply to Brano
I've re-done the tests with the 2.12(AQU.1) firmware release.
...the results are not a bit better 
--
openSUSE 11.1, KDE 4.2 |
|
Anav
Sarcastic Llama? Naw, Just Acerbic
Premium
join:2001-07-16
Dartmouth, NS
| reply to Brano
Hi Brano, based on your experience, if one has a 25/25 service and wanted to procure a USG router which one would you lean towards?
A. assume firewall plus AV plus IDP in use.
I did find some differences in available literature between US and UK sites both with 2009 data.
According to the spec sheet from zyxel USA for the USG200
Firewall / VPN / AV / IDP / AV + IDP + FIREWALL /
150 Mbps / 75 / 50 / 65 / 45 Mbps
and for the USG300
200 Mbps/ 100 / 70 / 75 / and 70 respectively.
I am assuming that those are all two way throughputs and for one-way we would have to divide all those numbers in half.
USG200 - 75 / 37.5 / 25 / 32.5 / 22.5
USG300 - 100 / 50 / 35 / 37.5 / 35
From the UK site a slightly different story ????»www.zyxel.co.uk/upload/doc/QSG_S···2009.pdf
There are no AV only or IDP only numbers for starters.
Firewall / VPN / AV + IDP + Firewall /
USG200 - 150 / 75 / 24
USG300 - 200 / 100 / 48
As you may surmise, it appears that the UK VPN and AV numbers are two way throughput and the combined AV + IDP + Firewall numbers are one way throughput. All very confusing????
--
Ain't nuthin but the blues! "Albert Collins".
Leave your troubles at the door! "Pepe Peregil" De Sevilla. Just Don't Wifi without WPA, "Yul Brenner"
LlamaWorks Equipment |
|
Brano
I hate Vogons
Premium,MVM
join:2002-06-25
Burlington, ON
·TekSavvy Solutions..
| Well, I have done only single session tests as per above and only on USG200. That said, AV speed seems to be the deal breaker for that test.
One would need additional tests, multi-session and single session tests each way to get the overall picture.
Solely based on ZyXel specs USG 200 should be able to handle 25/25 assuming they fix things in next firmware releases.
--
openSUSE 11.1, KDE 4.2 |
|
