dslreports logo
 
    All Forums Hot Topics Gallery
spc
Search similar:


uniqs
2441
jonboy545
join:2013-08-29

jonboy545

Member

5mb upload but can't get more than 1.5 for web server/SMB

Hello all,

Let me preface this by saying that I've been a long time tinkerer, but nowhere close to "professional" regarding my computer skills. I recently upgraded my internet to Time Warner "Extreme" 30/5 plan. I really wanted the upload, less concerned with the download. I have the Arris modem/router, with one port feeding a WRT54G running DDWRT as an AP to extend wifi range. I have a home theater PC and another "server" PC for IP Cam CCTV DVR functions, VPN access, and a few other things (SMB, Web Server) that is connected by Cat5 to the WRT54G.

Every speed test that I can come up with is showing that my upload is every bit as good as the 5mbps promised. DSL Reports, Testmy.net, Speedtest.net, etc. I know speed tests aren't everything so to further confirm I tested by uploading a youtube video and observing bandwidth using Windows' Resource Monitor or WRT54G's bandwidth monitoring. The SLOWEST I've seen it is 4mbps fastest is 5.5.

My main purpose for the 5mbps upload was to stream content from my HTPC to my phone/other devices when I'm out of the house. Unfortunately my experience has been lackluster.

Some scenarios:

>Cell phone connected by 4G (Samsung Galaxy Note 2 on Sprint LTE)
Run speedtest. 10mbps down, 3mbps up. Good connection.
Connect to VPN at home
Run speedtest connected to home VPN: 4.8mbps down, 3mbps up. VPN connection seems good!
Try and download a file from SMB share - 1200kbps (150KB/sec)
Thought maybe windows' samba was to blame, tried installing various web servers (apache 2.2, HFS, Quick'n'Easy HTTP) same thing. 1000-1400kbps
Disconnect from VPN to try and rule that out. Just accessing Apache's directory listing on web server. Virtual Server on Arris router forwarding port 80 so web server is visible outside router. Can connect just fine, still only 1000-1400kbps
Connect back to my home's WiFi at home: web server sends files at anywhere from 8-14mbps (usually around 1.5MB/sec)
SMB share from within the network is same speed as web server. 8-14mbps.
I'd like to safely say that it's not an issue with the server iteself, as data speeds within the wifi network are what I'd expect them to be. The trouble is the "outside world."

Scenario 2 to rule out cell phone

>Friend's computer who is also on time warner cable on the other side of the city. 50/5 connection at his house.
Run speedtests. 40ish down, 5 up. No connection issues apparent.
Pretty much the exact same as above. Whether I'm connected via VPN accessing SMB or web server, speeds are around 1 to 1.5mbps. Accessing the web server through port 80 (no VPN) results in the exact same.

I've tried changing the web server ports, maybe they're throttling port 80 (even though they PROMISE they don't). Tried 81, 8080, 8089, 8889, and some random ports here and there. All result in the same lackluster speeds.

QoS is disabled on the WRT54G. Have no idea about the Arris modem/router. Only QoS I could find is the WMM, which I have disabled.

The one "saving grace" is I have achieved speeds faster than 1.5mbps upload from my home computer, but there's a third party involved. I use a program called "iSpy" for my IP CCTV cams for motion detecting/recording. Technically iSpy doesn't allow remote access unless you pay for a subscription service. It's a web based interface. You are connecting to www.ispyconnect.com for the web interface but all the files are sent to you via your local LAN Ip on port 8889. I figured out that if I connect to my home network VPN I can access the iSpy content. When I download or stream a recorded video clip (connected to my home's VPN) I can achieve download speeds of around 4-5mbps. What I think is going on is my home PC is uploading the video to iSpy's servers, who is then sending it to me. These files are NOT stored on iSpy's servers. They are sent at the time of the request. No caching. I guess it's like a proxy?
I'm not sure if that has any relevance, but I found it odd.

Sorry for the lengthy post, but I wanted to cover what I've done thus far to try and rule some things out instead of starting with "level 1 tech support" type questions.

TL;DR:

I have 5mb upload. Uploads to "youtube" and other sites get 5mbps upload. Traffic from home to a client are only 1.5mbps
jonboy545

jonboy545

Member

Anyone have any ideas?

justin
..needs sleep
Mod
join:1999-05-28
2031
Billion BiPAC 7800N
Apple AirPort Extreme (2011)

justin to jonboy545

Mod

to jonboy545
So the issue is you can upload to youtube, or to speed test sites, apparently at ~5mbit, but when you run a server in your house, you can't seem to pull from it, from anywhere, at speeds more than 1.5mbit.

First VPN will always kill bandwidth, so that gets added last. If you can't get good speeds without the VPN you won't get them by adding that into the mix.

Second you're hoping that the WRT54G when you open a port on it, is actually not getting in the way. Uploading to youtube over it is not the same as opening a port on it, and mapping that port to your server. If you can think of a way of eliminating it from the equation try to do so. For instance can it go into bridge mode?

At your mates 50/5 place, see if you can convince him to run a tiny FTP server and see if you can upload to it from your house. Run the same FTP server on your end and see if you can pull from it.
jonboy545
join:2013-08-29

jonboy545

Member

Hi Justin thanks for the reply

I am aware VPN will eat up some bandwidth. When connected to my VPN on my phone's 4G connection I can pull 4 to 4.5mbps down which is about 90 percent of my home internet upload. Seems right.

The WRT54G is in bridge mode. DHCP, and port forwarding is all handled by the time warner Arris modem/gateway.

I have a FTP server running on my "server" on port 21. Port forwarding on the Arris points to that computer. Within the wifi network I can pull about 9mbps. When at my buddy's house accessing it results in the exact same Slow speeds.About 80 to 90 KB/s or 768kbps. This is done testing NOT connected to the VPN.

justin
..needs sleep
Mod
join:1999-05-28
2031

justin

Mod

VPN doesn't just eat bandwidth, it also caps the speed.

Well then you need to get a sniffer and capture the FTP session uploading data at 768kbps then breakout the TCP analysis tools.
jonboy545
join:2013-08-29

jonboy545

Member

Ill have to do some research. Have never done that, or where to start. If you feel so inclined throw a dog a bone, in the mean time ill start my research and see if I can make some headway and post said results if I figure it out.

At this point im willing to throw the VPN out of the window. Accessing a samba share on my android would sure be usefull, but nothing I couldn't get around running a simple webserver with some type of PHP fancy directory listing for accessing my content on my phone. Mainly videos. I have a feeling whatever is causing this slow down for http & ftp will also cure SMB issues.

justin
..needs sleep
Mod
join:1999-05-28
2031
Billion BiPAC 7800N
Apple AirPort Extreme (2011)

justin

Mod

There are windows packet capture utilities, free ones. I'm sure you can google up one. It doesn't have to be fancy, it just has to capture all packets on your interface that are from or to a certain IP and/or port.

If you go to the trouble of capturing the totality of a slow upload into a file and post the file, I can run tcptrace on it to see what is going on. Hell I might even write a tool where you upload a capture file, and get back the analysis on a web page, as I've already got the tools to turn the data into plot files, and there doesn't seem to be one out there everyone runs tcptrace on a capture under linux and looks at the textual output.

Raphion
join:2000-10-14
Samsara

1 edit

Raphion to jonboy545

Member

to jonboy545
[edit]changed mind on potentially wreckless advice[/edit]
jonboy545
join:2013-08-29

jonboy545

Member

Huh? Im fairly certain I can make a self assesment for what may be wreckless. I value any input!

Raphion
join:2000-10-14
Samsara

Raphion

Member

said by jonboy545:

Huh? Im fairly certain I can make a self assesment for what may be wreckless. I value any input!

My advice would only be valid if you're running the servers on Windows... are you?
jonboy545
join:2013-08-29

jonboy545

Member

Yes my HTPC which hosts the media content and handles VPN access is running XP SP3. My CCTV storage is a box running win 7. It also hosts some content on a windows smb share. Ive tried both the FTP and HTTP servers running on both boxes. Ive even ruled out the WRT54G by plugging the boxes directly into the TWC Arris modem/gateway. No change.

Raphion
join:2000-10-14
Samsara

1 edit

2 recommendations

Raphion

Member

Ok then, specific to Apache, you need to disable MMAP and SendFile. These are *nix functions not available in windows, yet the windows binary of Apache has these set enabled by default. The server thus hangs when trying to call them, times out, then proceeds without, but will attempt to call them again next time, always imparting a large delay. Disable them by finding "EnableMMAP" and "EnableSendFile" in httpd.conf and replacing the "on" after them with "off".

For windows uplink in general; window's default send window is far FAR too small for the latency of an internet connection. To increase it, go into the registry to (HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\AFD\Parameters) and if they are not there already, add DWORDs "bufferMultiplier", "DefaultRecieveWindow", and "DefaultSendWindow". I get fantastic performance on my 25/25mbps connection with the buffer multiplier set to 512, and both windows set to 1048576. That window size is best for high speed, high latency connections, but is too high if you have much packet loss. A quarter of that would be enough for the SendWindow on 5mbps upload.

There is also a DWORD "GlobalMaxTcpWindowSize" at (HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Tcpip\Parameters), I would expect, but I'm not certain, that if you don't manually configure that as high as the previously mentioned window settings, they would be overridden by this entry. I also have the DWORD TcpWindowSize there set the same.

Note, Apache config changes require restarting Apache, and the TCP parms will require a restart of Windows to take effect.

I believe that with those things done, you'll see much better performance, both on your upload and download.
jonboy545
join:2013-08-29

jonboy545

Member

BINGO!

DING DING DING ROGER WE HAVE A WINNER!!

Raphion I do believe you did it!

I added the three DWORD values as you described (they were not present), however "GlobalMaxTcpWindowSize" was not present under ....\Tcpip\Paramaters. I first tested without adding it, however I have immediate results so I'm not going to add it.

I achieved a sustained 5.~ mbps download for half an hour! Peaking at 5.3mbps usually hanging around 4.9!!

Downloaded a 1.10 GB file in just under 30 minutes, which comes out to about 5mbps average!

This was over FTP or HTTP. Seems my SMB speeds are still very slow when over 4G.

i also accessed the apache server over VPN by pointing the browser to »192.168.0.xxx and it too achieved speeds of 5mbps, so the VPN isn't really slowing me down at all.

Do you know of another area in the registry specifically for windows file share that may control the same things that you posted above?

Apache is working great though! I can stream a HD movie to my phone over 4G with less than 5 seconds of buffering. Amazing.

Thank you, thank you, thank you!!

justin
..needs sleep
Mod
join:1999-05-28
2031

justin

Mod

DrTCP lives on.

You'd have thought that windows would have sorted that out by now. It isn't like your latency is that high anyway.

Raphion
join:2000-10-14
Samsara

Raphion to jonboy545

Member

to jonboy545
Glad to help.

For SMB, try putting DWORD "SizReqBuf" at (HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\lanmanserver\Parameters) and set it at 65535 decimal. That's as high as that one goes, and should give you a significant improvement over the default.
jonboy545
join:2013-08-29

jonboy545

Member

Thank you for the help justin and Raphion!

Unfortunately that did not help my SMB speed. This has been the worst performer throughout this journey. I'm still only getting around 40KB/sec (320kbps). When on local WiFi it is slower than other methods, but still gets around 6-7mbps, was hoping for a little better.

I remember reading somewhere online a while back that windows file share (please forgive me if I describe this wrong) asks for a confirm receipt of packets too frequently, or maybe it was the packet sizes were too small, so you were heavily reliant on a low latency/fast upstream on the client end, or performance was diminished. There was a fix but 1) I can't find the site and 2) I think it applied to Windows Server 2003 only.

This may be too much information but maybe somebody's been down this road before. Doubtful but hey... ya never know.

I'm running the android version of XBMC (home theater software) on my android phone to hook up to a TV when traveling. XBMC organizes both local stored content, and streams (1Channel, IceFilms). I'm reliant on SMB for two reasons:
1) To keep the library database updated. I'm working on using SQL to do this, and should be able to point the SQL database over the web instead of a SMB location
2) The locally stored media location the database reads needs to be the same on the home PC as it is on my mobile device, so by having the library access the media content as a file share location on both the HTPC and my mobile, they both stay "in sync" and provide a consistent user experience. When the whole system works it's pretty awesome, but getting around some things has been a real PITA (like this speed limitation).

Everything seemed perfect on paper. When I'm away from home, as long as I have an internet connection good for 3-5mbps I was planning on VPNing to my home network, fire up XBMC and have it read the library database and content from home server. Even if I chose an online stream, the bandwidth from my home VPN would be more than sufficient, as much of the content is 1-2mbps video quality.

Since XBMC IS basically my cable box (no cable/sat provider, we went 100% streaming/download) I thought how awesome would it be to be able to bring my home "cable box" experience with me anywhere. Better than TWC or Uverse could ever do... not a limited selection of a few shows, and limited selection of live programming but to actually have the entire "cable box" UI on my mobile device anywhere. Everything else is in place... all that's left is getting SMB speed up to par with the web server and I'll be one happy nerd!

Just wanted to say thanks again for the help thus far. I finally feel like I'm getting my money's worth with my upload!
jonboy545

jonboy545

Member

Ah now I see that was precisely what you asked me to change. So is it safe to assume that 65KB being the max size, performance won't be any better without a lower latency? Being on 4g and connected to a VPN this is as good as is going to get 100-150ms ping time?

Raphion
join:2000-10-14
Samsara

Raphion

Member

I'm afraid I don't know what else to try for SMB right now. There is something in the back of my mind, but I'm not quite recalling what it is. I'll post if it surfaces.
Raphion

Raphion to jonboy545

Member

to jonboy545
No new advise on the SMB issue, but I thought I would suggest that you setup password protection on your apache shares, especially since you're accessing them from google devices (android). Google WILL index that stuff, and when videos and things on your server start coming up on google, you could get into some deep doo-doo.

Here's an authentication tutorial »httpd.apache.org/docs/2. ··· uth.html The examples use default linux paths, so you'll need to change those to work with the windows installation. Also need to make sure the relevant modules load of course.