dslreports logo
 
    All Forums Hot Topics Gallery
spc
Search similar:


uniqs
5669
jcremin
join:2009-12-22
Siren, WI

jcremin

Member

Mikrotik 5ghz 11n PTP advice

Hello All... I am having a problem that I'm trying to solve and keep running into dead ends. All of my tower to tower backhauls are 5ghz. Over the last 5 years, I've been using single polarity with XR5's and Nstreme enabled on a 20 mhz channel. Great throughput, consistent 1-2 ms pings, and very reliable.

I have recently been upgrading some of the main backhauls to dual polarity 802.11n using a few different minipci cards (DBII and UBNT SR71-15). I had been switching to using NV2 in a 40mhz channel, which has generally been working well, but my latency is way higher and I have a lot of jitter, especially when the throughput is lower.

I've been trying to convert some of these links back to regular Nstreme since it seemed so much more consistent, but now I'm running into a different issue. My TX/RX rates keep dropping way down (especially when the traffic is low) and this is causing my CCQ to drop, and ping times to go up. If I run a bandwidth test, the rates will slowly increase and pings will stable out again. I can run the test for an hour without any issues, but shortly after stopping the test, the rates drop again and I end up in the same situation.

I am trying to avoid locking the rates, because some of these links are over 15 miles, and the results of thermal ducting can sometimes cause my signals to drop quite low (most links are in the -55 to -60 range, and during ducting can drop to -80 or worse). I don't want the link to drop completely if the signal gets low... I do want it to be able to fall back, but I don't want it to be falling back when the signals are really good as it is creating these problems for me.

Anyone have any ideas?
jcremin

jcremin

Member

A little extra info, if I uncheck "enable polling" and "disable CSMA" in the Nstreme options, it looks like the rate is staying higher, but my throughput has dropped in about 1/2 or less and ping times are much more erratic.

Inssomniak
The Glitch
Premium Member
join:2005-04-06
Cayuga, ON

Inssomniak to jcremin

Premium Member

to jcremin
I cant say I have terribly good news for you, but I never had incredible luck with 11n nstreme or nv2 on Mikrotik. Seems OK with straight 802.11 though.

I had a 5km link, Single chain nv2, or nstreme, never worked well, it was a Mikrotik .11n card R52Hn I think on both ends, but for some reason it just worked like crap. It was a critical backbone link in the middle of our main backbone.

I replaced the whole link after one of the antennas leaked water, with a nano bridge M5 link, all other things being equal, (same channel, same width), and was expecting all sorta of the same issues, but it was perfect, fast, stable, and low latency. Ive never had to look it again.

What was the problem? No idea.

Only thing I would try, is a reset-configuration, make share HW retires is 12 or better if a PTP. Run latest FW from mikrotik.
jcremin
join:2009-12-22
Siren, WI

jcremin

Member

said by Inssomniak:

I cant say I have terribly good news for you, but I never had incredible luck with 11n nstreme or nv2 on Mikrotik. Seems OK with straight 802.11 though.

Hmmm, interesting. I did try dropping to straight 802.11 and for some reason the data rate would never go above the regular 54mbps, despite being set to "N-Only". One of the links is about 25 miles, so I'm not sure I could even run straight 802.11 on it because of the distance and ack problems. I know it wouldn't work at all when it was 36 miles before I broke it into two hops.
said by Inssomniak:

I had a 5km link, Single chain nv2, or nstreme, never worked well, it was a Mikrotik .11n card R52Hn I think on both ends, but for some reason it just worked like crap. It was a critical backbone link in the middle of our main backbone.

And that's the funny part, all of my links are great. Awesome throughput and very stable, except two minor issues. If I run NV2, the latency is all over the place, and if I run Nstreme, the data rates are all over the place. If I keep at least 30 megs of traffic flowing, it never has an issue.
said by Inssomniak:

I replaced the whole link after one of the antennas leaked water, with a nano bridge M5 link, all other things being equal, (same channel, same width), and was expecting all sorta of the same issues, but it was perfect, fast, stable, and low latency. Ive never had to look it again.

I've seriously been considering trying UBNT because of all the posts on the mikrotik forum that mirror your statements. I can't tell you how many posts I read that went something like "I fought with MT for months, tweaking every setting, but put a cheap pair of UBNT radios up and they just worked".

I have serious reservations about UBNT on my main backhaul. I hate troubleshooting problems with UBNT, and have never been able to bring myself to use them on a main backhaul before. I have a couple 2.4 AP's switched over, and almost every day I consider switching them back to MT. I do have a couple Rocket M5's in the back room that I guess I could slap up between a couple towers a backup link and try them out.
said by Inssomniak:

Only thing I would try, is a reset-configuration, make share HW retires is 12 or better if a PTP. Run latest FW from mikrotik.

I am running a few versions older than the latest. I do see a few improvements for NV2 latency. Nothing specific to Nstreme or data rates, but if NV2 can have lower pings with less jitter, I never had any other problems with it so I'd gladly switch back if I could get those under control.

Thanks
jcremin

jcremin

Member

Well after upgrading all radios to the latest production version (5.22) I can say that NV2 latency is much improved from the 5.17 version I was running before. Of the 4 802.11n links that I worked on tonight, I ended up setting 3 of them back to NV2, with average (no traffic) pings of 2-3 ms and only occasional spikes, which were all under 10ms. Two of these links are about 15 miles and one of them is about 1/2 mile. The one longer 25 mile link ended up staying in standard Nstreme mode because NV2 still had a lot more jitter despite most pings being ~4 ms. With regular Nstreme pings are a rock solid 1ms.

Having NV2 enabled keep the data rates up near their max like I would expect them to, and appear to only fall back if there are lost packets or if the signal drops. On the short links with Nstreme, every time there was no traffic the rates fell down to 30mbps or less even though the signals are all -55 to -60. For some reason, the one longer 25 mile link doesn't appear to have this same problem, and despite signals in the -70 range, the data rates stay pretty solid at 180 in both directions.

I'm happy with my results so far, and while I'd love to have the consistent 1-2 ms pings that Nstreme seems to produce, NV2 is only slightly higher and definitely stays much more stable under heavy traffic.

Semaphore
Premium Member
join:2003-11-18
101010

Semaphore

Premium Member

Click for full size
1/2 of my network backhaul is 5.22 with A/N + dual pol + NV2. On PtP links you can set the NV2 Distance to actual + 10% and the period size to 1. I find that the NV2 links Idle at 12%/12% CCQ. When traffic hits the interface they pop to actual CCQ (which should be >98%) and Modulation stays max for the link. I have a dozen links like this and they sustain 70Mbps with nominal increase in Latency.
Above is latency graph across 5 x NV2 hops for last 30 hours. We hit 70 Mbps at 5 and 11PM
jcremin
join:2009-12-22
Siren, WI

jcremin

Member

said by Semaphore:

When traffic hits the interface they pop to actual CCQ (which should be >98%) and Modulation stays max for the link. I have a dozen links like this and they sustain 70Mbps with nominal increase in Latency.

How low is your throughput at the lowest (idle)? I don't hit anywhere near 70mbps on any of my links, they top out around 30 - 40 mbps. I'm guessing that means that my "idle" periods are much lower than yours too maybe??? I was seeing the most problems late at night when the throughput fell well below 10 megs.

Also during the day, if the traffic dipped down for even a minute or two and the link had the chance to modulate down, a sudden burst of traffic would cause ping times to spike before it had a chance to react and increase the rate back up. Until I had a chance to work on the links again last night, I actually started a btest (limited to 10megs) just to keep traffic going and keep the data rates up. Worked like a charm, but not necessarily the best use of spectrum.
jcremin

jcremin to Semaphore

Member

to Semaphore
said by Semaphore:

I find that the NV2 links Idle at 12%/12% CCQ.

I should add that the only time I had issues with the link modulating down and causing issues was in plain Nstreme mode. NV2 (which I realized is what you said you were using after I made my last post) doesn't exhibit the same behavior for me. The 3 links that I switched back to NV2 last night sat steady with 300/300 data rates all night long and are still there now.

Just wanted to make sure I was clear about that. Thanks for the input!

Joe

Semaphore
Premium Member
join:2003-11-18
101010

Semaphore to jcremin

Premium Member

to jcremin
Bottom end would be not less than 10Mbps, and more often 20 to 30Mbps. I have 2 fiber drops and the network is load balanced based on geographic locations... otherwise I'd be running over 120Mbps at peak on a single pop.