dslreports logo
 
    All Forums Hot Topics Gallery
spc
Search similar:


uniqs
566

TAZ
@qwest.net

1 recommendation

TAZ

Anon

Quit comparing to multi-WAN routers

This is completely different. Multi-WAN routers only assign a flow to a particular connection, so the throughput you get on that connection is limited to the throughput of whatever connection it was assigned to.

This, on the other hand, is able to provide the aggregate throughput of all connections, by fragmenting packets, sending them through all connections to their proxy service (ignore the "cloud" bullshit marketing), and reassembling them there.

It's nothing new either, this has been done for awhile (Google "VPN bonding" and there have been some other startups doing it too).

ilikeme
Premium Member
join:2002-08-27
Stafford, TX

ilikeme

Premium Member

Sounds like someone works for the company that makes this.

Mellow
Premium Member
join:2001-11-16
Salisbury, MD

Mellow to TAZ

Premium Member

to TAZ
Exactly.

But now you have 1 central point of failure added to the mix.

exocet_cm
Writing
Premium Member
join:2003-03-23
Brooklyn, NY

exocet_cm to TAZ

Premium Member

to TAZ
Untangle can do WAN balancing but not WAN aggregation.
»wiki.untangle.com/index. ··· Balancer

This product claims to do aggregation, which is what you said. But even then, how many have the money to pay for multiple ISPs?

Nightfall
My Goal Is To Deny Yours
MVM
join:2001-08-03
Grand Rapids, MI

Nightfall to ilikeme

MVM

to ilikeme
said by ilikeme:

Sounds like someone works for the company that makes this.

It doesn't matter if he works for the company or not, what he says is true. There are many questions that pop up in the network section of this forum related to aggregating the bandwidth of two or more broadband connections. To see a device that can do this is nice, but I wouldn't do it. I like the idea of load balancing and failover more than joining the connections together through a cloud of some kind.

TAZ
@qwest.net

TAZ to ilikeme

Anon

to ilikeme
said by ilikeme:

Sounds like someone works for the company that makes this.

For the record, I do not.
TAZ

TAZ to exocet_cm

Anon

to exocet_cm
said by exocet_cm:

This product claims to do aggregation, which is what you said. But even then, how many have the money to pay for multiple ISPs?

IMO, businesses. All their marketing is talking about Netflix and other consumer applications, but you are correct in that it will never take off in that market. In that market, I see far more promise in service provider bonding offerings (like what CenturyLink and AT&T are doing now).
rradina
join:2000-08-08
Chesterfield, MO

1 recommendation

rradina to ilikeme

Member

to ilikeme
No -- it sounds like someone who understands how it will work instead of speculating that it's the same as a multi-WAN router.

TAZ
@qwest.net

TAZ to Mellow

Anon

to Mellow
said by Mellow:

But now you have 1 central point of failure added to the mix.

Not only that, but their use of MPTCP isn't ideal. It's not going to handle packet loss very well. They are imposing ordering and reliability on the entire aggregated stream which will only add overhead, on top of the overhead from using TCP for this in the first place. (Remember, at L3 there are no guarantees about packet ordering or reliability; higher layers are expected to handle that.)
rradina
join:2000-08-08
Chesterfield, MO

rradina to Mellow

Member

to Mellow
That assumes it's not smart enough to fall back to a simple but stateful round-robin load-balance scheme or just use one link when the central site is unavaialble.

(Since the availability of the central site will never be perfect (shit happens), it seems unlikely the device's engineers turn it into a brick during those times.)

Justin Colle
@tssg.org

Justin Colle to TAZ

Anon

to TAZ
Ok, to be clear, I do work for Multipath

Dropped packets do a fast retransmit on the link with the lowest latency. In general, the system will favour the link with the lowest latency, and back off when it see's packet loss on any particular link.

This is exactly the same as regular TCP.

The overhead of doing MPTCP is between 5 and 15%. So, if you have 2 x 10Mb links, you will get an aggregated speed of between 17 and 19Mb/s.

These numbers are field test results, not lab tests and are true when bonding any combination of DSL, Cable, Wireless, 3G or 4G.

Thanks!
Justin

TAZ
@qwest.net

TAZ

Anon

said by Justin Colle :

Dropped packets do a fast retransmit on the link with the lowest latency. In general, the system will favour the link with the lowest latency, and back off when it see's packet loss on any particular link.

First, thanks for coming by and getting involved in the discussion!

MPTCP is still enforcing reliability and ordering guarantees on the entire aggregated stream. You're tunneling IP packets over that TCP stream - you don't need to enforce those guarantees, and doing so only adds overhead in a place where it may not be needed.

If there's a slight burst of packet loss, your TCP stream is now completely blocked until the packet loss is detected and those packets are retransmitted. While that's probably only going to be a matter of milliseconds, that blocking is going to negatively impact real-time applications like VoIP that don't care about momentary bursts of packet loss.

It's easy for me to say this, and I understand that MPTCP is an "easy solution" to this that will work well enough and allow you to ship the product quickly, but my advice at least for a future version of this would be to investigate another solution. Don't bother with packet reordering or reliability at all. Fragment the packet, send the fragments, the other end can buffer the fragments until it has received all of them (or disregard the fragments if they haven't all been received within a certain amount of time) and then reassemble it. Let the endpoints handle packet drops and reordering; that's the best way to do it because there isn't a "one size fits all" for every application (of course TCP and some other L4 protocols like SCTP will handle it themselves, but then other applications using UDP will implement those guarantees as appropriate for their specific use).

MovieLover76
join:2009-09-11
Cherry Hill, NJ
(Software) pfSense
Asus RT-AC68
Asus RT-AC66

MovieLover76 to rradina

Member

to rradina
Exactly, while I admit for someone with access to a 50/25 FiOS connection I see no need for this, this is more for those with only a few slow links available to make single transfers and high speed streaming possible.

Imagine the guy that only has access to 3mbps dsl trying to stream netflix, it's going to suck, but team up a couple dsl lines and maybe a 3G connection and you can do it.

Only issue I see here is their should be some cap control added, so that you can monitor how much data you allow over each connection.

Justin Colle
@tssg.org

Justin Colle to TAZ

Anon

to TAZ
said by TAZ :

First, thanks for coming by and getting involved in the discussion!

No problem! Delighted to answer questions
said by TAZ :

You're tunneling IP packets over that TCP stream

We don't use standard tunnels, what we do is closer to a transformation at the client and the server. Thats the secret sauce and what we do over and above MPTCP.
said by TAZ :

If there's a slight burst of packet loss, your TCP stream is now completely blocked until the packet loss is detected and those packets are retransmitted

Like standard TCP, if a single link experiences a burst of packet loss transmission on that link backs off and the packet gets retransmitted on the link with the lowest latency. This appears as a momentary increase in latency, but it would be wrong to describe it as completely blocked. We do not break the end to end nature of TCP which is important in these scenario's.

Our first system used only 3G connections where this scenario can be common, and even in lossy environments you will see efficiencies of 85%+ as very little data gets sent over the lossy link. There is no real difference here between TCP and MPTCP, they both see the lossy link the same way.

Thanks,
Justin

TAZ
@qwest.net

TAZ

Anon

said by Justin Colle :

We don't use standard tunnels, what we do is closer to a transformation at the client and the server. Thats the secret sauce and what we do over and above MPTCP.

My first thought is you're assigning every 5-tuple flow a short (like 4 bytes) identifier and sending that instead of the full IP+L4 (whatever L4 protocol it is) header. I won't ask for any details on that since I do understand it is, as you say, secret/proprietary. But I do want to ask one thing: can other lesser-known L4 protocols be tunneled over this, such as SCTP? Or, what would happen if I sent IP packets specifying an unassigned protocol number?
said by Justin Colle :

Like standard TCP, if a single link experiences a burst of packet loss transmission on that link backs off and the packet gets retransmitted on the link with the lowest latency. This appears as a momentary increase in latency, but it would be wrong to describe it as completely blocked. We do not break the end to end nature of TCP which is important in these scenario's.

How would it not be completely blocked? If you tunnel, say, 3 packets over your MPTCP stream (assuming each one is a different send() call and therefore would end up being fragmented by the MPTCP stack and sent individually) and lose one of those fragments, the entire stream is blocked until it's retransmitted and received. Even if the other packets that have been received, are for different flows entirely (perhaps not even TCP flows) and shouldn't be affected by that packet drop.

Unless you're using a single MPTCP connection for each flow? Although that still suffers from the same problem, just to a much lesser extent (UDP or other unreliable L4 protocols).

Justin Colle
@tssg.org

Justin Colle

Anon

said by TAZ :

can other lesser-known L4 protocols be tunneled over this, such as SCTP?

Yes. Everything we do is completely transparent to both end points, the packet leaves our server exactly as it entered the router, bar being nat'ed.
said by TAZ :

Unless you're using a single MPTCP connection for each flow?

If you have a lossy link you will experience a rise in latency until the congestion window becomes so small that little or no traffic is sent over the lossy link. What you are saying is not exactly wrong, just exaggerated. We've been doing this for 2 years now on all sorts of networks, fixed, cellular, 80211 etc. and TCP basically does it's thing with each network pretty well.

TAZ
@qwest.net

TAZ

Anon

said by Justin Colle :

If you have a lossy link you will experience a rise in latency until the congestion window becomes so small that little or no traffic is sent over the lossy link. What you are saying is not exactly wrong, just exaggerated. We've been doing this for 2 years now on all sorts of networks, fixed, cellular, 80211 etc. and TCP basically does it's thing with each network pretty well.

I've never disagreed with you. I think your current setup will provide good results in the vast majority of situations. It isn't "ideal" in that there are some other conditions that could perhaps be handled better with another approach, but MPTCP is at least an easy, proven method for getting it done quickly and satisfactorily in most situations. As usual (with anything) there are always improvements that can be made; I do hope you'll investigate something like this in a later iteration of the product, as it would provide better results at least when there's intermittent connection issues, but for now, I can agree with the current approach - get it out there and work on the "hard to do, but maybe a little possible benefit" optimizations later!

I will also agree with this: if Joe User wants to bond a DSL, cable and LTE connection with this, they'll probably be pretty happy with the end result. (I'm basing that off the technical details only. Obviously, if, for example, the proxy service had poor uptime, that'd probably change things, but you guys don't sound like idiots and I really doubt that would be the case!)
prairiesky
join:2008-12-08
canada

prairiesky to MovieLover76

Member

to MovieLover76
exactly!

I would suggest balancing be weighted on speeds up and down, Caps, and latencies. That algorithm might be difficult, but that would be their selling point. PFsense can do weighting based on connection speeds, but there's no cut off when you hit a cap.

Imagine having a DSL with unlimited and a cable connection that's capped. If the device was smart enough it would send all the low priority / low size stuff through the unlimited connection, saving the faster cable connection for larger downloads that you want in a hurry. As the cap on the cable connection is reached, it minimized data through that connection until it cuts it off just before the cap.

That being said, there would have to be other factors, like when the caps start and stop in a given month.
rradina
join:2000-08-08
Chesterfield, MO

rradina to MovieLover76

Member

to MovieLover76
What's really interesting is that the central aggregation service can make a profit for $5/month. Out of that paltry sum, they have to pay for servers, routers, admins, developers, billing and in + out bandwidth. I would guess their bandwidth needs will be huge. Unlike a typical high-volume site (i.e. YouTube), the aggregate service is in the middle. Every inbound byte will send an outbound byte. That's a pretty strong indicator of how cheap data really is.

MxxCon
join:1999-11-19
Brooklyn, NY

MxxCon to TAZ

Member

to TAZ
I guess it's faster to do packet-loss recovery router-proxy rather than router-destination. that's why they reassemble packets on their proxy.

TAZ
@qwest.net

TAZ

Anon

Not sure what you mean? The proxy service reassembles packets because that's the only way you can bond multiple residential connections from different providers and receive the aggregate throughput of all of them.

Justin Colle
@tssg.org

Justin Colle to TAZ

Anon

to TAZ
Agreed on all counts!

batterup
I Can Not Tell A Lie.
Premium Member
join:2003-02-06
Netcong, NJ

batterup to ilikeme

Premium Member

to ilikeme
said by ilikeme:

Sounds like someone works for the company that makes this.

He may not be the only one who has a connection to a provider posting on this forum. The doers are an asset, the hangers on, like MCI was, are the danger as they promise everything and deliver nothing.

The problem as I see it combining four 3 meg DSLs will get one a 12/3.2 loop for about $340 a month. I would never put a 4G in the mix because of cost; that loop would most likely get most of the traffic too as it would be by far the fastest.

Eagles1221
join:2009-04-29
Vincentown, NJ

Eagles1221 to TAZ

Member

to TAZ
I run an business and we use load balanced DSL, Cable and fiber. True we can't aggregate them but with hundreds of users it works well to spread the load of 7.1mbit DLS, 50 mbit cable and 100 mbit fiber.