TSI Marc Premium Member join:2006-06-23 Chatham, ON
1 recommendation |
TSI Marc
Premium Member
2013-Jan-31 3:00 pm
Changes in Bell AGAS networkHey all,
This is a bit of technical stuff for those who care to know.
Next week we will start migrating from 2 tunnel end points to 1 tunnel end point on our AGAS setup with Bell.
We've hit a hard limit on their network where we cant add any more capacity unless we migrate...
We dont believe this will cause any down time, as you reconnect you'll simply end up one a tunnel end point same as before except that there will be fewer of them.. which will allow us to keep adding more.
We also believe that it will help better even out the load on each link but I guess we'll have to wait and see to know that for sure. |
|
HiVolt Premium Member join:2000-12-28 Toronto, ON |
HiVolt
Premium Member
2013-Jan-31 3:03 pm
I hope this will help the evening speeds... Its been terrible at times especially weekends.
Any progress on Bell offering 10gig links? You'll never be able to offer 50/10 reliably when that is available to you. |
|
|
said by HiVolt:Any progress on Bell offering 10gig links? You'll never be able to offer 50/10 reliably when that is available to you. They technically could do it on 1GbE. It just wouldn't be financially viable due to the tariffs' structure and how CBB is billed on a per-link basis rather than aggregate peak. |
|
|
GuspazGuspaz MVM join:2001-11-05 Montreal, QC |
to TSI Marc
Yeah, I was going to ask about the 10 gig ones.
How is the MLPPP stuff set up these days? Is that still on separate hardware/links? |
|
HiVolt Premium Member join:2000-12-28 Toronto, ON |
to InvalidError
said by InvalidError:said by HiVolt:Any progress on Bell offering 10gig links? You'll never be able to offer 50/10 reliably when that is available to you. They technically could do it on 1GbE. It just wouldn't be financially viable due to the tariffs' structure and how CBB is billed on a per-link basis rather than aggregate peak. I wasn't talking about that... I was talking more about congestion on the links since there is no load balancing between them, resulting in some being overused, and others under-used. This has always been an issue, and the more 1gig links TSI offers and the faster speeds that are offered, the bigger this problem becomes. |
|
TSI Marc Premium Member join:2006-06-23 Chatham, ON |
to HiVolt
said by HiVolt:I hope this will help the evening speeds... Its been terrible at times especially weekends.
Any progress on Bell offering 10gig links? You'll never be able to offer 50/10 reliably when that is available to you. We just added two more links last week, that's a total of 8 gig in the last 3 months that we've added.. we'll continue monitoring it but whatever is going on now is likely more due to improper balance between the links still as they continue to load balance.. going to one link should also help this. |
|
|
to TSI Marc
Thank you for the update Marc.
I do hope that this will help with the congestion issues (still seeing an average of 1% loss per line. 2 lines = about 2% average packet loss). I too would be interested to know why Bell is not moving you guys to 10G links given the issues and your size? |
|
MaynardKrebsWe did it. We heaved Steve. Yipee. Premium Member join:2009-06-17 |
^^^ $$$$ |
|
|
to TSI Marc
said by TSI Marc:said by HiVolt:I hope this will help the evening speeds... Its been terrible at times especially weekends.
Any progress on Bell offering 10gig links? You'll never be able to offer 50/10 reliably when that is available to you. We just added two more links last week, that's a total of 8 gig in the last 3 months that we've added.. we'll continue monitoring it but whatever is going on now is likely more due to improper balance between the links still as they continue to load balance.. going to one link should also help this. Won't Bell let you LACP those links in some sort of way? I mean, if you're paying for each individually, I don't see how they could justify not doing it from that perspective. |
|
|
to MaynardKrebs
Sure but at some point the cost of maintaining that many links comes into play too. 10G isn't as costly as it once was either. |
|
|
to HiVolt
said by HiVolt:I wasn't talking about that... I was talking more about congestion on the links since there is no load balancing between them, resulting in some being overused, and others under-used. This has always been an issue, and the more 1gig links TSI offers and the faster speeds that are offered, the bigger this problem becomes. If capacity billing was on the aggregated peak rather than individual links, getting enough links to reduce the likelihood of maxing any of them out would be much more affordable. What I was saying is that the way the tariffs and CBB work make a bad situation worse than it should be. |
|
TSI Marc Premium Member join:2006-06-23 Chatham, ON |
TSI Marc
Premium Member
2013-Feb-1 6:36 am
So, after poking a bit more, last night I was informed that there are 4 of the links that are actually congested on the Bell end of things and they need to be re-provisioned to free up the full capacity. This does help to explain the discrepancy between the feedback we're getting from you guys and what we're seeing on the network on our side.
More to follow once I have more details. |
|
GuspazGuspaz MVM join:2001-11-05 Montreal, QC |
to TSI Marc
Good to hear. YouTube 1080p was unwatchable last night. I had a friend over and we was trying to show him a few YouTube videos, and had to constantly appologize for the stream freezing to buffer. |
|
|
to TSI Marc
Ah that would explain some things. Will this be part of the switch to one endpoint next week or is this separate (and if separate is there an ETA for it)? |
|
TP-Link Archer C7 Technicolor DCM476 Grandstream HT701
|
to Guspaz
said by Guspaz:Good to hear. YouTube 1080p was unwatchable last night. I had a friend over and we was trying to show him a few YouTube videos, and had to constantly appologize for the stream freezing to buffer. Same issue here this morning. Reminds me of the old days with 3web. |
|
MrMazda86 (banned) join:2013-01-29 Kitchener, ON |
to TSI Marc
Does this outage affect Kitchener/Waterloo (namely, the old Sherwood exchange) by chance? I've been noticing some interesting trends over the last few days that I haven't seemed to notice before. |
|
|
to TSI Marc
Thank you for the update, Marc. I'm glad to hear that TSI is aware of the problem and working to fix it.
I like TSI, so I always give you benefit of the doubt when there's a problem and I just wait it out, figuring that someone must be working to fix it.
But my connection getting extremely slow every day during peak time for months is starting to wear on me. Like many people, the internet is such a big part of my life that it's hard to shrug this off...
Hopefully this will be fixed for good soon.
Cheers. |
|
JenSuisUn Premium Member join:2006-02-23 Chatham, ON |
said by Liberty25:But my connection getting extremely slow every day during peak time for months is starting to wear on me. Like many people, the internet is such a big part of my life that it's hard to shrug this off... Could we ask that you post in the » /forum ··· avdirect forum so we can look at what's going for you. Thanks, Martin |
|
|
Liberty,
Bring it back to the public forum if you get somewhere.
I'm massively affected too - to the point of unusability. North Bay, ON area.
Also, if you get chance - do you have neighbours on, say, Bell?
How are their connections when your connection is bad?
Cheers Dave |
|
|
to InvalidError
said by InvalidError: If capacity billing was on the aggregated peak rather than individual links, getting enough links to reduce the likelihood of maxing any of them out would be much more affordable.
What I was saying is that the way the tariffs and CBB work make a bad situation worse than it should be. 10G AGAS and/or LAG would help significantly. |
|
|
said by HeadSpinning:10G AGAS and/or LAG would help significantly. LAGs would indeed help, as long as multiple links are connected to the same chassis at both ends. If links are spread across 3+ routers at either end, it becomes that much less effective since many may end up being orphans. Makes you wish Nortel's SMLT became standard. It would make this sort of situation so much simpler. |
|
|
to JenSuisUn
Hi TSI_Martin,
Since I seem to be having the same problem as many people (I'm on DSL near ottawa; things slowing down during peak time, then back to fine off peak; especially noticeable with video sites like youtube, vimeo, streaming gaming sites like twitch, gomtv, tou.tv, news sites, etc.. they all pretty much become unusable unless you let it buffer forever, etc) and it seems to be a known issue, is it really useful that I spend the time filing one more individual ticket?
I'm honestly asking. I know in theory it could be something else, but it seems really obvious to me that this is congestion on the network and that this needs to be fixed on your side.
But if you think it's worth it and it can help, I'll take the time and create a post in the TSI direct forum.
Thanks. |
|
|
to InvalidError
said by InvalidError:said by HeadSpinning:10G AGAS and/or LAG would help significantly. LAGs would indeed help, as long as multiple links are connected to the same chassis at both ends. If links are spread across 3+ routers at either end, it becomes that much less effective since many may end up being orphans. Makes you wish Nortel's SMLT became standard. It would make this sort of situation so much simpler. We use MC-LAG between Extreme and Foundry gear, and it works just fine. » en.wikipedia.org/wiki/MC-LAG |
|
|
said by HeadSpinning:We use MC-LAG between Extreme and Foundry gear, and it works just fine. I had no doubt companies had their own proprietary implementations of SMLT-like features. I was just saying it was a shame that there still isn't a standard for SMLT/MC-LAG more than a decade after the first proprietary implementations on Ethernet gear. Then again, I suppose a standard is not absolutely required since router/switch manufacturers can achieve almost the same result by bending the LACP/LAG rules a little to allow proxying. |
|
|
to TSI Marc
At this rate, why even bother with 10G...
Telus and Bell are already implementing 40G and 100G backbones.
TSI should go for gold and think longer than 3 months down the road!
Demand 100G aggregation links! |
|
|
+1
Funny but still +1 |
|
TSI Marc Premium Member join:2006-06-23 Chatham, ON |
TSI Marc
Premium Member
2013-Feb-5 11:14 am
ok so yesterday the config changes were made.
Things will start to balance out.
We also added another gig link. So that's a total of 30 gig links.
By looking at the traffic across all of them, we should have plenty of capacity. Now it's a matter of balancing the load evenly. Some links I can see last night were overloaded but most were not.. i.e. the balance is the problem... we're going to see what we can do to force people off the congested ones over to the lower used ones..
we are also waiting on confirmation from Bell on a few links that may also have issues on their end. We're not sure if it's an actual problem or if they are simply being proactive. |
|
|
to openvz_ca
said by openvz_ca:Demand 100G aggregation links! While 100G may be available, it is not cost-effective for small clients. TSI has only ~30Gbps of peak traffic on Bell's side so if TSI put everything on a single 40G/100G link, all of TSI's Bell-GAS would go down on a single-point failure. So 40G will not make much sense until TSI gets over 80Gbps, which may take a while at the rate this seems to be growing since so many people have switched to TPIA. Another problem with 40G/100G is that most routers have only 160Gbps of backplane bandwidth per slot so using 40/100Gbps for small clients who use only a fraction of it wastes a lot of precious rack and chassis space: with 4x10G links on a router populated with 16x10G line cards, TSI would use 1/4th of a single card while with 2x40G (minimum for redundancy), it would use between half to a full slot depending on whether the line card has 2x or 4xQFP slots. |
|
GuspazGuspaz MVM join:2001-11-05 Montreal, QC |
to TSI Marc
Right, so when is Bell going to start activating 10 gig links as the crtc has mandated they do? |
|
TSI Marc Premium Member join:2006-06-23 Chatham, ON |
TSI Marc
Premium Member
2013-Feb-5 11:44 am
I'm not aware that there is any such mandate.
we keep knocking on that door but nobody's ever let us in. |
|