Here are some thoughts on how we have "migrated" to a more reliable network.
The difference is in the following in no particular order of importance: redundancy, provider, wireless equipment, policy, tech support, network management and reporting.
I'm going to leave out standard business stuff and focus on the above categories.
REDUNDANCY:
Back in the day we had one of everything and if it failed we had to drive/climb to the point of failure and fix it or lose customers.
Now we have almost two of everything. It's not as expensive as it sounds and takes a huge load off of us in terms of response times. It also appears to the customer that no downtime ever happens.
We do this by having two boarder routers configured in tandem (ours is VRRP, your may be hot fail, or BGPed). We use Mikrotik on pentium servers so the costs is hundreds, not thousands of dollars.
Our backhauls are configured in a wireless ring in case one fails (we use RSTP in bridged networks, OSPF in routed). The third or more leg in the ring can just be a low cost low bandwidth link if necessary as long as it can keep the customers going while a repair is done.
Our sectors are doubled/overlapped so that CPE's can see more than just one AP. Capacity is kept under maximum so if one fails the other AP can temporarily handle the load. Having overlap and keeping sector count low does use a bit more frequency, but generally allow for greater speeds per customer.
Network layout in general has a dual/split path for every client from CPE to the internet is the idea. So if you have two AP's facing a section for redundancy plug each one into a different router/switch in case of router/switch/cable failure.
IP space directly from ARIN split among two or more providers assures that your public IPs will not be unreachable in an outage of one provider. This is usually done with a BGP protocol at the ISP. Or simple hot failover to another provider creates better uptime.
PROVIDER:
This one is critical as it's what you are selling in the end. In the beginning we had a DSL line with a few public IPs. Now we have access to a 2.5Gbps fiber line and upgrades to a BGP address space.
Your provider has to be able to handle the capacity it is selling you AND have the same features you are selling your customers. If they can't uphold their end of the bargain then you won't be able to either for your customers.
Upgrade your provider sooner rather than later and get the best you can afford.
WIRELESS EQUIPMENT:
We started with 802.11b and have had to migrate to Canopy for various reasons. Take careful stock of your service area and use the right equipment for the topology and environment from the beginning. Whether that is 802.11, mesh or proprietary stuff.
This is a hard one to figure out, but lots of reading, planning and some testing before deployment will give you a head start. Even then a curveball will probably come your way in the form of a manufacturer suddenly leaving their platform in the dust or spitting out crappy PCB's etc.
If possible use a blend of two topologies and manufacturers. Then if something really goes wrong you still have something to work with instead of starting completely over.
Make sure your chosen distributor and manufacturer has a good track history, company record and RMA policy.
POLICY:
No matter what we do, you can't please everyone all the time. But setting the proper expectations up front will help down the road. Your customer's perceptions are set by the industry, marketing (yours and competitors) and whether they woke up on the right side of the bed etc.
Get a written policy SLA/Terms of Service together to define WHAT it is you are selling and NOT selling. And then treat that like your company bible with few exceptions.
Eventually the disgruntled customers will come to respect it and your company.
Some examples are pricing, speeds, installation procedures, equipment ownership, TECH SUPPORT, disconnect policies etc. etc.
Make sure your policy is reflected in your web site, written documentation, sales call, tech support visits, the whole enchilada. Consistency is key.
TECH SUPPORT:
This deserves a whole separate thread, but the important point is to commit to it. Even if your policy says the customer can take a flying leap off a tall cliff, go the extra mile and earn their appreciation.
A few minutes more on the phone or an on site visit will usually be greatly appreciated. Even if the customer wants you to come re-configure their router after hours for no extra charge (why DOES everyone expect this for $24.95 a month?), bite your tongue until it bleeds and politely schedule a time and price according to your policy (you have that in your policy, RIGHT?).
The bottom line with tech support is to give a little more than expected or per policy to create the proper perception. Tell them your hourly rate for non customers is $150 an hour, but for them it's $55 or $90 if they want emergency service (as per your rate policy).
If someone returns the phone calls or answers the phone within reasonable times (again, whatever expectation has been set) then they will feel good about it.
Good tech support leads to referrals in our experience.
NETWORK MANAGEMENT MONITORING AND REPORTING:
In the beginning we didn't even have an idea of how much bandwidth was being used and no real time or historical charting. What can you tell your customer without having this information? "Uh, you're down again? Since when?" is not a good response.
Even most of the inexpensive CPE/AP/Router/Switches today have SNMP or can be pinged using some monitoring program like MRTG.
The customer will actually listen and respect your expertise if you can tell them more information than they already know. "Your radio has been up for 21 days, 2 hours and 32 seconds and is running at 100 percent" stuns them to silence (sometimes).
Monitoring is the dividing line between what's your responsibility and theirs. What's free support to fix your end or paid support to fix theirs.
Get equipment and software that allows you to control your network. Nothing like watching your network get crushed by that one user running 1000 simultaneous connections on their Azereus BitTorrent client. Or that collection of customer Netgear routers that suddenly decide to ICMP blast the entire broadcast domain for days on end.
Usually this means proper traffic shaping and routers/managed switches at key points in your network. I can't speak enough of how Mikrotik shaping has improved our network capacity and reliability (or any similar product).
Traffic shaping, managed switches and proper use of routing/vlan techniques keep the nastyness of virus/icmp/arp storms at bay. P2P is allowed on our network and keeps the power users happy along with the business and casual residential users.
Live packet reporting like Wireshark and Mikrotik Torch allow complete investigation of what is happening NOW in the network. Historical graphing of SNMP and programs like NTOP/MRTG data gives us the complete picture over time.
SUMMARY:
To sum it up we wouldn't be where we are today without adhering to these principals. I hope the information can help someone else improve their operations!
 feedback form
 feedback form
by IntraLink 
last modified: 2007-07-20 21:26:39 |
