republican-creole
site Search:


 
   
story category
A Look at Verizon's Decimated NYC Network
Miles of Damaged Copper Getting Ripped Out Beneath City
by Karl Bode Monday 19-Nov-2012 tags: dsl · business · hardware · bandwidth · telco · networking · Verizon · Verizon FiOS · Verizon Online DSL
The Verge offers up a stunning look with photos at some of the damage caused to Verizon's New York City infrastructure by Sandy, highlighting the miles of copper made useless by the city's flooded underground. The Verge's Dante D'Orazio got a personal tour of Verizon's underground cabling two weeks after the storm, getting to follow Verizon repair crews as the company sought out damaged copper lines.

While the disaster will be a costly one for Verizon, it has the benefit of forcing the company to more quickly replace aging copper infrastructure with fiber optic lines, which obviously has the added benefit of lower maintenance costs (and better future storm resilience):

Click for full size
Miles of copper is ruined not only in the cable vault at Broad Street, but also at 20 or so manholes around the area. Even worse, paper insulation in the copper wiring sucks water through the cabling from capillary action, destroying cabling even in dry areas. Levendos says it’s "far too tedious, time consuming, and not effective of a process to try and put this infrastructure back together," so Verizon’s taking the opportunity to rewire with fiber optics instead. Service has been restored to FiOS customers for over a week — unlike copper, fiber optics aren’t damaged by the water. As part of this process, crews have already pulled fiber up the major corridors — including Water, Broad, and Pearl Streets — to ultimately connect the fiber network to buildings.

The photo to the right shows the copper feeders just chopped off and capped, raising questions on how this impacts any CLECs that may have been using that infrastructure for access. The timing of Sandy comes as Verizon was already pushing to have a lot of this copper replaced -- and this obviously gives them a large-scale opportunity to accomplish this belatedly in New York City -- with insurance help. Verizon won't give specific numbers, but notes that thousands of DSL customers still lack service in lower Manhattan.

view: topics flat text 
Post a:

cdru
Go Colts
Premium,MVM
join:2003-05-14
Fort Wayne, IN
kudos:7

Conspiracy?

quote:
The photo to the right shows the copper feeders just chopped off and capped, raising questions on how this impacts any CLECs that may have been using that infrastructure for access.
So Sandy was a conspiracy by Verizon to run the CLECs out of business.

Anyone seen my tinfoil hat?

Alex J

@sunwave.com.br

Re: Conspiracy?

It doesn't claim conspiracy. It simply asks what happens to the CLECs who shared that infrastructure, which is a legitimate question. Do you have the answer?
elefante72

join:2010-12-03
East Amherst, NY
Reviews:
·Time Warner Cable
·Verizon FiOS
·voip.ms

Re: Conspiracy?

Equipment maybe. These are access trunks. It doesn't change out the POP and VZ still has contracts with CLECs. The DSL DSLAMs are still owned by Verizon, and they will need to replace if they are damaged.

I don't know what VZ runs in NYC (probably ATM), so they just upgrade the line cards to optical.

What happened to Earthlink when Time Warner moved to HFC? Nothing, except better speeds and quality.
CXM_Splicer
Looking at the bigger picture
Premium
join:2011-08-11
NYC
kudos:1
Reviews:
·Verizon FiOS

Re: Conspiracy?

quote:
Equipment maybe. These are access trunks. It doesn't change out the POP and VZ still has contracts with CLECs. The DSL DSLAMs are still owned by Verizon, and they will need to replace if they are damaged.

The DSLAMS are not being repaired in NY. The Verizon customers are being migrated to FIOS instead.

Alex J

@speakeasy.net

Re: Conspiracy?

Thanks for the info. So, yes, there are real questions to be asked here about what happens next for CLECs. Readers yelling that asking a question about this is "conspiracy" apparently need hobbies.

bhan261

join:2001-02-12
New York, NY
Just curious...what if a building landlord refuses to allow Verizon to install FiOS in their building? Do the tenants have any recourse?
CXM_Splicer
Looking at the bigger picture
Premium
join:2011-08-11
NYC
kudos:1

1 edit

Re: Conspiracy?

Not really, no. It is common for owners of large buildings to be approached by Time Warner who gets an exclusivity contract in exchange for $$$. Unfortunately, you are at the mercy of the building owner.

battleop

join:2005-09-28
00000
No. Landlords can pretty much say who can and can't enter their buildings. I

TheTechGuru

join:2004-03-25
TEXAS
Reviews:
·WesTex Connect
said by bhan261:

Just curious...what if a building landlord refuses to allow Verizon to install FiOS in their building? Do the tenants have any recourse?

What Verizon needs to do is what AT&T does in my area. AT&T runs fiber out of their switch to area "hubs/nodes" which then run POTS copper for the "last mile". They run POTS, ADSL, and VDSL on these.

This would save Verizon a lot of time as they would not have to replace all the good copper just before and inside buildings.
--
CompTIA Network+ Certified

djrobx

join:2000-05-31
Valencia, CA
kudos:2
Reviews:
·VOIPo
·Verizon Wireless..
·RoadRunner Cable
·AT&T U-Verse
said by elefante72:

Equipment maybe. These are access trunks. It doesn't change out the POP and VZ still has contracts with CLECs. The DSL DSLAMs are still owned by Verizon, and they will need to replace if they are damaged.

Verizon doesn't own CLEC DSLAMs. Earthlink is a good example - Earthlink resells both CLEC DSL via Covad, and ILEC DSL using Verizon's DSLAMs. Verizon can figure out some sort of fiber fed solution for customers on their own DSLAMs, but what will happen to Covad's lines?
--
AT&T U-Hearse - RIP Unlimited Internet 1995-2011
Rethink Billable.

battleop

join:2005-09-28
00000

Re: Conspiracy?

"Verizon doesn't own CLEC DSLAMs" no but they own the last mile copper that runs from the CO to the Customer. It's this copper that's being cut that's in question.
--
I do not, have not, and will not work for AT&T/Comcast/Verizon/Charter or similar sized company.
TBBroadband

join:2012-10-26
Fremont, OH
Reviews:
·AT&T U-Verse
·MegaPath
Actually its MegaPath right now and to answer the question on what they are doing. Karl should contact them. My company represents MegaPath and I will contact the PR department and find out. Currently we're being told they're repairing and bringing on lines as power comes back on in commercial buildings. Nothing has been said about copper being cut.
elefante72

join:2010-12-03
East Amherst, NY
Reviews:
·Time Warner Cable
·Verizon FiOS
·voip.ms
I'm assuming that the CLECs get fibre line cards if fibre routes to the DSLAM. Or if it's a ring, maybe nothing. Good points though...

I was also reading that on the VZ owned side they are just pulling fiber to the prem and converting to FIOS.

That can be a problem if the existing customer is contract DSL, but OMG I'm sure if I were getting FIOS, I'd be happy. Of course this comes at a premium price, but that part of town definately is not the low rent district so I'm sure uptake rate will be acceptable.

Nobody has mentioned how cable (TWC) is operating in this situation.
Any news? I don't know how much HFC they have in lower Manhattan.

cdru
Go Colts
Premium,MVM
join:2003-05-14
Fort Wayne, IN
kudos:7
said by Alex J :

It doesn't claim conspiracy. It simply asks what happens to the CLECs who shared that infrastructure, which is a legitimate question. Do you have the answer?

It was a joke, hence the fine print comment about my tin foil hat. That is, unless you really think Verizon has that much clout that it can conjurer up a hurricane.

Alex J

@secureserver.net

Re: Conspiracy?

In other words, you don't know.

Suggesting someone's claiming conspiracy by simply asking what happens now to CLECs doesn't seem like a joke, it seems like you were just being a jerk. Nobody thinks Verizon conjured a hurricane. But they do have an undisputed track record of crushing CLECs, so I'm curious what happens to companies riding that copper now? Are they SOL?

battleop

join:2005-09-28
00000

Re: Conspiracy?

Let me fix that for you...

So Sandy was a conspiracy golden opportunity by Verizon to run the CLECs out of business.

I don't think it's a conspiracy or some pre-planned plot you can bet that Verizon is not going to pass up this golden opportunity to take a whack at CLECs.
--
I do not, have not, and will not work for AT&T/Comcast/Verizon/Charter or similar sized company.

mikedz4

join:2003-04-14
Weirton, WV

Re: Conspiracy?

How could verizon make a hurricane? Last I checked we had NO control over the weather. Unless you ask the environmentalists that is.

battleop

join:2005-09-28
00000

Re: Conspiracy?

Comprehension FAIL! Nowhere did I say it was a conspiracy.

mikedz4

join:2003-04-14
Weirton, WV

Re: Conspiracy?

I didn't mean to reply to you. Sorry I meant to reply to the OP

PaulHikeS2

join:2003-03-06
Manchester, NH
said by mikedz4:

How could verizon make a hurricane? Last I checked we had NO control over the weather. Unless you ask the environmentalists that is.

You didn't know that telcos can control the weather?
--
Jay: What the @#$% is the internet???

pnh102
Reptiles Are Cuddly And Pretty
Premium
join:2002-05-02
Mount Airy, MD

Ugh The Whining

quote:
The photo to the right shows the copper feeders just chopped off and capped, raising questions on how this impacts any CLECs that may have been using that infrastructure for access.
People whine and complain about Verizon NOT replacing copper with something better... and when they do... they complain about these lamer CLECs who never built out their own network in the first place?

Sucks to be them. Shrug.
--
USA 2012 - the mooches won.

Alex J

@sunwave.com.br

Re: Ugh The Whining

Who's whining? I'd like to know what happens to those CLECs? Verizon doesn't share access to FiOS facilities as far as I know. What happens to the existing business relationships with CLECs?

DC DSL
There's a reason I'm Command.
Premium
join:2000-07-30
Washington, DC
kudos:2
Reviews:
·Covad Communicat..
·Verizon Online DSL

Re: Ugh The Whining

said by Alex J :

Who's whining? I'd like to know what happens to those CLECs? Verizon doesn't share access to FiOS facilities as far as I know. What happens to the existing business relationships with CLECs?

I wouldn't be surprised if the contracts have some clause that lets VZ say "too bad" in case of plant destruction or other "conditions beyond [their] control."
--
"Dance like the photo isn't being tagged; love like you've never been unfriended; and tweet like nobody is following."

davoice

join:2000-08-12
Saxapahaw, NC
Verizon does offer wholesale access to FiOS on an unregulated "competitive basis". I have clients on connections using it. Problem is Verizon charges a butt load to the CLEC for the cross connect (i.e. more than a residential customer pays for their whole FiOS bill!). There's also a new group inside Verizon that's working just on FiOS wholesale. So hopefully things will get better quickly as CLECs in NY want to sell it (or have to sell it to keep their customer).

anonanon

@cloud9.net

Re: Ugh The Whining

said by davoice:

Verizon does offer wholesale access to FiOS on an unregulated "competitive basis". I have clients on connections using it.

This was discontinued around 2007 and isn't coming back. There are rumors of Verizon Business offering enterprise metro Ethernet over the FIOS plant (not using FIOS electronics, just the cables) but no concrete plans. And that is just for Ethernet, not T1, DSL, or anything like what CLECs generally do with copper now.
TBBroadband

join:2012-10-26
Fremont, OH

Re: Ugh The Whining

The Ethernet Wholesale was added to the Wholesale Group at the end of last week. But it's not ready to go yet.

And that is true, CLECs are not given access to FiOS nor are other ISPs.
majortom1029

join:2006-10-19
Lindenhurst, NY
kudos:1
said by pnh102:

quote:
The photo to the right shows the copper feeders just chopped off and capped, raising questions on how this impacts any CLECs that may have been using that infrastructure for access.
People whine and complain about Verizon NOT replacing copper with something better... and when they do... they complain about these lamer CLECs who never built out their own network in the first place?

Sucks to be them. Shrug.

You do realize that version used federal money to build the network in the first place right? So I bet replacing the copper lines with fiber would still have the same strings attached. Since they would be replacing the federal governments lines.

Alex J

@secureserver.net

Re: Ugh The Whining

So I bet replacing the copper lines with fiber would still have the same strings attached. Since they would be replacing the federal governments lines.

Verizon has spent years gutting any sharing regulations, so are you sure about this?

NormanS
Premium,MVM
join:2001-02-14
San Jose, CA
kudos:9
said by majortom1029:

You do realize that version used federal money to build the network in the first place right?

As in money transferred from the Treasury? I don't think so. I am aware that the ILECs got significant tax breaks; but until the money is in Federal hands, it isn't "Federal Money".
--
Norman
~Oh Lord, why have you come
~To Konnyu, with the Lion and the Drum
TBBroadband

join:2012-10-26
Fremont, OH
Reviews:
·AT&T U-Verse
·MegaPath
Not totally true. As I pointed out before, AT&T under Ed in the Ameritech region did just that in some areas under Project Lighspeed to test FTTH. The copper is GONE and no CLECs nor other ISPs are allowed on it. You get FTTH U-Verse at U-Verse VDSL speeds and that's it. No other options available unless you want to pay a CLEC to build out a T1 for you, which is not really an option with cost. Monroe County Michigan is one area that has FTTH that was gutted copper, and Wisconsin has a couple as well. The pilot wasn't expanded very far.

battleop

join:2005-09-28
00000
You mean CLECs that don't have super deep pockets to fight the ILECs that continually try and crush them via the FCC and their deep pocket campaign contributions?
--
I do not, have not, and will not work for AT&T/Comcast/Verizon/Charter or similar sized company.
elefante72

join:2010-12-03
East Amherst, NY
Reviews:
·Time Warner Cable
·Verizon FiOS
·voip.ms

baloney

Maybe they want to reduce CLEC's but keep in mind that converting to fiber is the smart thing. Copper had its day. The storm only hastened the movement, there is no conspiracy.

Telcos will offer CLECs if it's good for their bottom line. Look at MVNO in cell co's. The market is robust and growing like gangbusters. FIOS has been marketed as premium, so once saturation hits Verizon will need to go after the mid-market and the way to do that is to offlabel or offload to "CLECs". Since they are bulk reselling (making a tidy profit), they don't have to incur the high SGA costs with the brand label, and at the same time utilize the network (the more it's used the more profitable).

I'm sick of the whiny nature of every post. It's like the entire mission of companies is to screw people constantly. Companies don't exist without customers. Now one has to look to government picking winners or losers..a la our current and past administrations..and that is the problem not the open competitive market.

See 11 replies to this post
unixwolf

join:2007-05-04
Flower Mound, TX

pics-

I would love to see more pics...

mackey

join:2007-08-20
kudos:3

Re: pics-

Did you click the link? First 2 words of the article. A bunch of pics plus a video...

/M

IowaCowboy
Want to go back to Iowa
Premium
join:2010-10-16
Springfield, MA

They might as well upgrade

They might as well upgrade their destroyed networks. Compare that to a transit agency (PVTA, MBTA, NYC Transit) fixing an aging fleet of buses (that are breaking down more and more often) to buying brand new, more fuel efficient vehicles.

OSUGoose

join:2007-12-27
Columbus, OH
Reviews:
·Insight Communic..

Hey

What I don't get is why are the capping the copper stubs. I would think they would of drug out the cable and slid fiber back in the same hole. Plus gotten some scrap value in the copper to help deal with helping CLECs get back up, since this is clearly outside VZs control what happened. I'd like to know why they are leaving that copper behind vs scrapping it.
rradina

join:2000-08-08
Chesterfield, MO

Re: Hey

If it's profitable to extract it, they will. If not, it makes sense to leave it. I know that seems messy and wasteful but customers certainly don't want to pay to extract it unless it's necessary or profitable.

OSUGoose

join:2007-12-27
Columbus, OH
Reviews:
·Insight Communic..

Re: Hey

Well the assumption on here is that whatever costs they are incurring is being covered by insurance or some government assistance. I would just think it would make sense that once whatever facilities that cable pair was switched over to, that copper then gets yanked out to be scrapped, freeing up more space that could then let the CLECs run their own fiber, since I get the impression the FCC might make VZ either install fiber to restore the CLECs or put new copper for the CLECs once enough CLECs start bitchin about VZ abandoning the copper and moving it all to fiber.
rradina

join:2000-08-08
Chesterfield, MO

Re: Hey

Will insurance pay for removing it if it's not required to repair it? I agree if to run fiber, copper must be removed. However, fiber will require a lot less room than copper. That means they won't need to remove all of it. Insurance is only going to fund what's necessary to restore service.

It really depends on how much effort is required to remove it. If it's relatively easy to pull out of the existing conduits (i.e. the insulation has not swelled with moisture and made it impossible to remove), they probably will.

As an example, I've owned two different houses and at each the cable company ran a new drop. They didn't bother to remove the old one because it would have been a PITA, damaged the lawn and most definitely had a negative return.

tshirt
Premium,MVM
join:2004-07-11
Snohomish, WA
kudos:3
Yes they will eventually want the conduit space, but leaving the cable in place prevents it from collapsing /being crushed by other work or "accidentally" being connected to a water/waste line.

tshirt
Premium,MVM
join:2004-07-11
Snohomish, WA
kudos:3
dup post
BeakersBro

join:2011-06-24

paper?

Somewhat surprised that they still had paper insulated copper wiring in service in this area.

Them new-fangled plastic insulation has been around for a while.

And I assume they will eventually pull the bad copper and recycle, but the crews that do fiber don't do copper, so I would think this will occur once the crisis is over.

Linklist
Premium
join:2002-03-03
Longport, NJ
kudos:5

Re: paper?

said by BeakersBro:

Somewhat surprised that they still had paper insulated copper wiring in service in this area.

Some of those trunk cables have probably been there since they wired NYC back in the early 1900's.
--
A democracy cannot exist as a permanent form of government. It can only exist until the voters discover that they can vote themselves money from the public treasury.
silbaco

join:2009-08-03
USA

Old copper

They will probably not pull any old copper to scrap unless it is very easily accessible. As valuable as copper is, it would cost more to have crews pulling it up at union wages.
majortom1029

join:2006-10-19
Lindenhurst, NY
kudos:1

hmm

I would like to point out that just because the copper will be replaced by fiber does not mean they can stop offering the clecs service. If its the Government funded network that's being replaced then they still have the right to use it.

I am on long island and we have telephone service through earthlink business. We have an old very thick Verizon copper line coming into the building and a Verizon fiber line coming in.

The last time we renewed our contract erthlink business moved out t1 from the old copper line to the fiber line coming in. They used a cisco box to emulate the copper service.

I would think Verizon is going to do this. Lay down the fiber to every affected building and use equipment in the basements to connect the copper equipment in the buildings to the newly run fiber.

Will probably take a lot less time to do it that way.
CXM_Splicer
Looking at the bigger picture
Premium
join:2011-08-11
NYC
kudos:1
Reviews:
·Verizon FiOS

Re: hmm

The CLECs will not be moved to FIOS, that was one of the agreements Verizon worked out with the regulators to start rolling it out in the first place. A copper T1 will have to be rebuilt on fiber but wholesale T1's can be put on VZ fiber, no problem. It is just going to take a long time to get it all done.
majortom1029

join:2006-10-19
Lindenhurst, NY
kudos:1

Re: hmm

It has nothing to do with FIOS. Fios is the consumer and small business product.

Verizon has business fiber that in most cases is a direct fiber connection to the CO. Just like what I have at work. I am willing to bet the Clec service that's down in lower manhattan will be replaced with fiber from the building to the Verizon Co and then will be connected to the existing Copper from the offices in the building to the new fiber with something like the cisco box that I have in my building.
ITALIAN926

join:2003-08-16
kudos:1
Reviews:
·Verizon FiOS
How does a storm exempt Verizon from being responsible to provide CLECs access for DSL and Voice? Im not following the logic here. It is THEIR RESPONSIBILITY TO MAINTAIN THE NETWORK and provide access. If they want to replace copper with fiber, then shouldnt clecs have access to the fiber? Where in the 96 Telecom Act does it state that a hurricane exempts RBOCs from allowing CLEC access?
TBBroadband

join:2012-10-26
Fremont, OH
Try telling that to AT&T who has already done just that. No CLECs.
cooldude9919

join:2000-05-29
kudos:5

Mux

Just because they remove the copper between the C.O and buildings, doesnt mean all copper service is gone. We are on an old oc3 fiber ring with a ds3 mux, with t1's broken out of that, so we have around ~15ft of copper before it goes to fiber. Legacy services from anyone (CLEC inculded) could be easily serviced like this if needed.
Capt_video

join:2001-12-28
Grapevine, TX

A reality check

Having worked in that cable vault around 1978, I can verify that a lot of those cables date to the late 1940s. When it still works, you don't replace it. I also know that Verizon has been in the process of replacing a lot of that old infrastructure and moving to fiber for quite a few years. Bottom line always suggests that you do it in phases and it would have taken years to get all that replaced. Now with this disiaster and the use of insurance money, the job will get done a lot faster. As of today, a lot of that traffic has been rerouted away from this area but that still leaves a bunch of folks out of service. There is also a large effort being made to set up and use temporary microwave links to help with the situation
tmc8080

join:2004-04-24
Brooklyn, NY
Reviews:
·ooma
·Optimum Online
·Verizon FiOS

scrap the copper

only to replace it with fiber.. don't worry.. Verizon's already broken so many deals-- don't think the NYS PUC won't have something to say about giving CLECs access to fiber infrastructure once they rebuild with government assistance..

if there was over $5 million for time warner for upstate fiber runs, then verizon's good for at least that much in manhattan. hint.. they might want to run water-tight conduit as they do in underseas cables. then you won't see a saltwater excuse the next time around.

anon562

@mich.net

Fiber can still get ruined

The one thing that I don't necessarily agree with saying fiber is not damageable by water. It most certainly could be if water got inside of the fibers contaminating them with all kinds of stuff. The difference is that it looks like most of their fiber runs are unspliced or have water tight splices. Just looking at mess of old phone wiring with thousands of those little splice connectors shows how many times that wiring has probably changed over the years. It will be interesting to see how well their fiber infrastructure ages and if they would have the same problems with their lines in another 20 or 30 years.
rradina

join:2000-08-08
Chesterfield, MO

Insurance Money

I keep reading posts referencing the fixes (however they do it -- fiber or more copper) are funded by insurance money. Does anyone have the skinny on how Verizon insured this? A lot of big companies self-insure. They might have a "stop loss" policy (i..e anything beyond $1M or $1B), but are we sure this is covered by insurance? If so, can someone point me to information from Verizon so I can educate myself? Thanks.

See 9 replies to this post
bigpapae35

join:2002-10-25
united state
kudos:1

fios

Love fios, but my landlord will not let them in my office building in nyc.

I would like to raise a few bucks from every tenant in the building to convince (bribe) the landloard to let them in.
HeadSpinning
MNSi Internet

join:2005-05-29
Windsor, ON
kudos:5

Re: fios

said by bigpapae35:

Love fios, but my landlord will not let them in my office building in nyc.

I would like to raise a few bucks from every tenant in the building to convince (bribe) the landloard to let them in.

The fact that a landlord of a multi tenant building can refuse a telecommunications undertaking from entering a building is mind boggling.

The fact that they'd WANT to prevent someone from bringing in fiber is also mind boggling - it just makes your building more valuable and rentable.

In Canada our Telecommunications Act specifically allows that an owner of a multi tenant building must allow non-discriminatory access to telecommunications facilities of any carrier authorized to operate under CRTC regulation.
--
MNSi Internet - »www.mnsi.net

hjdiet

join:2004-02-12
Bethlehem, PA

Now's the time

Now's the time to move to fiber - if they can... can't see replacing copper with copper...

PSWired

join:2006-03-26
Annapolis, MD

Sign

In one of the linked photos, there's a sign outside that says "we are using water only." Anyone know what that's all about?

Smith6612
Premium,MVM
join:2008-02-01
North Tonawanda, NY
kudos:22

Can't seem to load the Flash video

Their player doesn't want to load. Any other means working (maybe YouTube?) that have this video?

Wednesday, 19-Jun 18:58:43 Terms of Use & Privacy | feedback | contact | Hosting by nac.net - DSL,Hosting & Co-lo
over 13.5 years online © 1999-2013 dslreports.com.