Verizon FiOS → Very strange issues with FIOS connectivity, Wifi and Ethernet

Yesterday morning everything is working fine then suddenly we lost all connectivity via Ethernet and Wifi on every Windows based system but access was still available on an iPad and an iPhone.

I performed the following processes in an attempt to resolve the issue:
Reboot all systems on the network and the router.
Release and renew the IP addresses on all devices.
Checked every Ethernet cable connected to the router.
Reset the router to defaults.
Checked every configuration option in the router.
Tried to access sites by IP address.
I've also bypassed the router by plugging a laptop directly into the wall jack and still nothing. Network status reports that it's an Unidentified Network.
Everything failed.

All of the systems on the network showed that they were connected, had an IP address but we could not access any resource.

Called Verizon, went through the usual troubleshooting options and when it couldn't be resolved they scheduled a visit from a tech who was there within an hour.

The Verizon tech replaced the ONT and the router. I released and renewed the IP on every device and service was restored, for about 30 minutes. Then every device lost connectivity again including all idevices but one Windows desktop on WiFi continued to work.

Went through the usual troubleshooting options and still couldn't fix the problem. Verizon sends out another tech. He's switched out the router 3 times and the problem continues. The Ethernet connection on one desktop worked for a while but now even that has failed. The techs tablet also shows that it's connected and has an IP but cannot resolve any URL.

There are two Verizon techs at the house now, they've been there all afternoon and both are stumped as to what the problem might be.

So, has anybody, seen anything, like this before?

One more thing, VOD isn't working either.

I'd like to think they've checked your cabling with their sunrise meters. And the data light with their fiber meter...

Thanks, I'm inquiring now. Three techs here now.
The Verizon techs said if there were any problems with that, TV and phone wouldn't work either, makes sense. Ethernet and WiFi are working again but not the Cisco phone system. They're coming back in the morning to start again.

How long have you had service? Is this a recent install?

No, they've had service since it was available a few years ago. Nothing new has been added to the network, no changes have been made. The Cisco phone system has been running for 2 years+, a smart TV and a WiFi Blu-Ray player were added months ago but nothing more recent than that. The problem just suddenly appeared yesterday morning.

Slightly alarmist; but make sure the wifi is quite secure, WPA not just WEP. Someone could be into it and *attempting* man in the middle between your systems and the router, but failing to pass the data, or just maliciously messing with you.

It's setup with WPA2-PSK, the Cisco system is running over a VPN so I don't think that's an issue.

Im thinking that there is something on your network is flooding the system. I would disconnect all devices, then start adding them back. start with the hardwired ones 1st

Yup, I thought about that. Disconnected the Sony WiFi Blu-Ray device but not the Vizio smart TV, I'll try disconnecting that from the network as well tomorrow.

The normal setup that has worked forever includes the following:
3 iPhones, 1 iPad, 1 desktop connected via WiFi, a Sony Blu-Ray device, a Vizio smart TV and a Cisco router connected to the Verizon router solely for the Cisco phone used over a VPN, 1 desktop connected via Ethernet and the occasional guest accessing the network. This setup has been working for a long time, years, until yesterday morning. Absolutely nothing new has been added to the network for at least a few months.

I've seen wireless printers and security cameras jack up WiFi connections but I've checked for both possibilities and found nothing. I've changed channels on the router and so have the Verizon techs but that hasn't made a difference. And why would that kill the Ethernet connection?

I've done troubleshooting on tons of Verizon Fios installations and I've never seen a problem quite like this one. Three Verizon techs can't even figure out what the hell is going on.

The only thing I know for a fact that has changed recently is the new FIOS IMG update.

And thank you PoloDude for taking the time to make some suggestions.

Did you power down all STB/DVR? Any MoCA LAN connections could be messing up network connectivity by flooding the network, etc. Not sure what AT Rev they are bringing you.

Check your MoCA stats.

I would also break the tunnel (VPN) and shut down the IPphone and disable wireless.

Start with hardwired devices ONLY and one at a time, and that means all wifi is disabled and all MoCA devices asleep (physically unplugged)

You didn't mention if they provisioned coax or ethernet out of the ONT. That would be interesting to know.

No devices are reporting IP conflicts, and you don't have a mix of hardcoded and DHCP address in your network?

Typically when I find networks get messed up they either exhaust IP DHCP addresses (which can lead to random connects), or there is a mix of hardcoded and DHCP and they are clobbering each other. Also if you had a STB update that went bad, MoCA devices could be flooding the AT causing a DOS.

Thank you!

In regards to provisioning coax or Ethernet I can't really say for sure since I didn't think to ask. Quite frankly there have been so many Verizon techs onsite trying various solutions that I don't know where things are at this point. I'm not onsite now so I'll have to check tomorrow. I'll also try pulling a new update for the STBs tomorrow.

This is one of those situations where you've tried so many things that the whole issue starts to become a blur. I probably need to step back and reevaluate my approach to the problem.

Thank you again for the suggestions! I'll try again tomorrow morning and will post an update in case the solution is of use to anyone else.