Could be a router beginning to fail, could be a loose connection. Reseat all cables. If that does not help, swap in the other router. Could also be the modem, the line or an issue in the Central office of the phone company.
If reseating/replacing cables and swapping out the router does not resolve this, call the support team 24/7/365 at 800.556.5829 for complete trouble shooting.