I've been having the same problems in the Queens, NY area.
Around morning time I won't have any problems with Youtube or Twitch.tv Then around 4PM or so most YouTube videos take forever to buffer. Sometimes even buffering 240p videos take too long to watch it instantly. The worst is trying to watch Twitch.tv streams. It would be perfectly smooth at 1080p+ and then when evening rolls around you get constant stuttering and pausing even at the lowest resolution setting.
If I do a tracert on Twitch.tv, it seems to start timing out after 18.104.22.168. I asked TWC about this in the direct forum and they said they have no problems on their end.
Also, trying to reboot the modem just causes it to not reconnect for several hours so I'm never rebooting the modem again.