I've been getting 400-600 ms every evening between around 8pm and 3 or 4 am. I just started a time-stamped ping that I'll run for a couple of days to see if I can find any pattern vs the time range I thought I noticed was just a fluke.
Here's what I've got from pingtests between 1:40am and 4:56am. I'll post what it looks like after running for 24 hours and running for a couple of days once I've collected that data.