It has been really odd because you can watch a video that buffers perfect, followed by a video that won't buffer at all, followed by another video that goes perfect..
I find if I try to replay a video later that didn't buffer earlier, it still doesn't work.
I would check which cache node you're hitting with each time you're playing. It sounds like you're hitting a different node each time the behaviour changes.