I finally found a fix for this issue. The culprit was UPnP. I had disabled UPnP on my router even though it wasn't affected by the UPnP vulnerability and since then I have not had to reset my router while streaming Spotify.
Hope this helps anyone who might encounter this issue in the future. Also, after discovering that my issue was UPnP related, I was able to find others with this same issue.