Even if the IP out of the modem is 192.168.5
.x and the IP out of the router is 192.168.6
.x - you are still doing double NAT, which is a problem...
Best method would be to put the modem in "bridge mode" where it hands the real IP from your ISP (not a 192.168.x.x) directly to the router.
Second best would be connect the Xbox to the modem directly.
Third train-wreck of a solution would be to port forward through both the modem and the router: »support.xbox.com/en-US/x ··· box-live