This is precisely why I always pull CAP_SYS_MODULE after boot. I HIGHLY recommend those use:
May want to remove CAP_NET_ADMIN as well after system boot.
/sbin/sysctl -w kernel.modules_disabled=1
I also highly recommend, from a security aspect, to panic on kernel oops.
/sbin/sysctl -w kernel.panic_on_oops=1
Pulling the POSIX CAP_SYS_MODULE capability is one-way and cannot be undone until reboot. Additionally, many newer Linux distributions have a nasty habit of allowing userland to dynamically insert/modprobe kernel modules (udev perhaps?) that may be vulnerable. Pulling CAP_SYS_MODULE goes a long way to securing your system.