Does that mean you still have the service defined but it's not starting? That's pragmatically good enough, but not very satisfying
It's very weird that deleting the registry entry doesn't remove all mention. Unless you're sick of this, can you check whether the registry entry magically 'comes back' after reboot?