Yes. webmail in a browser won't access Android contacts. but mail in a phone browser doesn't work nearly as well as mail in a phone mail app.
I suppose I need another phone mail app that doesn't get permission to use all my android contacts. Or if someone knows how to truly disable Yahoo Mail Android App from seeing Android contacts, that would be even better.