IMHO that is part of the problem, but only part. It doesn't matter if it's E-drums or acoustic drums, keeping the volume down will change our perceptions about how the drums sound. I think it entirely possible to get a good tone at a lower volume, but below a certain volume threshold, your body will not produce the autonomic response that makes you perceive the sound as exciting. If you desire the full the experience of loud, then the only way to get that experience is to make it loud. If the goal is to keep the volume down, we need understand, accept, and deal with the fact we don't always get everything we want.
If your going to think of your self as a professional, I think it a good idea to tailor your musical presentation to the patron and the audience's needs. By this I mean strive to play at a volume that is appropriate to the situation, rather than focusing solely on what gets you off.
The drummer calls the shots if he/she is the group leader or in certain situations where they are free to play exactly as they want, because it has no impact on anyone else. For example a huge stage where the band is spread out and the PA is so big/powerful that the audience can't hear any of the drum's direct sound. Otherwise, drummers should use their judgement and initiative to try and adjust to the situation. We are not always in a position to accurately judge how loud we should play. So it's also necessary for the drummer to take guidance from whoever is in charge. That could mean playing louder or softer than we think is appropriate.