I have read about 2/3rds through the thread and find it pretty humorous...or should I say sad.
I could be wrong, but I get the impression the OP is more of a set-it-and-forget-it type. The OP repeatedly infers a distaste for going down the old "old rabbit hole." The drummer is more of a tinkerer type, who likes to constantly assess and try to make improvements. So right off there are two conflicting mind sets.
I am probably more like @Analogeezer as I have decades of experience as both a pro touring bass player and also a pro touring audio tech. I am squarely in line with the tinkerer mind set, and I live constantly in the place the OP would describe as the "rabbit hole."
One problem is the the term rabbit hole has a negative connotation. Here's the definition: Rabbit Hole (is) "used to refer to a bizarre, confusing, or nonsensical situation or environment, typically one from which it is difficult to extricate oneself."
Here's the deal, running sound is part science and part art; so the environment I work in can tends to appear "bizarre, confusing, and nonesensical," to someone who has no grasp of acoustics and/or how to build a mix...but it's actually a medium I enjoy working in, and I feel that I can generally make better. I have received tons of positive feedback from audience members after shows.
Unfortunately due to the nature of acoustics and how sound propagates, I generally don't have total control over the acoustic environment. If the musicians take an inflexible attitude and refuse to negotiate on levels and possibly EQ, it tends to degrade or even totally eliminate my ability to build and maintain a good mix. There is generally no set it and forget when you are running audio. There is almost always something you need to change. People tend to blame the audio tech when the mix is bad, but I can assure you the musicians frequently have a significant role when the sound sucks at a concert.
My personal feeling is there needs to be a reasonable compromise, because it's generally impossible for everyone to get what they want. The audio tech is an advocate for the audience, but the musicians must have a certain quality level in order to perform. Frequently the audio tech will desire the musicians to turn down so he/she has more control over the house sound. In other words, the idea is to reduce the level of the stage wash, so it does not mask and degrade the clarity the PA can provide to the audience. In many circumstances the audio tech would prefer if there was no sound coming from the stage, but of course that is entirely unreasonable unless everyone is in IEMs and the drummer is playing an E-kit.
IMHO this is where the compromise comes in. The musicians accept less stage volume than they want, and the audio tech accepts more stage volume than he/she wants. Nobody gets exactly what they want.
Here is my best guess as to what is going on. The drummer wants to hear the bass more clearly. It does not matter that the OP has a fabulous bass amp, the drummer can not hear as well as he wants. In my experience every drummer is different, some want to hear only somewhat incoherent lows, and some want to hear more of the attack and tone center.
The front of the cab is where most of the clarity and focus are. The side and back of the cab will only provide the incoherent lows. Remember, the low frequency dispersion of a cab tends to be omni-directional, and the mids and highs tend to shoot out the front of the cab.
I don't find it unusual whatsoever for others to comment how they perceive the tone of my bass. I generally worked with full PA systems where everyone had a monitor, so if anyone wanted more or less bass they worked it out with the audio tech. Sometimes I did need to turn down, but due to the way I setup my amp, this was rare.
I also did some gigs where my amp carried the house, and little if any bass was sent through the PA. Generally the band leader will go out and call audibles: turn up/turn down, more/less treble, more/less bass, etc. What the leader wanted to hear out in the house was often the opposite of what I wanted to hear on stage.
In addition, sometimes the other musicians may critique or ask for changes to the bass sound. As a tweaker, I understand that there are times I can make changes that will make someone happier, but you probably can't make everyone happy. If the band has a leader, as far as I am concerned the leader calls the shots. If the band is more of a democracy, everyone should negotiate.
Unfortunately many people seem to expect that their listening experience must be perfect. IMHO there should be an expectation that the bass amp is the bassist's personal monitor. So there should be some priority given to the bassist's preferences and needs, but there still should be some degree of compromise.
Here's part of why it's impossible for everyone to have a perfect listening experience. First you need to realize that SPL drops off at a rate of 6dB for each doubling of distance. So lets assume you are hearing 124dB at a distance of 2 meters from you amp.
Here are the doublings and result SPLs on axis.
12m = 118dB
24m = 112dB
Let's keep in mind the 2m is about 6", 12m is almost 40', and 24m is almost 80'.
Now the next part of the puzzle is each person typically wants to hear themself about 6-10dB louder than everyone else.
Let's assume a huge stage where you (the bass player) are 12m/~40' from the guitarist. At this distance your 124dB drops off 6dB to 118dB at the guitarists position. In order for him/her to get the desired 6-10dB, the guitar must be set for 124-128dB at their position. If they are at 124dB, you may perceive the guitar as being right on the edge of too loud. If they are at 128dB, if their amp is aimed in your direction, you will hear them at 122dB, which is only 2dB below the level you are hearing your bass. Remember you want your own 6-10dB, so you will probably think the guitar is too loud.
The idea here is if everyone sets their volume to 124dB, all of the musicians must be separated by approximately 12m/40' to get close to the desired balance. Most bands perform in spaces that are way too small to separate everyone by 12m/40'. Also spreading people out too far tends to make it feel like you are playing in different times zones.
Hard to say for sure, but IMHO: OP you seem to be taking this way too seriously/personally. I doubt the drummer is really critiquing your sound. Rather, it's likely he is simply saying he's not getting exactly what he wants. Maybe there is room for compromise, and maybe there is not. A solution may involved changing your, volume, changing the position and aim of your amp, and even adjusting your tone controls.