Congrats on handling and analyzing the situation in a very level headed way.
I'll be the first to say that learning other styles is a good thing, but if you're an amature musician, and you have a day job, then enjoying yourself is first priority. Why play music you don't like if you don't have to? It sounds like you weren't enjoying yourself at this point and it was all for the best. God speed finding a new gig that fits you better.
When I was 18, I was into heavy rock and roll and progressive rock. Rush, Yes, Zeppelin, Pink Floyd, Van Halen (not Van Hagar...) - but you get the idea... I was a rocker in the middle 80's who had an original 'melodic heavy metal band' on the side...
At that time I was at UMSL in their Jazz band, the keys player asked if I'd be interested in doing a 'pep-band' gig that he ran. It was sponsored by Grey Eagle Distributing (essentially Budweiser) - steady gigs, good pay... sure, why not?
The music was mostly "Fake Book" + whatever snippets of any popular 'sports event, get the crowd going' songs. Think Paul Schaeffer's band - there to fill the gaps in the action. But in my mind, the music was bland, boring, 'old-fogie' stuff...
The Keys player was about 9 years older than me, so he sort of looked at me as 'a kid who played bass but was probably not all that mature, musically'. He even said to me, "...this type of music really requires a special 'touch' to sound right on bass. It's not Rush or Zeppelin, you know... do you think you can do that?"
I looked at this as an opportunity to prove to myself (and him) that I could deliver the goods as needed and that just because I was the 'baby' in the band and that I was a "rocker", it didn't mean I couldn't be musically broad.
He liked what I brought to that gig and asked me to join his 'other band' - a cover band that did Jackson 5, Cure, Steely Dan, Jeffry Osbourne, even Spyro Gyra, - at that time, I called it "adult contemporary" and got a dirty look from Mark (keys player) but was offered the gig anyway - in spite of calling his band "Adult Contemporary".
I played with them for 10 years after that and the music I was playing with them early on was so far from what I claimed to be "my flavor of music" at times I was even surprised. We were playing stuff I'd never in a million years have sought out on my own due to my prejudices about such styles.
But if I'd allowed my "comfort zone" to influence my choice to play with them, I'd have never discovered the great, great motown, soul, R&B, and even the alternative pop that was coming out throughout the 80's. I eventually became a central figure in that band and ended up influencing song selection, helping select new members when others would move on... It ended up being one of the most professionally and musically rewarding gigs I've ever had - today included.