For many situations I fully agree with you. I also do not recall many places where I had to read double sharps or flats in bass music, although if you look at the classical literature as a whole, e.g. at piano works by many composers (e.g. Chopin, Liszt, Debussy,...) you'll find them frequently.
Yeah, I mean, go download the Well-Tempered Clavier, or Chopin's Preludes, which both run through all keys, and you'll find plenty of examples. E.g., I'm looking at a copy of the 3rd prelude from the Well-Tempered Clavier. It's written in C#, so there's an E# and a B# in the key signature, and plenty of double sharps sprinkled throughout.
Surely bassists in pit orchestras, for example, must have to know how to sight-read well in those keys.
Trying to show him how to play something, or trying to get him to show you something was just painful.