I have found that sight-reading while playing lined fretless is a lot like driving. I can keep my eyes on the car in front of me, and the yellow lines, and the rear view mirror, and the side mirrors, maybe even send a text message (sorry, officer) and I can do this reliably time after time, with my life on the line. Likewise I can sight-read the sheet music, sneak a peak at my lined fretless, watch the conductor, make eye contact with the drummer, and give good stage presence, all at the same time.
You make some good points, but I've seen too many counter-examples to agree. A famous example is Sting. Watching his performances, I get the feeling Sting has actually practiced "sneaking a peak" at his left hand, in front of the mirror, as a specific and useful skill. He does a little "head-bob" that kind of tricks the audience into thinking he's rocking out to the beat, but actually he's looking at his left hand. These are the little touches that make Sting such a great performer. But he wasn't born able to do this; at some point he consciously decided, "this is a skill that is worth having." Go watch some Sting concert video and you won't be able to un-see it.