I think the key is that you don't really need the sound guy to be
an expert music player. You DO need them to be an expert music listener.
Any sound engineer worth their salt will know the different mix requirements of various genres. They will also know roughly when to expect some dynamics. "OK, this power ballad is about to kick up"
There's a lot of truth here, IMO. One of the other FOH guys at my church doesn't play anything, doesn't sing anything, and probably hasn't done either since he gave up the trumpet in high school. I don't know if he can carry a tune in a suitcase. But every time out, his mix is solid and consistent. I know that he prepares ahead of time by listening to the specific songs we'll be doing as well as reviewing any notes he has (mental or otherwise) from working with these band members previously. He also listens extensively to the genre we do. Seldom do I sit in the house and think "oooh, I might have done that bridge a bit differently" (subjective criticism), and very rarely do I think "I couldn't hear the lead line on that verse" (objective criticism). It's more likely that I'll think "Hmm, I wonder if that would have worked as well with player X as it did with player Y" or "I need to ask what 'verb setting that was on the last song, so I can try it out in two weeks when we're doing _______."
I've thought about this before, and have wondered if his ceiling is lower than it would be if his musical knowledge / ability were greater. His mix is indeed solid and consistent, but it is never spectacular. I often say to myself "Yeah, that was nicely done," and never say to myself "Wow, the band really hit it out of the park today." So does his lack of "musicianship" keep him from being an even better FOH guy? I dunno.
Part of me wonders if there's a parallel between FOH and conducting. In a former life, I was a classically-trained musician. I was certainly not the best tuba player in the country. Or state. Or county. Or city. Heck, I wasn't the best player at my small college. I did ok and that's about it. And once I did far less than ok, and was humiliated at a community band audition by making an astonishing error and losing out to a high-school kid with a plastic sousaphone.
BUT... put a concert band score in front of me and a baton in my hand, and THERE I was at my best. Not going to wave my own baton because that means absolutely nothing to anybody on a forum 25 years after the fact, but I was pretty good. Was it because I was a good tuba player? Certainly not - I was ok, not good. Was it because I could tell you the difference between an Italian Sixth and a French Sixth at fifty paces? Heck no. Was it because I understood the intricacies of the register break on an A clarinet? Uhh... no. But there was something there that at an elemental level just made sense in my ears and my head, and I was good.
Do some of the same things apply for FOH?