I've been doing one a month or so for the last year.
It's great fun. However it's hard work, as you're 1 step from a train wreck all night. You need a band with BIG ears, that can keep up with a singer who's just sung the verse double speed and has hit the chorus 4 bars early. I'm lucky to have a drummer and guitarist who can pull that off - if either of them quit the project I'd probably have to stop doing it, because they'd be too hard to replace. It's a fun night, and low preasure because you can make a few mistakes, but it's not the time to be worrying if someone in the band is going to screw up.
Big book of lyrics is a must, and someone in the band who can sing is a big help too.
In terms of the talent we've encountered - generally it's better than we ever expected. We brought in a few ringers for the first few gigs (it's always helpfull to have someone you can drag up to sing at the start of the night), but we've now got a small following. Of course there are the odd guys who are clueless to a level you wouldn't believe possible, but more often than not people give it a good shot. A reasonable proportion of singers are actually VERY good, to the extent we get their numbers, and try and fix them up with real bands.
Financially it works - the band get paid a little less than in our regular four piece bands, but we only have to split it three ways, so the bar and the players both come out ahead.
We all love it. Keeps you sharp, and is never boring.