For me, there aren't many instruments that sound universally bad (assuming their played correctly). But there are some instruments that only sound good in one or a few closely related styles of music. Banjo sounds great in bluegrass, country, and folky music, but when I hear it over any modern or non-American roots music, it sticks out like a sore thumb. I heard a Latin jazz song with a banjo player once and it sounded absolutely horrible. I feel similarly about harmonica, but I think it's a little more versatile than the banjo (e.g. players like Sonny Boy Williamson and Little Walter vs players like Stevie Wonder and Lee Oscar). I cringe when I hear harmonica over modern rock and pop, but I don't like those genres anyways.
I do find really bright, high-endy, clanky bass to be particularly bad. It never sounds good to me unless there are a lot of other instruments absorbing the sound of the clank.