Learning to read music: Worst decision I ever made.
First of all, it turned me into an elitist, just like the OP pointed out. Why, with each quarter-note I read I become an even bigger snob. I'm only about two measures away from being completely insufferable. Before learning to read, I was a nice, tolerant guy but now I openly despise anyone who can't read music - even if they don't play an instrument.
Secondly, it stifled my creativity. Gone are the days when I could freely create by throwing a bunch of notes together without an inkling of how they relate to each other. Everything I play now sounds calculated and boring and predictable. Oh, to be freed from the bondage of knowledge!
Third, and worst by far is that now, when I walk down the street, dogs bark and babies cry and people point at me and speak in hushed tones.
Reading music sucks.