The potential downsides are not learning "proper" tenchnique, possibly not learning to read music, etc.
The potential upsides are developing a good ear; not being restricted to playing only chords, scales, etc. that you know the "proper" names and theoretical uses of; being able to think in terms of the sound itself, rather than in terms of the names and symbols for sound.
It's always good to take a couple of college-level courses in basic music theory and composition, even if you don't end up using or retaining every detail that's taught. You will retain enough to help you understand how and why things work and fit together musically, basic principles and terminology for musical concepts, etc. The really basic stuff like dynamics, major vs. minor, octaves, steps, half-steps, common intervals, tension and resolution, etc.