Well, you can have too much of a good thing. For instance, the input impedance of a non-inverting op amp could easily be in the tens of gigohms - that's like 10,000,000,000 ohms! This means that ANY signal or noise, no matter how weak or at whatever frequency can make it's way into the circuit. You'll be picking up Monrovian ham radio operators and WWII Japanese submarines! Usually engineers want to have control over all the parameters, so they design in a specific impedance and filter characteristics. They want the device to work in any situation, so they limit it's operational parameters so the likelyhood of picking up earthworm farts is greatly reduced!
You're on the right track re; capacitance and linearity. You cannot escape having some capacitance, so rather than fighting it, they work with it. Usually a couple of filters create a little nook for audible frequencies to pass through, but everything else is kept out. And adding resistors just helps to coral the various dynamic parameters into something stable, known and predictable.
