No you don't. The loudspeakers don't know or care what part of the signal is clean and what part is distortion. Putting too much power into a loudspeaker will tend to burn it out. More than 90% of the power you put into a loudspeaker driver doesn't turn into acoustical energy--it turns into heat. That's about the same efficiency as an incandescent light bulb.
Imagine the heat coming off a 100-watt light bulb. That's about what you get putting 100 watts of audio into a loudspeaker driver. If you have, say, an amp rated at 400 watts into 8 ohms, and it's feeding an 8-ohm cab, you're probably putting an average of about 50 to 100 watts into the cab most of the time if you're only occasionally tickling the clip LED and not using heavy compression or other processing that squeezes the dynamic range. If you crank it up so you're running the amp into a lot of clipping, or maybe you've got lots of distortion or compression that squeezes the peaks that would otherwise alert you via the clip LEDs, you might be putting out an average of 200 watts or more. Theoretically a square wave from the amp in this example could reach 800 watts, but by that time you'd have really obliterated any musical sense in your playing.