Speakers have mechanical limits (a point beyond which you could overdrive or over exert them so much that you may see physical damage) and thermal limits (beyond which you may start to see burned voice coils.) When these limits are exceeded for extended periods of time due to excessive operation above the rms power rating or clipping that causes the supplied power to exceed to rated limits, it degrades the life of the speaker and can in some cases can kill it fairly quickly.

Speakers can be damaged if you do not provide sufficient power to get them as loud as you want them and crank the head unit volume or gain to the point where an excessively distorted signal is sent to them. They can also be damaged or worn out if you overpower them by so much that they spend lots of time during normal listening at or above their nominal power limits. Some people who buy top of the line speakers and can be conservative with the amplifier's gain dial connect amplifiers making 2-3 times as much as the speaker's rms rating and can run it like that, and run it hard, for years with no problems. Some people can hook up an 80 watt speaker to their 16 watt head unit or to a 35 watt amplifier and can kill it fast by sending it clipped signals and/or excessive bass frequencies.