You sir, unfortunately, or fortunately, are unable to hear the poor quality of your audio. So, I'm going to do the neighborly thing and show you how poor your audio is and explain it.
The results you are going to see is similar in two M35 07, one G37, and an FX (the fx I didn't test, I could just hear it). It sure would be a mighty big coincidence if all five of us had abused systems

, and one of the M35 the owner doesn't even listen to music, only sports radio.
Before I go on let me first say, 3/4 of my mp3 are my own direct rips from official CDs. So I know the quality. All ripped with studio software. Some of the mp3 not ripped by myself are tested to see the original encoded resolution through a spectrum test. So when I say they are 320, they are 320. And remember I did say I can hear the issue through CDs as well. My background. For 15 years I've been a recording/mastering engineer and built and redesigned recording and film production studios. So when I say I can hear the quality and resolution of audio...I think i have the history and a
little bit of the qualifications to say that.
Anyway, what you are about to see are a couple of tests that I run when building recording studios, except with equipment a little more sophisticated. The first is with a sine wave, the second is with pink noise.
The first pic shows the distortion at a little less than HALF volume output of the headunit, clear and evident since YOU can't hear it.
This pic shows the clipping at 3/4 the volume output of the headunit, again clear and evident.
Those pics were just from my car at 60hz @ -5db if I remember correctly, but the same test was done on four Infinitis at 40, 60, 500, 1000, 10000 hertz all giving the same results. Those should be perfectly smooth all the way around. Like this:

This is really sad because this comes from a nano amp that I'm using to power the center speaker off my computer that is producing 55wrms and not giving ANY distortion at max volume which powering the center channel louder than the factory...$50 priceless. (more on this in my upcoming write up)
The following pics are of tests only done on my car, because I wasn't going to take my unit out to test on other cars what I already knew.
This is what the Bose is actually putting out. Shows the disgusting mess the bose is producing.
This pic is the final output, the best the fostgate could do with the screwed up spectrum of this Bose. NOW, the system is outputting a relatively leveled frequency spectrum except for the base which is most likely on a low pass crossover internally off the factory amp.
This pic shows the same thing the very first pic shows in a different form. Instead of a little less than half it is at half the volume output. This isn't showing clipping, it is showing distortion:

This is at 3/4 volume output.
Clipping is done with a different test that I didn't do...but can if you insist.
One of the biggest problems is that the factory system is a two ohm system. This doesn't inherently mean that it will cause poor audio quality or distortion (Except in Infinitis case

), but it does mean it is more likely to do it than a four ohm. Most other "luxury" cars are running 4-16 ohms...anything above 4 is a little ridiculous in my humble opinion...and that is because they know that they operate with more efficiency and generally produce better sound. I would also put money on the fact that this is probably a class D amp system adding to the mess, but I don't know for sure.
Today to top all this off I ran a little test. I decided to run my music through my carputer over bluetooth direct to the fosgate processor, totally bypassing the bose and headunit. OH MY GOD, the difference in sound was UNBELIEVABLE. (I'll get into detail in my write up...especially why I actually decided to do this). ALL distortion went away, volume output tripled with NO clipping...I didn't push it more because I didn't calibrate the audio out the computer to make sure I wasn't pushing out too much that would blow the speakers. I did end up picking up the expected artifacts from the noise floor by going bluetooth (or it could have been the carputer), but it was slight and still a better sound. What does this mean? Getting rid of the factory headunit and/or amp in the equation is a must! Just a matter of how.
By you saying at 100% your system doesn't distort it is clear you simply can't hear distortion unless your speakers are baffling and buckling...no offense...EVERY headunit by their nature will start to distort around 90% becoming real noticeable around 95% (we wont even get into this amp). It doesn't matter how good the headunit, amp or speakers are or how expensive, or if it's factory or aftermarket...although spl comp speakers could mask it a little. You not believing that isn't going to change reality, physics, or nature of the components. This car unfortunately starts failing at an earlier point...speaks to the system it is

.
Side note, I am curious though, exactly what crispness or lack of distortion am I to hear from a stereo playing an mp3, through speakers 3ft away, being recorded with a cell phone, playing back from YouTube

.
Yes I am new to car audio, simply because I've never had the need to improve it before, but audio is audio and the principle wrms, thd, clipping, distortion, and voltage is the same no matter what. At any rate, as long as you're happy with your "good quality files" and your system...as I've said before, that's all that matters.