Power window motors tend to wear out on the '03 & '04' models, check carefully for notchy shifting in the gearbox (it usually shows up most in 3rd gear).
The '03 & '04 models also had some alignment issues that resulted in some feathering and rapid tire wear, but that wan't entirely Nissan's issue. I think a lot of buyers had never had a car with grippy rubber with a low treadwear rating (the OEM tires were Bridgestone RE040's and a meager treadwear of 140). There was a TSB issued by Nissan that basically said there was no tolerance for a wheel alignment...the alignment needed to be spot on.
I'd still get it checked out by someone with access to a lift in order to get a good look below. It's over the 8yr/80K mile warranty for cat replacement so even if you choose not to have a mechanic check it out I'd take a code checker to the lot and at least run that and make sure that all the maintenance is up to date - I would even try and have them take care of the 100K service. Remember, you can afford to be picky...there are plenty of Z's out there for sale.
Welcome to NICO (and hopefully to the Z family)!
