3rd gear grinds are fairly common in higher mileage '03 & '04 models, hard/fast shifting will certainly make it more noticeable. Grinds in 5th are less common because it's not a gear you're moving through a lot - but it still happens. Like you, I'd say it's the synchros.
If you're not looking to have your transmission rebuilt you can try different transmission oils. I've heard people have success with
Redline Shockproof (aka Smurf Jizz),
AMS 2000 75W-90 and
Royal Purple 75W-90 Max Gear Oil. Do some research on your own and figure out which one you want to use.
Keep in mind though that chemicals will never really
fix mechanical problems.
EDIT:
AMS Severe Gear 75W-90 has replaced the Series 2000 75-90 that I linked to earlier.