To be honest, I haven't personally removed the blower motor on a y34. I just recall pulling the panel years ago to see what was there and it looked pretty accessible.
... but the FSM says you need to remove the glovebox, so....they make the car, they write the manual
On leaves, as long as your in-cabin filter is intact, then the likelihood of them making it to the blower is little. And leaves and sticks eventually break down, if it sounds like a buzzing/humming/growling, then I'm still looking at those bearings.