Changing Door Speakers in 95 Pathfinder

[1995Pathfinder]

Could anyone walk me through how to change the door speakers on a 95 Pathfinder. My daughters speakers need to be changed and the Haynes book does not give enough detail on how to get the panel off. I can pop out the panel, but I can't figure out how to get the door handle latch off; there is no screw to take out, but only a rubber plug. Same with the armrest; I can't find a way to get that off either. Any help or direction would be appreciated.