Take the tensioner off, put it in a vice or C-clamp and compress it til there's only like 1 or 2mm of the piston showing. Put in the stopper bolt. Bolt the tensioner back on loosely, push it against the belt, tighten up the bolts and remove the stopper bolt. Rotate it like you did and check it again. Allowable is 3.5-5.2mm, but 4mm is like perfect.
Also, the reason your belt has movement front to back is because you don't have these on:
When everything is bolted on (like the crank pulley/balancer), these spacers keep the timing belt from moving front to back. You can't do the timing belt with the crank pulley on so expect it to move some.