I think learning how to do it is a good skill. Even thought I usually use my clutch, sometimes I practice upshifting and downshifting without it... just for fun.
It doesn't hurt the car because when upshifting, you wait until the revs of the next gear drops down to match the speed of the collar. When downshifting, you rev the engine of the previous gear to match the speed of the collar.
Why don't you guys give it a try... In 3rd gear, you'll be driving at about 35 mph and 2500 rpm. If you shift into 4th at the same speed, you'll be at 2000 rpm. So, if you didn't want to use the clutch on that shift, you put the transmission in neutral and wait for the revs to drop to 2k rpm, and then the shifter should be able to slide into 4th. It usually takes about 1 and a half seconds for the revs to drop, so I don't even watch the tach. I just wait until the right time.

Hope this helps.