I do not agree about damage to the player.
The character should be able to go about smashing relatively intelligently. They may try to apply leverage, they may try to put all their weight on a shelf, they may try to stomp on parts, etc.
Basically, punching furniture would be a last resort unless the character is bullshit enough to karate chop it to tinders.
Zombie-wise, I’d think it’d be more case-by-case. A wire fence would just be pulling and pushing until it gives up, no damage there. A boarded window might be the same, but a bookcase they’d just have to smash.
On the other hand, zombie skin is harder than human skin, so they probably escape that with no damage.