Yeah.
It’s probably a combination of early-game having few clothes, and having clothes which are not very damage resistant.
Once you’re dripping in gear, you’ll find the zombies mysteriously ripping your holsters and ammo pouches and armored neckties or whatever you’re wearing more often than tearing your polo shirt.
… What, you don’t wear an armored necktie to the apocalypse?