As a proud owner of a mighty beard myself I fully support your statement, but do tend to agree that shaving normally gives men a more “professional”, a more smooth “business-ready” look. Not on me personally, or maybe you, but there’s a reason why customer services and jobs that revolve around dealing with other people require you to shave and be as presentable as possible to their clients. For example, working in a bank requires you to shave, at least where I come from.
You can make this a game-y thing, where shaving equals trying to be presentable, and be valued by other humans more after access to basic hygiene-preserving tools has been lost in the post-apocalyptic world they’re living in. It shows an attempt to try and appeal to another person, which when contact with other humans is scarce can score you extra points in their book. “We’re all starving and freezing to death but this guy manages to look good even at times like these, surely he cares greatly about others’ comfort”.
That’s the way I’m looking at it, even if it doesn’t necessarily make sense from a realistic point of view.