Rather then make houses less common I’d rather they spice up the potential loot you can find in houses, just going by my own house as an example I’ve got a few replica weapons and an absolute boat load of books. My neighbour is the very definition of a hoarder and my parents have a shed literally full of tools (you actually can’t step inside and need to partly empty it to get at stuff at the back) and cupboards stuffed full of tinned, jarrred and dried foods.
Edit: Last I counted I personally owned over 200 books, that was some years back, the number has only grown since then.