Concrete and metal shouldn’t be burning, but a lot of buildings are designed with Walls and Floors, which are made of wood, when they should actually be designed with Concrete Walls or Metal Walls and so on.
You can use the examine key (x by default I think) and scroll over to a square to look at it. Then pressing e will give you a description of the terrain type. This will let you be sure you’re not setting a fire by something that can burn.
If you want, you can change many building spawns through JSON pretty easily. You’d just want to replace entries for Walls and Floors.
Fire does definitely spread too fast through flammable buildings, and can’t be put out effectively with water, although fire extinguishers work if you get to it early.