Another thing that comes to mind for what you might be missing here: has it been raining in your world, or are you building in a garage?
Solar panels need sunlight, as you might have guessed, but what’s not so obvious is that they follow realistic rules for what that means.
Solar panels won’t work at all inside (even next to windows), their production also drops when it’s cloudy, goes to a trickle when it’s raining, and a storm is essentially night.
Large battery banks and well-protected multi panel arrays are your friend with solar.
Make the most of those sunny days, and store enough to get you through the cloudy ones.
My scout vehicles typically have a 6 panel array so they can keep a fridge running full time without draining the battery too quickly. A three panel array can work if you supplement it with a one-cylinder engine with a generator or several alternators on it. (Those are a good idea anyway for the inevitable times when you have to hole up in a garage or fire station, or get a long string of cloudy days.)
For long-term, large scale solar power, build as large an array as you can. Snag every panel you see- even the broken ones can bag you solar cells you can use to build new panels or upgrade existing ones. My stationary bases usually have an array of at least 12 panels and 4 storage batteries for a steady flow of power to a fridge while also stockpiling a small surplus. That way when I get back to base and need to recharge a lot of equipment, and hook up jumper cables to my scout vehicle so my fridge doesn’t cut out while I’m unloading, and do a lot of fast crafting at the same time before things start to spoil, I have plenty of reserve power to fall back on.