I just tried it and came up with the same problem, intermittently. In my Testworld I jumped to a nearby town and wished up a road map, and it worked perfectly. Then I popped open my longest-running game (Rob Keys) and wished up a road map for the massive town of Aina that he’s currently parked in, and it did nothing. The roads run up to the edge of the fog-of-war and stop.
I don’t think it’s the “four building” thing happening. It’s possible it’s getting the wrong town, but the next town is quite a ways. If I had to guess, maybe there’s more than one town named Aina? His map is colossal and there are a lot of repeated town names. I don’t have time to scroll through the whole thing and check though.