What exactly is the difference between a foldable and non-foldable vehicle?

Just as it says on the tin.

a foldable one can be folded and wielded or put into a container - like a backpack or trunk

A foldable vehicle you can pick up after folding it and either wield or put in your inventory, depending on its size, in order to carry it around.

So, is there no downside to making a vehicle foldable? I was about to install a fold-able wooden frame and a pop-up asked me if I was sure, as if it could potentially bad or something.

A vehicle that has - even just one - non-foldable part(s) will not turn into a foldable vehicle by installing one foldable tile. All tiles have to be foldable to keep/make a vehicle foldable.
Foldable objects usually have lower durability than the non-foldable ones, but are lighter in weight.