Im pretty sure that the reload time is only meant to simulate how long it would take you to grab the mag manually fill it with bullets and load it again into your gun, with guns that have no mags/internal mags its probably just the time you spend doing the action and placing the bullet.
The time it would take you searching for them in your inventory is not a factor for now, so it makes no sense to bandoliers (and other items) to help with anything, they could however be a way to carry ammo without using your main storage volume (like quivers)