but is it really the voltage increase, or the current increase that allows faster switching?

Or is the voltage a sneaky way of bypassing current limits?