What can I wear to hold myself in while running?

It makes no difference to the elasticity of your skin if you are bouncing around or not. Compressing it won't tighten your skin as the weight comes off. You'll just have to be patient, and lose the weight at a healthy rate, allowing the skin to naturally tighten.