I try not to fall into the temptation of putting my weight on the opponent, because that usually ends up with me being thrown around like a ragdoll.
This is the number one beginner mistake that holds people back from getting better at passing. I totally understand where you're coming from because I used to think that way too, but you will never pass anyone's guard that way.
Start working on putting your weight on your opponent. You will get thrown around a lot at first, but eventually you will learn how to ride it out and stay heavy on top.
The big advantage you have during guard passing is your weight. You need to use it to control your opponent's hips. Focus on putting your weight forward onto your opponent's hips.