Depends on what you're doing, wrestling or bjj?
In wrestling you can try to make room with your hip and face him or reach down and "hook" his elbow with your elbow, hold it really tight lean forward go to your knees and "pivot" your arm under you to pull him to the ground, plant the arm not holding his on the ground and try to lift yourself over his body into a side mount. ( I hope you can picture this)
Or before he tries to take you down make sure you get your feet wide, reach down with one or even both hands, grab him behind the knee or his heel and yank it up while sitting down, you should land right on his chest and his back to the ground which being a wrestler he won't like and may turn to his chest.
BJJ Do a kimura, watch a Sakuraba fight to get a good idea how to do it.