I can't remember how vital closing the gates are to completing the game, but I'm pretty sure you have to close a certain amount of them to advance, so you eventually have to start clearing them. I've heard two ways you should do it. Either right at the beginning where you and the enemies are equally shit, or at the very last possible moment where you're practically a God. Anything in between that, and you're getting proper fucked.
I never actually beat the game, but I remember the gates being on a completely different level than the world. Little shit goblins are shooting fireballs at you that scorch you for like 40% health, and they take like 20 hits to die. Then when you manage to actually get into the tower(or whatever) and it's nothing but beefed up end game enemies killing you in two or three hits. I believe I got through a few by save scumming, and it just burned me out and killed my interest in the game.