Ears in or out?

Ok, this may seem like a rather silly question. Where exactly do your ears sit in your headgear? I have a small bit of cauliflower ear and wearing headgear irritates it more, but I find that having my ears inside the provided holes leads to less irritation. Plus, it hurts like a mamma jamma. Are your ears supposed to actually sit in the provided holes or do they just sit under the headgear and the holes are there to allow for hearing?

If it makes a difference I'm using the standard Fairtex sparring headgear.

 

Maybe you need bigger headgear.