You do if the alternative is unconsciousness. Letting just the slightest bit of air out slowly can make all the difference without taking away much of your core stability. I'm not talking about emptying your lungs - just letting a tiny bit of air escape slowly. I'd prefer to be able to just hold my breath 100%, but if I did I'd have to sit down and wait for my vision to return after half my reps at 85+% of 5rm. Some people seem to be far more prone to dizzyness, vision loss, etc. after holding their breath under this kind of stress.