If you think about it, consider this situation:
If Brock Lesnar and Fedor Emelianenko both punch with a pretty similar force. Which I think is pretty likely because Brock had great size but not great technique, and Fedor had decent size but great technique and ferocity. So lets just pretend for the sake of this test that it balances out.
So let's say they both represent 100 punching power and lets say Brock has a hand size 10 and Fedor has a hand size 7, people often think you multiply the power by the hand size to get how much total force is there but that's wrong. What you do is divide because force is because spread out over the area of impact.
So Brock hits you for 100 over an area size 10, that means you only feel 10 damage.
Fedor hits you for 100 over an area size 7, that means you feel 14.285 damage.
Brock's fist is like getting hit by a big tree stump, which sucks but Fedor's fist penetrates into you like a spear piercing deep tissue and organs.