edit. Obroin's calc was better, but still proves the general point, even if it's a vastly simplified approach.
Alright we're getting to the bottom of this. Our calculations haven't even come close to approximating the situation. Your calculations assumed point particles with I=mr^2, and the problems with mine were already discussed.
Robbie Lawler is credited with a 188 cm reach (1.9m). Cut that in half, and now were talking a 0.95 m radius for the fist. Now, lets assume he weighs 180 lbs (80 kg), as described earlier. The problem with our approach is that most of that weight is centered in his torso/legs/head. The arms are a very small percentage of total weight (5.7% of TBM according to ExRx). So we will let one arm be 6% of TBM, or 4.8 kg. Now, I have roughly 6 inches (15 cm) between the center of my sternum and my shoulder. So the remaining 70 kg is centered in a "disk" (if you view it from above) of radius 15 cm.
So lets calculate the moment of inertia of 3 different segments. 1) the arm 2) the body and 3) the additional weight at the fist.
1) mass density p1 = 4.8/0.8 = 6
I = p1/3 (0.95^3 - .15^3) = 1.7
2) Moment of inertia of a disk I = 1/2 * m * r^2 = .5*70*.15^2 = 0.7875
So from here we can already see the arm has a much greater moment of inertia than the body (almost twice as much). Now lets do the fist.
3) I = .1kg * .95^2 = 0.09 (11% of the body inertia)
Now, that doesn't seem like much, but that means that the extra 100g at the fist has 11% of the inertia of the 70 kg in the body. AND that's assuming that all 70 kg is used to generate a moment of inertia, which is impossible. Assuming that 100g is rather small on the total natural variation, it shows that that hand size is pretty significant.