Your point is certainly valid. However, many of those losses were later in his career when Penn was clearly not in his prime.
Consider this, BJ finished Hughes for the WW title in his first UFC WW match. Then BJ left the UFC and gave up the WW belt. Eventually came back and almost beat GSP in WW title eliminator fight in 2006. BJ Penn won 2 out 3 vs. Hughes although the 3rd was not really relevant since both were past their prime. For us old guys who saw Penn in his prime, we understand the talent and shook our heads at his not being more disciplined in order to have a legacy comparable to GSP, Anderson Silva or Jon Jones. Penn was truly an amazing fighter in his era and I was not a Penn fan. I was a GSP fan, so I rooted for GSP big time in their two fights. Hard not to respect Penn's talent at his prime. His comments, lack of discipline and pissing and moaning when things didn't go his way were turn-offs for me.