Physical prime means nothing if you’re not at your peak in terms of skill. When people say “prime X” they mean X at their best in a particular sport. This is why we say “prime X” and not “X in their 20s”.
That is a half-truth, when speaking about an athlete's prime, it is about that person's complete mental and physical ability. You can be the most fundamentally sound and skilled person in the world, it means nothing if your natural physical ability isn't there to back it up. In Silva's case, he was still an incredibly skilled counter-striker when he faced Weidman, however; he was slowing down by that point, his reflexes were slower, his chin wasn't as durable, his timing was off, he likely didn't have the same explosiveness that allowed him to pull off the same acrobatic strikes he was doing 6 years prior.
Right, that’s the point though. It’s a combination of mental and physical performance. So arguing that Silva may have been in his peak when he had those losses makes little sense because he subsequently improved. Cejudo was in his late 20s when he lost to DJ the first time but it’s obvious he wasn’t in his MMA prime yet.