Fact is when Fedor wasn't completely washed up he was always in the orgs with the best HW talent at that respective time and the early to mid 2000s era in particular remains by far the best HW era
These are the top guys he fought in that time frame: Werdum, Sylvia, Arlovski, Rogers, Bigfoot, Hendo - losing to three of them.
These are the top guys he didn't fight, despite being in the same org: DC, Reem, Barnett, Sergei
These are the top UFC guys he didn't fight: JDS, Cain, Couture, Bork, Murr, Gonzaga, Carwin
He fought 6 out of 17 of the "top" HWs of that timeframe. Hendo wasn't a top HW and Rogers barely just made the top 10 after beating Arlovski, so in actuality 4 of 15, but we'll count those as top guys to be fair. Nog and Mirko fights were prior to the timeframe we're talking about, so I don't think they're applicable. I also don't think it should count against him either, so we'll leave them out. So, 6 out of 17 - that's just over a third - and he went 50% against them.
One big factor for me is that he didn't fight Reem, Barnett, or Sergei despite being in multiple orgs with them. Opting to fight Lindland and Hong Man Choi after PRIDE was acquired by the UFC is another big factor for me. He should've immediately went to the UFC. Fedor is undoubtedly great, but his resume is extremely overrated by his fans, and extremely padded with no names and freakshow fights. No other fighter considered in the GOAT talks (Silva, Jones, GSP) have even a fraction of the number of unranked and irrelevant opponents that Fedor does. The very first time in his career that he fought 4 top 10 guys in a row, he lost. Then he lost two more in a row after that. It speaks for itself in my opinion.