I'd add legacy to the mix too. When I picture "the greatest of all time" I picture someone with a mystique whose achievements reached outside of the sport and had a long-term impact, and while mainstream popularity isn't a factor, I feel it waters down the "GOAT" moniker by giving it to someone the fans don't care for.
I also place some weight on a fighter's pre-prime losses. Losing on the regional level is forgivable since fighters are starting out, as are losses against the elite fighters, but there's something about those early UFC/ top level promotion losses that sticks with people. For me it's hard to buy Stipe as the greatest heavyweight of all time when he has an embarassing loss to Struve, or Local Woman getting stopped by the world's softest GnP against Alexis Davis.
The best example of this is Ayrton Senna in Formula 1. Even though there were drivers who had more wins and won more titles then Senna, Senna had a mystique that transitioned to the mainstream. He was as much an icon as he was a good driver, and because of it most rate him as the greatest over guys like Schumacher, Prost, Fangio etc. (although personally I'd choose Fangio because of how high-risk his era was.