Stylistically, Anderson is the worst match-up for Belfort - he is the perfect counterstriker, who can bob and weave out of the way of Belfort's blitz, and catch him coming in, time and time again.
True, if they did rematch, Anderson would be older/slower/coming off an injury, but his style would likely stay the same, thereby making Belfort the underdog.
Lastly, I agree that it is perhaps better for both Anderson and the fans, if he either retires, or spends the rest of his career fighting new opponents (I don't mean using him as a gatekeeper for new contenders) - no rematches, just exciting match-ups with guys like Le or even Nick Diaz.