It took that long because Leonard wanted no part of Hagler. It was only after he was retired and wanted to make a comeback that it made sense for him. Possibly because Hagler began to seem more vulnerable. Hagler was chomping at the bit to fight Leonard for a long time.
How long did it take to make Leonard vs Hearns or Duran? Sure it takes time to iron out an agreement but when it takes years for a fight to come together it's usually because one fighter or team just doesn't want the fight. Same with Canelo vs GGG and Mayweather vs Pacquiao.