Drago would be a clear #1 if we're talking about villains. He was more than just a boxer, he was the collective will of a nation personified. His cold reaction to killing Creed was one of the great moments of the entire series, and set him up as the ultimate opponent.
Clubber brought the fire and intensity, and was not to be denied from his shot at the title. As over the top as he was, Clubber still was much more human than Drago. I remember that in an old Rocky videogame that you started out with prison fights with Clubber, showing how he came from rock bottom, having to make the very most of his extremely limited opportunities.
Tommy Gunn was easily the most hateable opponent Rocky faced, but he was too much of a chump to top either Drago or Clubber in any other way.
I don't see Apollo, Dixon, or Conlan as villains. Apollo was far too likable from the jump, his obvious Ali influences and Carl Weather's charisma making him a great foil for Rocky. I didn't want to see him get smashed into the ring like I did Drago, Lang or Gunn, it was more about seeing Rocky prove himself against the great champ. Dixon's story was as much a redemption story as Rocky's was in their movie, and Conlan never stepped into villain territory, aside from that quite intimidating ringwalk.