He was beating a lot of top ranked middleweights, that's not can-crushing. Guys like Macklin and Murray who the division's no.1 (Sergio) had very tough fights with, even Rubio, Proksa, Curtis Stevens, Daniel Geale, and obviously Lemieux, those fighters were all either in the top 10 or just outside it when GGG annihilated them.
It's just that the division was pretty weak without many named fighters around until recently. Before Canelo being lineal middleweight champ, we had Cotto briefly and Sergio, and I don't think there's much doubt GGG would've walked through both those guys without any problems.