It's worse in terms of what?
It's worse in terms of ability to achieve your goals. For the passer that's to stay on top and eventually pass, for the guarder it's to not get passed and eventually sweep or submit. In general (there are exceptions, but not many), the passer has the mechanical advantage due to gravity, greater agility (since he's on his feet vs. the guarder on his back or butt), and greater ability to exert force (again, because he's on his feet). The guarder certainly can minimize these advantage by disturbing the passer's balance, breaking his posture, etc, but notice how all those goals presume that the passer has those good positional facets naturally. You rarely hear about breaking posture in guard because guard players have no postural advantage. You don't have to break balance because the guarder has no balance to speak of already being on his back. This disadvantage is exacerbated as you go higher up in weight due to declining strength/weight ratios, but it's there even among the little guys.
Oddly enough if you look at which divisions have multiple time champs who spent the most time on bottom and still won titles, it's actually middle heavy. This is mostly because of Braulio and Romulo's dominance and the fact that both have exceptional guards. But at the lower weights, typically thought of as double guard pull city, most of the really dominant champs won their semi- and finals matches on top. Celsinho, the Mendes bros (especially Gui), Bruno Malfacine, Robson Moura, Marcelo Garcia, Leandro Lo, all those guys played a ton of guard until the later stages where they seemed to switch mostly to the top game to win. My guess is that 's because they recognize that when dealing with inferior opponents it's most efficient to take them out of their games by pulling guard but when faced with opponents as good as they were (or nearly) it's too dangerous to pull and risk the pass.
Even though I don't really like chess analogies applied to BJJ, I'll use one here. In chess how you open the game is very important in determining the course of the action, and if you can force the game into a position you like you have a much better chance of winning. Many players, especially when playing black, will play tricky openings that aren't necessarily the best but take their opponents out of their comfort zones, and they pick up a lot of early wins this way. But once you start getting to master level, you basically have to play mainstream openings because you opponents will be good enough to refute your inferior strategy even if they don't see it that often. I think there's less of a line between top and bottom in BJJ than there may be between good and bad openings in chess, but you get the idea.