Why are vegetarians so large vs carnivores any evolutionary reason for this?
Not getting eaten. As an herbivore you have a few main options.
A: Medium sized and fast (gazelles, antelope, zebra, wildebeest).
This allows you to outrun predators. But there are two problems with this route. In just about every single environment, the fastest carnivore evolved to become faster than the fastest herbivore (the American Cheetah went extinct, leaving the pronghorn uncontested, however). The other problem is that certain carnivores evolved pack mentality and exceptional endurance, so the prey may be able to escape, only to eventually tire and be caught (Lions, Wolves, Hyenas, Wild Dogs). These herbivores countered that by evolving a herd mentality, which makes them much stronger as a group, as well as by breeding somewhat prolifically.
B: Large and slow (hippos, elephants).
This is the safer option from a predator / prey standpoint, since carnivores are highly unlikely to evolve to these exceptional sizes due to the massive food requirement, which is more difficult to come by for a carnivore. Herbivores have no problem with this due to the crazy abundance of grass and other vegetation. Their numbers are kept in check with the predation of young, old, and injured animals (and humans being humans of course). Evolutionarily, their biggest problem is what happens to them when the climate changes and their food supply dwindles. They require a massive amount of calories, and when those calories dwindle, they die off.
C: Small and slow, but you can hide, as well as breed like a mother (rodents).
Apart from intelligence (potentially), this is likely to be the most successful evolutionary strategy of terrestrial mammals. Being small, hidden, and extremely prolific breeding-wise has proven to be an exceedingly good survival strategy. They're weak as hell and most predators can pick them off without any problem, but when one single female can give rise to hundreds, and potentially even thousands of the little fuckers every year, it's an acceptable trade-off.
Now do I think it's impossible for a carnivore to evolve to the size of an elephant? Of course not, since we already have examples of that happening in the past. But nature works by necessity. Right now there are billions of zebra, antelope, gazelle, deer, buffalo, et al, for carnivores to prey upon. They don't need to be able to eat elephants to survive, so growing to that size would not only be pointless—it would actually be detrimental since it would not only greatly increase their caloric requirements, but would likely make them slower or less agile, and thus less likely to be able to prey upon the aforementioned smaller herbivores which make up a far greater number of potential targets.
Caloric requirement is a very important factor in evolution. It's why crocodilians have survived for nearly a hundred million years, while all those huge dinosaurs died out. Being a big, warm-blooded meat-mountain is a serious detriment when the shit hits the fan and food supplies run low. Being stuck out on a prairie or savannah is as well. Crocs can take to the water and go an entire year without eating if need be. Now that's a beast built to survive.