I have two problems with hanging out with stupid people:
1) They are extraordinarily good at small talk. They often go on for several minutes basically repeating the exact same thought in about 100 different ways. It's painful.
2) If god forbid we actually get around to actually talking about something of substance, I've already considered every single point they make, but I still have to sit there listening to them repeat the same cliches. And there is no way I'm going to make an on-topic response to them and give them a chance to make a response to me that I've also already considered.
So ya, I prefer not to hang out with a ton of people since most people bore the ever loving christ out of me. I do enjoy hanging around intelligent people, but there are just fewer smart people than stupid people so it limits the pool of people who I can tolerate.
I imagine a genius level brain researcher would be pretty bored talking to me. Or be studying me rather than conversing with me.