Well, I'll preface this by saying that I've never seen anything with either one of them (Carano or Rousey) more than one time, and I'm going by memory on the details. And I don't really make an effort to keep the nuances of Rousey and Carano performances secure in my memory.
The short answer would be that they've given about the same level of performance as far as I'm concerned, that being unacceptable and about equally so. But I think Carano has faced a higher degree of difficulty in delivering this low level. She has faced more unique challenges and burdens with her performances and hasn't sunk any lower than Rousey while doing them. I can't prove that Rousey would do worse in these situations, but I predict that she would sink to an even lower level in those situations. So if Carano gives (and has given) a D- performance in a given scene or situation, I am guessing that Rousey would probably show up and give an F performance. They've both risen to (and no higher than), in my opinion, the level of "not really hurting anything at the moment."
I guess I'll start with Carano...
She came in and got hand-held through her first movie by one of the top directors in Hollywood, who has directed Oscar worthy performances. She also had the huge benefit of delivering her scenes mostly with legitimate masters of acting, the whole way through. She wasn't trading lines with some dead fish. It was Ewan McGregor and Michael Fassbender, Bill Paxton and Michael Douglas. And I'm willing to bet Soderbergh paid more attention to her scenes and gave her more guidance and acting direction than he's had to do, or been willing to do, with anyone in decades.
And the result was that her performance was way, way worse than Carrie Henn in her debut role. Way, way worse than Edward Furlong.
How do I think Rousey would have done under the same circumstances? I think she'd also crumple under the burden of carrying a movie, also probably have problems looking at the camera with that many scenes and also have been a green screen for Fassbender and McGregor to act against. I think she would have been just as bad, but also much more annoying and hard to watch for that much time.
What I've seen from Carano since is very little improvement, and also more problems when she's faced with specific demands. I remember she had a phone acting scene in that shitty movie The Heist and the phone acting was about as bad as it gets.
She was okay at being ambience or imposing scenery in Deadpool, but didn't succeed at anything beyond that. When she had to talk or react to lines, it wasn't very good. And she was OKAY at being ambience / scenery. Keith Jardine is GOOD at being ambience / scenery.
I don't think Carano really brings anything to the table acting wise. Her performances are consistently bad and take me out of the movie and out of the scene. But I guess she isn't a truly annoying presence on the screen, which is...better than nothing.
As for Rousey, she has probably barely scraped her way up to "not hurting anything for the moment" and has delivered consistently bad, bad acting on screen. But she has also never had Steven Soderbergh walking her through things. The director of Expendables 3 was Patrick Hughes. Shrug. I'd say that having Stallone around should have been a guiding factor, but not on those fucking Expendables movies. Stallone is one of the best writers and directors in the business, but he's like BJ Penn, and all three Expendables movies were unmotivated Stallone.
That said, the demands on Rousey for Expendables 3 with her lines and reactions were pretty light, going by memory, and she fell short. And nothing else I've seen is any better.
Now, if she could maintain her shitty level of normal acting through a fake phone conversation, then she would be better than Carano was in that one regard. But do I think Rousey would fuck it up even worse? Probably.
As for stuff outside of movies, there was SNL and some god awful wrestling promos on the street (where I can't really blame her for being utter dog shit as there was possibly no script or director, and her acting companions were Marina Shafir and Shayna Baszler and such). I'd give that performance literally a zero out of ten, but I guess I can ignore it, even though it was done after all of her movies and SNL etc. And there was some terrible computer service company commercial from around when she started in the UFC. I guess I'll ignore that too.
So what have I seen from both? They both range from occasional moments of "C" level, all the way down to F. Maybe C minus at best for Rousey, and only for passing moments. But I think the worst moments I've seen on film from Carano were probably a little worse than the worst I've seen from Rousey.
I think if Rousey had been in all of the situations Carano has been in, she would have delivered even worse results, but I can't really prove it. I think if she had actually remade Road House and had to carry the film as the lead character, it would have been absolutely unbearable. Even as a second lead in that Wahlberg movie, it would have been unwatchable.
So...my best guess is that Carano is the slightly better actor. I mean, I can't really make an apples to apples comparison based on what I've seen. But if I had to bet on who would do better with any given role, I suppose I'd pick Carano.