My wife is Brazilian. Besides the fact that we've been too broke to pay for airline tickets, she really hasn't wanted to go back (even though she hasn't seen her mother in over ten years) because she says wouldn't be able to relax traveling with me because there's absolutely no way I could blend in (too tall, too Scandinavian), and I'd be a walking target for thieves. She's from Rio.
We probably would have no problems as we'd take a lot of precautions (not driving at night, etc), but it just wouldn't be that much fun for her. Too stressful.
We'll end up going eventually when we have more money, but to those who don't know or have never been (and I used to live in Ecuador, where threats of violence/crime are very comparable to Brazil), the threat of violence in many South American countries is very, VERY real and is nothing to fuck around with.
I don't blame BJ at all, honestly. It's completely understandable.