Dude, I know the feeling. You feel like a fraud. You feel like their lower belts will wreck you (and they probably could). You want to train at the place where everyone is strong, because you know it will make you strong too.
Some call that ego. But I think it comes from a place of purity; you just want to be what you think you SHOULD be. You don't think it's right that you hold a superior position, yet are inferior in skill or knowledge.
But the ugly question is, what is BJJ for you? If it's your profession and how you'll make bank, then you're right. Shit is unacceptable. Go to Atos immediately. Your future is on the line.
But if it's not...if it's just a hobby? Then you may want to accept it and let it go. Accept that an Atos purple means something different compared to the local recreational gym purple. Shit, Tainan Dalpra's purple belt means more than my first degree black belt.
You may want to ask yourself why you started BJJ, and why you stuck with it. Was it to be the best? Or was it because BJJ was fun?
Also if you aren't going to be a competitor, you have a much larger window of BJJ activity than the window to get to hang with your kids. They're going to be doing their own shit and ignoring you before you know it. If you want to spend the window of time where your kids want to hang out with you trying to become a marginally better recreational grappler, then do as you wish.
BJJ culture is all fucked up. You'd never have this kind of dilemma in any other recreational sport. My fencing coach made no allusion otherwise that I was, as someone starting at 17, never going to be an olympic hopeful. I could probably reach mediocre success at best. In Judo, there's no push to be part of the "Judo lifestyle" the same way as in BJJ. I doubt frisbee golfers are out there wondering if they should sacrifice evenings with their kids to really start sinking those drives. Yet, here we all are.