It probably won't, but it can. You can end up with a deviated septum (I have one, but it's barely noticeable until I start snoring) and you can end up with post-concussion syndrome (my brother got it). Most people never suffer any permanent consequences, but only you can decide if its worth the risk.
If you are only doing it for fitness, you don't need to spar and there's no risk. If you are doing it for self defence, you can limit your sparring and there is minimal risk. If you want to go pro and fight once a week for 20 years, there is significant risk. There's risk in every sport, so you decide if the risk in boxing is worth it for you.