The issue is that it's unconscious for the most part. The mind works by creating cognitive models (schemas) of reality. You rarely see reality for what it is but rather you see an internal model which you've unconsciously superposed onto reality and which is so convincing that you mistake it for reality itself. When you meet someone for the first time, your mind immediately starts creating a schema of what that person is like or what you believe they're like. Creating new cognitive models is tiring, and the mind is always looking for ways to save cognitive bandwidth, so at subsequent meetings it wants to add onto the initial schema (the initial impression). The schema becomes more rich and complex, but it's not torn down and built back again upon subsequent meetings. This is done through selective attention, the mind discards what doesn't fit the schema and only notices what does fit.
So when you meet new people and your first impression game isn't on point, chances are whatever impression you leave them with will persist forever. There's also a name for this effect when dealing with news or information, the continued influence effect. This is when a piece of news is reported incorrectly or with a strong bias, this leads to a new schema which is self-reinforcing. Even when you tell people that this information was false or misleading, they acknowledge it consciously but their attitude and beliefs about that thing remain unchanged, again for cognitive bandwidth saving purposes the schema is resistant to being torn down and built again. The media can be a powerful manipulation tool because whichever version of the story is heard first will persist no matter how many times you tell people it's false.
