So according to you, saying that "Trump's policies are harming minorities, women and LGBTQ+ persons" is okay, but saying that "Trump is a racist, sexist, transphobe" is not okay? Because all those things are true. Labels apply to people when they display behavior deserving of those labels.
Labels exist because they help us parse the world. Sometimes those labels are wrong and need to be updated. Sometimes a person may grow and change and old labels may no longer apply. Neither of those circumstances provide a compelling argument to stop using labels.
Your argument is like saying that since we can't really judge a book by its cover/title, we shouldn't have covers or titles at all. That's a nonsensical take. Imagine if you went into a library and none of the books had covers or titles. How would you make a decision about what book to buy? By reading the first page, or the summary? But those things are just longer labels. They don't accurately convey the nuance, depth or complexity of the story, just like outward actions don't accurately convey people's nuanced internal lives.
How would you talk about the book with someone else?
"So there's this book where the hero rides giant desert worms, you should totally read it!"
"Hey, I read that book about the hero who rides giant desert worms, it was awesome!"
You've gotten tangled in semantics, and left logic at the far shore. I hope you find your way back.
Saying "Trump is a racist, sexist, transphone" is shorthand for describing the actions he takes, actions that discriminate and promote the discrimination against vulnerable groups. Notice how almost no one was saying these things when he was merely a gaudy New York real estate tycoon. In fact, he was embraced by many progressives. It stands to reason then, that when we call him these things we are judging the actions he takes, not the character. Now, as I understand your claim, you say that this doesn't matter because Trump is obviously a terrible person, and any difference between judging his actions and his character is a practice in semantics. And for judging negative qualities, I'd agree: there isn't much difference in the short term. The problem, however, is that when we judge character, we also judge it in the positive, and that's where problems arise.
When we judge (prior to scandal) a Louis CK, a Shaun King, a Bill Cosby, an Amber Heard to be "a good person," there's a tendency to lionize them and make them our role models. We form relationships with some degree of dependency. Sure, you may see yourself as above this phenomenon, but nevertheless, it happens to many. What happens then, is that if and when the bad behavior is discovered, there's a tendency to excuse it and sometimes model it. Each of the people mentioned, and many other public people, retain a lot of their fans because of that emotional connection of them being "a good person" we should emulate. And some of their bad behavior gets ingrained into our culture and normalized. Once we have made a judgement of character, it colors our perception of their actions. Rather, if we see someone who seems very progressive, helpful, and empathetic, we an describe him just in that fashion: "He really listens to people; she really fights for civil rights." When we go further and say "She's such a great role model" or to some extent "He's a national treasure!" (the latter, a metaphor for the "goodness" of a person) we make an unnecessary, unfounded, and perhaps misguided judgement on the internal life of a person, something we have next to no information about.
I claim that there is little to no positive value to making character judgements of people we only know from afar, and quite a bit of potential negative value. Character judgements are best reserved for our family and intimate friends, where loyalty has social and moral value. For people we only know through the public, it's much more helpful to just describe their actions.