There's the crux of your problem in a nutshell. Not all stereotypes are invidious, and many stereotypes are based on what we run into all the time in our everyday lives, and are a shorthand way of communicating general impressions.
If I were to say "Only hire Asians for IT positions," then that indeed is a form of racism [against everyone other than Asians], but to say, "in every math class I've ever taken, there are more Asians in that class than in the general population," how can that possibly be racist? It's an empirically true statement based on my limited and localized experience.
It's nice to be enthusiastic about a cause, yet you don't want to be such a stickler for purity that you drive away people who otherwise would be your natural allies.