Thanks. The nuance you've added with this comment makes a lot of sense to me, and it wasn't quite apparent in your original comment. But even then, I (or most American progressives) wouldn't consider it a major red flag or blocker for hiring.
I know it would be fine with a lot of sensible people, but a lot of people just lock in on what they perceive to be a problematic remark and stop listening. You must have experienced it yourself at many points where someone you talk to just took something you said and refuse to listen to anything else. This is just as prevalent among all political sides, just that you are familiar with the group you grow up with so you don't trigger their irrational reactions as much.
Now I've read a lot about American culture and politics online, but a more ignorant person with my experience would just bluntly say that as if it was common sense and as you saw you homed in on forcing people to speak English yourself. If you were a bit less understanding it would have ended the hiring process right there.
