The same thing happened to me, except I'm 100% Armenian, but my native language happens to be Russian, because it was the primarily spoken language in our family.
It's just xenophobia.
its more than that. I remember being bullied and called little jew by kids, by teachers, by school principal, by janitors, by strangers on the subway when I was 7 or 8 because I have soft Rs and curly hair, and I am not Jewish.