However, if you actually talk to them, it isn't. What would you replace it with? Rom or Roma? In that case, you would be mislabeling a lot of them since the Rom are only one of the Gypsy groups (but it is the largest). You would also be pissing off all Romanians who absolutely, positively do not under any circumstances want to be confused with Gypsies.
Would you say "Roma and Sinti"? That's the two largest Gypsy groups (or is it the largest and third-largest?). There are many others.