Noting the conflicts, hardships, and burdens "multiculturalism" and "diversity" place on society is not "racist". Racist would be if you think that something is "wrong" with members of other races, if they don't have social outcomes like your group, or meet your group's standard of achievement, and then conclude that they need to be "fixed" so that they can be more like your group. That would be racist.