These problems may be overstated. Referring to someone by a physical trait isn’t necessarily considered negative. African Americans are Blacks and Euro-Americans are Whites. But even in these contexts, these references to physical traits can be problematic, perhaps depending on what is meant or who uses the word. “Black is Beautiful” (you might have to be older to remember that phrase). One of my best friends calls herself a “Black Girl.” On the other hand, Donald Trump says he has “a great relationship with the Blacks” and when I’m in South Africa and I hear someone use the term “The Blacks” I become wary, because the next thing the person says may not be very nice. Still, it is not universally the case that a reference to physical features is negative. (I should also mention that in Francophone contexts, “Pygmy” is considered superior to another term often used, “Negrito.”)
The term “Pygmy” as used in biology is unfortunate and also causes confusion. I’ve met people who thought that “Pygmy hippos” were hippos that lived where Pygmies live, but ironically the pygmy hippos live in West Africa only and the Pygmy people live in Central Africa only. I don’t want to go into some of the confusion I’ve encountered between “Pygmy chimps” and Pygmy people. One way to solve this problem, of course, is to change the animal names. These are common names (though the word “Pygmy” is sometimes the root for the Latin binomial which is harder to change) and they can be discarded or altered. In fact, that has happened with the pygmy chimps, which are never called that any more for this reason (and the fact that they are not really smaller than regular chimps). You probably know of them as bonobos.
The real problem is probably the use of the word “pygmy” as a derogatory term, including its use as a slur of someone’s intelligence.
This sort of problem is widespread. There are many cases of people who’s “name” is somehow considered negative. One of the more interesting examples is the term used to refer to the traditionally foraging people of southern Africa, including the Ju/’hoansi and Baswara people, but also many other groups. The two terms of concern are “San” and “Bushmen.” The former is said to be a negative term applied to these folks by their cattle keeping neighbors who looked down on them. It may mean something like “wild.” It is part of the phrase “Khoisan” which relates to the term “Khoi Khoi.” Put simply, San are foragers and Khoi Khoi (or just Khoi) are herders, but otherwise the people referred to by these terms share languages and other aspects of their culture, and are physically similar. Khoisan (Khoi+San) refers to all of these folks together. (These terms are mainly South African and older, used in the 17th century and onwards and not used so much recently.) The term “Bushmen” also means “wild” and that is considered an inappropriate term as well.
During the 1980s, when various anthropologists were working with groups of San/Bushmen in Namibia and Botswana to help them organize politically, they (the San/Bushmen) of that area decided to call themselves “Bushmen.” There were two reasons for this. The primary reason is that the word “Bushmen” was widely known and they wanted to use a term that everyone would recognize as they strove for political and cultural recognition and improved status. The second reason is that by this time it was cool to be a Bushman. Bushmen are Beautiful, if you will. So Bushmen is the correct term in that area of Namibia and Botswana. But, to the south in South Africa, the term “Bushman” never caught on among the politically thoughtful. There, “San” was considered to be a better word. Many of these folks adopted the term “San” in South Africa, while their distant relatives to the north eschewed the term. And that’s the simple version of the story. There are many different cultural groups with different languages living in different parts of Namibia, Botswana, and South Africa who prefer their more local name such as Barwa or Sho or other terms.
So, in the case of the San/Bushmen (let your eyes settle on either of those words depending on if you are from Botswana or South Africa!) one part of the solution was to ask the people what they want. This has been done with the Pygmies to some extent. I did it myself. I have had this conversation with Efe people on many occasions. Efe is the term used, by themselves, for Pygmies that live in a certain part of the Ituri Forest in the Congo. The Efe are both the largest and smallest Pygmies. They live over a larger area of Africa than any other group, and there are probably more Efe than any other group of Pygmies. Since Pygmy stature (they are indeed short) is of such great interest, every group of Pygmies has been repeatedly measured by anthropologists. So, we know that the Efe are the shortest of the Pygmies. Largest (in area) and smallest (in stature).
A local term for the Efe and other Pygmies of the Ituri is “Bambuti” (Singular: Mbuti). The Pygmies that the famous anthropologist Colin Turnbull lived with and wrote about called themselves Mbuti, and they live to the south of the Efe, speak a different language, and have some other cultural differences. Having said that, I did some work in the area of overlap between Efe and Mbuti and the distinction isn’t perfectly abrupt or consistent. Anyway, the non-Efe people of the area often use the word “Bambuti” for the Efe (and the Mbuti) but some people consider “Bambuti” to be a little derogatory. But not everyone thinks that.
The Efe I’ve spoken to know the term Pygmée (the French for Pygmy) and they don’t regard it as derogatory. They never use that word to refer to themselves, however, which makes it inappropriate for local use. The problem is, though, that just like the Ju/’hoansi of Namibia and Botswana recognized, if we want to speak of these folks in a broader context–in classrooms or among those interested in human rights in the area, etc.–we are forced to use the word Pygmy so people have a clue what we are talking about. And, as far as I know, the Efe don’t particularly mind.
I suppose it is more important to be respectful than to speak respectfully.