I think the most commonly held theory is that fur works on quadrupeds, but once you stand upright, it is less effective, and less fur works better. For later time periods, clothing works better than fur because it is more adaptable. Consider that whatever fur-based system human ancestors had was based on needs in the tropics where it does not get that cold, so it is not hard to imagine that clothing is much more effective.
Recent studies of body parasites suggest that body lice unique to humans differentiated genetically only fairly recently, in the range of several tens of thousands of years. This body lice requires clothing … human clothing on human bodies is the habitat for these lice. This suggests there may have been a reduction from a certain level of furriness only with modern humans living in a wide range of environments and using controlled fire, clothing, and some kind of shelter (hut/house) to deal with the elements. So it is possible that the immediate ancestors to modern humans (perhaps Homo erectus?) were actually fairly furry.
As for details of the body hair, this is also interesting. Why do humans have pubic hair but not a lot of other hair? Why to males have more body hair than females in many cases? Why to human males have facial hair? The African Apes have much less facial hair than most modern human males. It has been suggested that this has to do with sexual selection. It is important to distinguish between the idea that the starting condition is a lot of fur and that females may have lost more than males, vs. the starting condition was very little hair and males have added more. The amount of fur, it’s appearance, etc. may be related to testosterone (this is true in males and females but more obvious in males) so facial hair may be a signal of “quality” in males.