Additionally, the kind of person who would reach for prescription medication vs accepting hair loss may be predisposed to depression. I.e. this may be selecting for people who struggle with self-acceptance generally.

I also wonder whether there's some degree of placebo going on. Patients know finasteride is anti-androgenic; perhaps when they inevitably experience some symptoms associated with hypogonadism they assume the worst and lament the choice between having hair and feeling youthful. This would also explain why many who get off finasteride don't notice their symptoms improve.

Personal bias: I've taken finasteride for years with no side effects.