The Dark Past of Algorithms That Associate Appearance and Criminality

“Phrenology” has an old-fashioned ring to it. It sounds like it belongs in a history book, filed somewhere between bloodletting and velocipedes. We’d like to think that judging people’s worth based on the size and shape of their skulls is a practice that’s well behind us. However, phrenology is once again rearing its lumpy head, this time under the guise of technology.

In recent years, machine-learning algorithms have seen an explosion of uses, legitimate and shady. Several recent applications promise governments and private companies the power to glean all sorts of information from people’s appearances. Researchers from Stanford University built a “gaydar” algorithm that they say can tell straight and gay faces apart more accurately than people can. The researchers indicated that their motivation was to expose a potential privacy threat, but they also declared their results as consistent with the “prenatal hormone theory” that hypothesizes that fetal exposure to androgens helps determine sexual orientation; the researchers cite the much-contested claim that these hormone exposures would also result in gender-atypical faces.

Several startups claim to be able to use artificial intelligence...