At the turn of the 20th century, a German horse took Europe by storm. Clever Hans, as he was known, could seemingly perform all sorts of tricks previously limited to humans. He could add and subtract numbers, tell time and read a calendar, even spell out words and sentences—all by stamping out the answer with a hoof. “A” was one tap; “B” was two; 2+3 was five. He was an international sensation—and proof, many believed, that animals could be taught to reason as well as humans.
The problem was Clever Hans wasn’t really doing any of these things. As investigators later discovered, the horse had learned to provide the right answer by observing changes in his questioners’ posture, breathing, and facial expressions. If the questioner stood too far away, Hans would lose his abilities. His intelligence was only an illusion.
This story is used as a cautionary tale for AI researchers when evaluating the capabilities of their algorithms. A system isn’t always as intelligent as it seems. Take care to measure it properly.
But in her new book, Atlas of... see more