Read My Gestures: Body language can trump facial expressions

Subtle cues in posture can confound attempts to read emotion from a face, neuroscientists have found. This opposes the prevailing idea that people infer each other’s emotions almost exclusively from facial cues.

Previous studies had shown that people quickly recognize emotions expressed on an isolated face or from a body topped with a blurred head. Scientists are now investigating how the face and the body, combined, affect people’s perception of emotion.

“We wanted to know to what extent [people] would be sensitive to any mismatch between the facial expression and the body,” explains Beatrice de Gelder of Tilburg University in the Netherlands. In the Nov. 8 Proceedings of the National Academy of Sciences, de Gelder and her colleagues describe their study, in which 12 participants were shown pictures of men and women and asked to immediately indicate the emotion on their faces by pressing buttons assigned to each emotion.

Some pictures were of just faces or just bodies, while others consisted of the four combinations possible with a frightened or angry face and a frightened or angry body stance. A frightened face was marked by high eyebrows, for instance, and an angry body had arms back and shoulders angled.

“We told [the participants] to only look at the faces and ignore all the rest,” notes

de Gelder. The researchers reported that when the volunteers were shown incongruent pairings of facial expressions and body postures—a frightened face with an angry body posture, for instance—their accuracy was 64 percent. In contrast, the participants were 81 percent accurate in their quick reads of angry faces shown without bodies.

The researchers also used an electroencephalogram to probe the participants’ brain activity. The machine detected unusually large voltage spikes in a face-recognition brain area when participants looked at images with mismatched cues. Similar neural responses are typical in people observing something unexpected, de Gelder explains. “If the message is congruent, [cognitive] processing goes through smoothly,” she says. “If there’s a mismatch, then there’s a red flag.”

This cognitive red flag and the participants’ poor accuracy in determining facial expression accompanied by incongruent body language reveal that people can be distracted by mixed signals, says de Gelder. “It means that our whole perceptual equipment is very finely tuned to the signals in our environment, even if our mind is occupied by something else,” in this case a face, she says.

“This is a very elegant study,” says Pawan Sinha of the Massachusetts Institute of Technology. “It shows that the analysis of body cues is tightly integrated with the processing of facial cues and that there may not even be a clear distinction between the two kinds of analysis.”