The Inverted-face Effect

Source: Laboratory of Jonathan Flombaum—Johns Hopkins University

In perception, it is often the case that the ability to recognize and interpret complex stimuli feels effortless but actually demands complicated and intensive processing. This is because processing is specialized and automated for certain types of very important stimuli. Among the best examples of this phenomenon is face processing. People do not try to detect and recognize faces. It just seems to happen. However, detecting faces and telling them apart from one another is actually a demanding computational task.

Human facial recognition abilities rely on specialized computations and dedicated brain networks. One simple demonstration of this is the inverted-face effect. Recognizing upside-down faces is far more difficult than recognizing them right-side up, but the same is not true for many other kinds of visual objects. The inverted-face effect is demonstrated in a variety of ways. This video shows an incidental encoding memory paradigm for investigating facial processing and the inverted-face effect.

1. Equipment and Stimuli

1. This experiment requires a computer and experiment scripting software.
2. In addition, the experiment requires a relatively large set of facial images, preferably with similar lighting conditions and without emotional expressions. Many databases of such images are available freely online for research purposes. A good resource is the MIT face database: http://web.mit.edu/emeyers/www/face_databases.html#oulu

2. Design

1. Assemble a set of 80 facial images. Divide them in half, into two groups of 40. Use one group in the incidental exposure portion of the experiment, and reserve the other group to use as foils in the test portion.
   1. Note that your two groups should have the same relative proportions of males and females. In other words, if there are 25 male faces in one group of 40, there should be 25 in the other as well.
2. The incidental exposure program
   1. The first part of the experiment involves incidentally exposing a participant to a set of 40 faces through a cover task.
   2. The experiment starts with a 'Ready' screen. Pressing the spacebar begins the exposure.
   3. Once the spacebar is pressed, each of the 40 faces is shown one-at-a-time for 1 s. After each image, a screen that says "Male/Female?" appears. (The cover task for the participant is to report whether a face was male or female.)
   4. Set the program to advance to the next face after the participant presses either the M or F key, (corresponding to Male and Female). Figure 1 schematizes the sequence of events in the exposure phase of the experiment.
   5. This phase of the experiment is a total of 40 trials-one for each face in the set.

Figure 1. Methods for an incidental encoding memory paradigm designed to demonstrate the inverted face effect. The experiment has two parts. In the first part, called the incidental-encoding phase, participants observe a set of 40 faces, one by one, and are asked to simply report whether each face is male or female. In the second phase, the participant is given a surprise memory test. In each trial, two faces are shown side-by-side. One of each pair is one of the faces shown in the encoding phase, and the other, called the foil, is a new face, never seen before by the observer. The task is to use the right and left arrow keys to indicate which face in each pair is the one seen previously. Crucially, half the face pairs appear upside-down. The measure of interest is report accuracy for right-side up compared with upside-down faces. Please click here to view a larger version of this figure.

3. The test phase program
   1. The test part of the experiment also includes 40 trials.
   2. For each trial, randomly select one of the old faces and one of the previously reserved faces (of the same sex). Place one on the left side of the screen and one on the right.
   3. The task is for the participant to indicate with the left or right arrow key which of the faces in a pair was seen before.
   4. The crucial manipulation is that in half of the trials, the two faces are presented upside-down.
   5. Randomly intermix upside-down and right-side-up trials.

3. Running the Experiment

1. Before the experiment begins, it is a good idea to ask the participant if she or he has any known visual impairments or difficulties, and in particular, if they think they have any difficulty recognizing people.
2. Seat a participant 60 cm from the monitor of the presentation computer.
3. Explain the encoding phase to the participant as follows, without mentioning the test phase to come:
   1. "This is an experiment about face perception. In each trial, you will see a single face for one second, followed by a screen in which you will be asked to identify the sex of the face you just saw. Press the M key for male and the F key for female. The task may seem very easy, as we are conducting a study with several experiments, some including easy-to-identify stimuli and some with digitally-altered, more challenging stimuli. The results, taken together, will help us understand the facial features used to recognize the sex of a face. Whether it feels easy or hard to you, please do your best. There are only 40 trials, and it will take less than five minutes to complete."
4. Start the program and stand nearby as the participant completes this phase of the experiment.
5. When the participant is done, say the following:
   1. “Thank you for completing that portion of the study. I would like you now to do a second experiment, one that will also last only five minutes. In each trial, you will see two faces, one on the left side of the screen and one the right side. One of the faces in each pair will be one of the faces you just saw, and the other will be a new face, one that did not appear in the judgment task you just did. Your task now is to identify the ‘old’ face, the one that appeared in the judgment task. Use the left arrow key to indicate the face on the left, and the right arrow key for the face on the right. In some of the trials, the two faces are shown upside-down. This is irrelevant—your task is always to do your best to identify the face that you think you saw once already.”
   2. Stand by as the participant completes the memory test.

To analyze the results, simply compute the proportion of faces correctly identified by the participant in trials with upside-down (inverted) and trials with right-side up (upright) faces. Compare performance using a bar graph, as shown in Figure 2. For most visually normal observers, accuracy will be much higher with upright compared to inverted faces. However, this is a difficult task, and you may find performance below 0.9 even for upright faces. For inverted faces, performance may even approach chance, 0.5-what an observer would score if they just guessed on each trial. Poor performance with inverted faces shows that specialized computations and brain mechanisms used for recognizing faces are tuned to take advantage of the fact that faces are almost always experienced in an upright orientation.

Figure 2 Memory accuracy for inverted compared to upright faces. Most visual-normal observers show considerably better performance recalling previously seen faces when shown upright, as opposed to inverted. Indeed, with inverted faces, performance can be close to chance (0.5) in a task like this. Poor performance with inverted faces is the inverted-face effect.

The discovery that inverted faces are difficult to process has many applications. Neuroimaging studies, for example, have taken advantage of the effect to identify brain regions involved in specialized face processing. Brain scans are taken when observers view upright as well as inverted faces. The responses to the two kinds of stimuli are then compared. Both sets of stimuli have very similar visual properties overall, leading to similar activity throughout much of the visual system. In one brain area though, upright faces produce a much more vigorous response than inverted ones, suggesting that inverted faces fail to engage specialized face-processing neurons. The area that responds this way is called the fusiform gyrus, or the fusiform face area (sometimes FFA for short). This brain region is implicated in many other studies that investigate specialized aspects of face processing.

A second application has to do with a disorder known as prosopagnosia. This refers to extreme difficulty recognizing, at times even detecting faces. Prosopagnosia can arise following brain damage to the fusiform gyrus. But it is now known to also appear in people with no known cause of brain damage. One way that prosopagnosia is assessed involves the inverted-face effect. In particular, individuals with prosopagnosia don't show a typical inversion effect. Although they have no more trouble recognizing upside down faces than they do right side up ones, they have considerable trouble with right side up ones in general. This lack of an inverted face effect suggests that prosopagnosia is caused by the absence of specialized face processing systems-the kind that seem to know that faces are usually seen right side up.