This work was supported by the Humanities Research Institute at Brock University in St. Catharines, Ontario, Canada and by the Cannon Research Center at Carolinas Rehabilitation in Charlotte, North Carolina, USA.

This protocol was approved by the institutional review boards at Brock University and at Carolinas Rehabilitation.
1. Prior to Testing
<ol>
	<li>Create 3 separate lists for the 15 film clips, each listing the clips in a different restricted order.</li>
	<li>Ensure that each list begins and ends with a neutral stimulus and that no two clips that target the same emotion occur consecutively.</li>
	<li>Create a separate folder on the desktop for each list of film order presentation and label the folder with the Order name (e.g., Order 1).</li>
	<li>Save the 15 clips into each of the three folders.</li>
	<li>Re-label each film clip so it reflects the presentation number and gives no clues regarding the target emotion (i.e., 01; 02; 03; etc.).</li>
	<li>Create six randomized orders of task (DANVA-Faces, DANVA-Voices, Film Clips) presentation (see Figure 1).</li>
	<li>Assign the incoming participant to one of the three restricted orders of film clips presentation.</li>
	<li>Assign the incoming participant to one of the six randomized orders of task presentation.</li>
</ol>
2. Day of Testing
<ol>
	<li>Bring participant(s) into the lab and seat them comfortably at the table.</li>
	<li>Review the consent form with the participant by reading each section with him/her.</li>
	<li>Answer any questions the participant may have about the consent form and have him/her sign it.</li>
	<li>Have participants complete the brief demographic questionnaire (Date of birth; gender). Have participants with TBI additionally complete the section of the questionnaire on relevant medical history questionnaire (date of injury, cause of injury, injury severity). 
	NOTE: When participants are completing the questionnaire, inquire about any known visual processing difficulties so you may seat participants with TBI accordingly</li>
	<li>Position the participant in front of the computer in a chair. Ensure that all participants (if more than one) have a clear view of the screen.</li>
</ol>
3. DANVA-Faces Task
<ol>
	<li>Give each participant a clip board with the DANVA-Faces response sheet attached and a pen to circle responses.</li>
	<li>Provide the participant the following instructions: &#34;After you see each item, circle your response (happy, sad, angry, fearful, or neutral) on the line that corresponds to the item number shown on each slide.&#34; 
	NOTE: If participants with TBI have indicated visual processing difficulties or fine motor difficulties are evident, provide the participant with the alternate response page. This page lists the five responses in larger text in landscape format.</li>
	<li>Open the DANVA-Faces task in a presentation and play it using the Slide Show View.</li>
	<li>Give participants the following instructions: &#34;For this activity, I am going to show you some peoples&#39; faces and I want you to tell me how they feel. I want you to tell me if they are happy, sad, angry or fearful. Fearful is the same thing as afraid or scared. If you are unsure of the emotion, choose the &#39;I don&#39;t know&#39; response. There are 24 faces altogether. Each face will be on the screen for five seconds. You must answer before the face disappears for your answer to count. Indicate your answer by circling it on the sheet in front of you&#34;. 
	NOTE: If the participant is unable to circle his/her own responses and is participating in in the study as part of a small group, provide the following instruction: &#34;Indicate your answer by pointing to it on the sheet in front of you&#34;. The examiner will then record the participant&#39;s response on the DANVA-Faces response sheet.</li>
	<li>Ensure the participants do not have any questions before starting.</li>
	<li>Start the task with the two practice trials. Give participants the following instructions: &#34;We are going to complete two practice trials so that you get a sense of how long each face will appear on the screen, and how long you have to provide your answer&#34;.</li>
	<li>Hit enter on the keyboard when the face disappears to move to the next stimulus. Ensure all participants are looking up at the screen before doing so.</li>
	<li>Ensure that the participants do not have questions. If none, begin the test trials. Ensure all participants are looking at the screen. Hit enter to start the test stimuli.</li>
	<li>Continue the procedure outlined for the practice tasks until the 24 stimuli are complete.</li>
	<li>Collect the participant response sheets.</li>
</ol>
4. DANVA-Voices Task
<ol>
	<li>Give each participant a clip board with the DANVA-Voices response sheet attached and a pen to circle responses. 
	NOTE: If participants with TBI have indicated visual processing difficulties or fine motor difficulties are evident, provide the participant with the alternate response page. This page lists the five responses in larger text in landscape format.</li>
	<li>Open the DANVA-Voices task by going to the following website: 
	http://www.psychology.emory.edu/clinical/interpersonal/danva.htm</li>
	<li>Click on the link for Adult faces, voices, and postures.</li>
	<li>Fill in the login and password by entering any letter for the login and typing EMORYDANVA2 for the password.</li>
	<li>Click Continue.</li>
	<li>Click on Voices (green circle in the center of the screen).</li>
	<li>Test the volume level. Provide the following instructions: &#34;I want to be sure that you can hear the sound on the computer and that it is at a comfortable volume for you. I am going to click test sound. Please tell me if you can hear the sound, and also if you need the sound to be louder or more quiet to be comfortable for you.&#34;</li>
	<li>Click test sound.</li>
	<li>Adjust the volume according to the participant&#39;s request (increase/decrease).</li>
	<li>Click &#39;test sound&#39; again. Adjust the volume according to the participant&#39;s request. Do this until the participant reports it is at a comfortable volume.</li>
	<li>Review the task instructions with the participant. Give participants the following instructions: For this activity, you are going to hear someone say the sentence &#39;I&#39;m going out of the room now, but I&#39;ll be back later.&#39; I want you to tell me if they are happy, sad, angry or fearful. Fearful is the same thing as afraid or scared. If you are unsure of the emotion, choose the &#39;I don&#39;t know&#39; response. There are 24 sentences. Before each sentence is spoken, a number will be announced. You need to listen to the sentence that follows. The sentences will not be repeated so you need to listen carefully. Tell me how the person is feeling by circling your answer on the sheet in front of you&#34;. Once you have made your selection, I need to select a response on the computer screen to move to the next item. I will always select the same emotion (fearful). My selection does not indicate the correct answer in anyway.&#34; 
	NOTE: If the participant is unable to circle his/her own responses and is participating in in the study as part of a small group, provide the following instruction: &#34;Indicate your answer by pointing to it on the sheet in front of you&#34;. The examiner will then record the participant&#39;s response on the DANVA-Voices response sheet.</li>
	<li>Click &#39;continue&#39;.</li>
	<li>Direct participants to circle (or point to) their answer on their response sheet when the sentence has played.</li>
	<li>Click &#39;fearful&#39;. Click &#39;next&#39;.</li>
	<li>Continue this procedure until the 24 sentences are complete.</li>
	<li>Exit the website when you reach the end of the task.</li>
	<li>Collect the participants&#39; response sheets.</li>
</ol>
5. Film Clip Task
<ol>
	<li>Seat participants comfortably in front of the computer. Ensure that the screen can be fully seen by all.</li>
	<li>Give each person a clipboard with a copy of the Emotional Film Clip Response sheet. 
	NOTE: If participants with TBI have indicated visual processing difficulties or fine motor difficulties are evident, provide the participant with two alternate response pages. The first page lists the five forced-choice responses and the second provides the 0-9 scale. Both response pages are in larger text and in landscape format.</li>
	<li>Open the folder on the computer for the assigned order of film clip presentation.</li>
	<li>Provide participants with the following directions: &#34;You are going to view a total of 15 film clips. In each film clip, you are going to see a character who looks like they are feeling a certain emotion. They might look &#39;happy&#39;, &#39;sad&#39;, &#39;angry&#39;, &#39;fearful&#39;, or &#39;neutral&#39;. First, I want you to tell me what emotion the character is showing. Then tell me what emotion you experienced while watching the film clip. Finally, tell me the number that best describes the intensity of the emotion you felt while watching the clip. For example, if the person&#39;s face made you feel &#39;mildly&#39; happy or sad, circle a number ranging from 1 to 3. If you felt &#39;moderately&#39; happy or sad, circle a number ranging from 4 to 6. If you felt &#39;very&#39; happy or sad, circle a number ranging from 7 to 9. Only circle 0 if you did not feel any emotion at all. Do you have any questions?&#34; 
	NOTE: Only the first question (Tell me what emotion the character is showing) was analyzed for the current study.</li>
	<li>Double click 01 to open the first film clip.</li>
	<li>Tell the participant which character to focus on while viewing the clip.</li>
	<li>Select &#39;View Full Screen&#39; in the options menu to play the clip.</li>
	<li>Direct the participants to respond to line a (What emotion is the main character experiencing?) on their response sheet when the clip has finished playing.</li>
	<li>Direct the participants to respond to line b (Choose one emotion that best describes how you felt while watching the clip) on the response sheet once line a has been answered.</li>
	<li>Direct the participant to respond to line c (How would you rate the intensity of the emotion you experienced while watching the clip?) once line b has been answered.</li>
	<li>Direct the participant to re-direct his/her attention to the computer screen once the responses are complete.</li>
	<li>Return to the folder on the computer that contains the film clips in the assigned order for the participant.</li>
	<li>Double cck on 02 to open the second film clip. Repeat Steps 5.5 to 5.12 until the participant has viewed all 15 film clips.</li>
	<li>Collect the response sheet from the participant(s).</li>
</ol>
6. Moving from One Task to Another (Optional Step)
<ol>
	<li>Provide participants the option to take a break between Tasks 2 and 3.</li>
</ol>
7. Scoring the Tasks
<ol>
	<li>Refer to the answer key on page 25 of the DANVA2 manual32 to score the DANVA-Faces task.</li>
	<li>Score items that match the answer key as 1 and incorrect items as 0. &#39;I don&#39;t know&#39; responses are scored as 0.</li>
	<li>Calculate a raw score by adding the total number of correct responses. Divide the raw score by 24 (total items) to obtain a percentage accuracy score.</li>
	<li>Refer to the answer key on page 27 of the DANVA2 manual32 and repeat steps 7.2 and 7.3 to score the DANVA-Voices task.</li>
	<li>Refer to the answer key for assigned order of film clip presentation for the participant. Score a 1 if the participant identified the target emotion when responding to the question about what emotion the character was showing, and 0 if the participant did not select the target emotion (including if the participant said &#39;I don&#39;t know&#39;).</li>
	<li>Calculate a raw score by adding the number of correct responses. Divide the raw score by 15 (total items) to obtain a percentage accuracy score.</li>
	<li>Create a subset score for the Film Clip task that contains only responses to the happy, sad, angry, and fearful film clips by adding correct responses to only these 12 clips. Divide the number of correct responses by 12 to obtain a percentage accuracy score.</li>
	<li>Create a score for responses to neutral film clips by adding the number of correct responses to the neutral clips. Divide the number of correct responses by 3 to obtain a percentage accuracy score. 
	NOTE: A subset score needs to be created because the DANVA tasks did not include portrayals of Neutral while the Film Clips task did.</li>
	<li>Calculate a score for responses to positively valenced items (i.e., happy) and negatively valenced items (combine responses to angry, fearful and sad stimuli) for all three tasks.</li>
	<li>Divide the score for positively valenced items for each task by 3 to obtain percentage accuracy score.</li>
	<li>Divide the score for negatively valenced items for each task by 9 to obtain percentage accuracy scores.</li>
</ol>
8. Data Analysis
<ol>
	<li>Conduct a mixed model analyses of variance (ANOVA) to examine responses to isolated facial emotion stimuli, isolated vocal emotion stimuli, and multimodal film clip stimuli by participants with TBI and Control participants.</li>
	<li>Explore main effects found in the ANOVA using follow-up univariate comparisons.</li>
</ol>

The author has nothing to disclose.

The manuscript describes three tasks used to assess emotion recognition abilities of people with TBI. The goal of the described method was to compare response accuracy for facial and vocal emotional cues in isolation, to perception of these cues occurring simultaneously within meaningful situational context. Film clips were included in the current study because their approximation of everyday situations made them more ecologically valid than isolated representations of emotion expressions. When carrying out this protocol with people with brain injury, it is important to consider the individual&#39;s visual scanning skills and ensure that the stimuli and response sheets are fully visible. To assist with this, response sheets were created that consisted of one line for each stimulus, and used alternated color coding for each line (e.g., Line 1 = gray; Line 2 = white; Line 3 = gray; etc.).
The DANVA2 is fully available on-line. However, the on-line version of the DANVA-Faces was not used because it is pre-set to have each face shown for only two seconds. Given the speed of processing issues often experienced by people with TBI, we were concerned that errors made on this version may be more indicative of speed of processing problems, than reflective of facial emotion recognition. Our presentation version of the DANVA-Faces increases the time each stimulus is shown to 5 seconds.
The stimuli used in the current study provided important insight into the emotion recognition abilities of people with TBI. However, there are many ways in which the stimuli may be improved for use in future studies of this nature. The stimuli used to assess recognition of isolated facial emotion expressions were from the DANVA-Faces task of the DANVA2. The benefit of using the DANVA-Faces stimuli is that they are standardized and used previously with people with TBI3,34. The 24 items include both high and low intensity portrayals of emotion by a small group of speakers so participants are exposed to the same speaker producing variable expressions. In addition, the DANVA-Faces task includes colored photographs versus the black and white images often used. Not only are colored images more contemporary, McDonald and Saunders15 argue they are likely less abstract to people with TBI. The primary issue with the stimuli used in the DANVA-Faces task, is that the emotion expressions are static. As previously discussed, static images provide people more time to interpret the emotion being portrayed, lack the temporal cues that occur when the face is moving12,13,15, and are not processed in the same area of the brain as dynamic ones17-23. Thus, using only static facial emotion expressions when assessing emotion recognition does not allow for appropriate generalization to real-life processing. In addition, research has shown that we can identify emotions in photographs more accurately than moving images, so tests using static images may overestimate abilities13,35,36. Therefore, future studies need to include isolated dynamic facial emotion stimuli to more accurately assess how people with TBI interpret emotion portrayed in only the face. These stimuli could potentially be created using the film clips by selecting frames in which the targeted emotion was portrayed, deleting the sound, and cropping the film to focus on only the head and neck of the character. This would eliminate additional contextual cues, including gestural cues and body language.
The vocal emotion expressions used in the current study were also taken from the DANVA2. Similar to the DANVA-Faces task, the DANVA-Voices task includes 24 items of both high and low intensity. Each item on the DANVA-Voices uses the same semantically neutral sentence (I&#39;m going out of the room now, and I&#39;ll be back later). The repetition of a single sentence allows listeners to focus on the non-verbal emotion cues (i.e., tone of voice) and ignore the verbal content. This is in contrast to stimuli used in previous studies that included different (semantically neutral) verbal content for each item15. Unlike the results of the current study, McDonald and Saunders15 reported that people with TBI had significant difficulty identifying emotion in isolated vocal emotion expressions when compared to isolated facial emotion expressions. The verbal content of the vocal stimuli used in that study varied for each item, so it is possible that participants were focusing on interpreting the verbal content, which was semantically neutral, and thus, meaningless. This assertion is supported by studies that have shown that people with (and without) TBI are more likely to use verbal content versus the non-verbal acoustic cues to interpret a vocal emotion expression37-39. Future studies should further explore the degree to which people with TBI make use of the nonverbal versus verbal content in isolated vocal expressions to determine whether the semantic information conveyed in the verbal message is more dominant than the emotion conveyed through tone of voice. This could be accomplished by creating stimuli with a limited number of sentences, both meaningful and neutral in verbal content. These sentences could then be combined with either congruent or incongruent nonverbal vocal cues. Sentences that are congruent in verbal and nonverbal emotion content are likely to lead to faster and more accurate identification of the emotion expressed40. However, it is unclear which cue participants will focus on when the verbal and non-verbal content is incongruent, and whether one channel would dominate throughout the task, or if the dominant cue will be dependent on the emotion portrayed through each channel.
Stimuli in the DANVA2 were created by presenting speakers with a short vignette aimed to elicit a target emotion and then instructing those speakers to express that emotion via the face or voice. It has been suggested that such methods lead to exaggerated and stereotypical expressions that may lead to overstated recognition abilities41. The film clips on the other hand included facial and vocal expressions that occurred with situational context, making them more ecologically valid than the use of isolated expressions. It is important to note that while not semantically neutral, the verbal content did not specify the emotional state of the characters in any way. The use of film clips of this nature with people with TBI is novel. While results of the current study suggest that the additional contextual information included in the film clips greatly facilitated emotion recognition, it is possible that participants were influenced by the additional cues present (e.g., body posture) and/or the cinematic design of the clips (e.g., pacing, lighting, and framing)42,43. Participants may also have been influenced by the presence of background music, so results need to be interpreted and generalized carefully. The influence of affective background music should be further investigated in future studies by comparing film clips without and without this cue to ascertain if the presence of congruent background music facilitates emotion recognition. While this information would not necessarily be generalizable to everyday social interactions, it could have important implications for treatment of emotion recognition deficits. It is also possible that participants in the current study were relying on the body posture and gestures of the characters in the film clips to facilitate emotion perception. Again, finding film clips that manipulate the presence and absence of such cues would be important for exploring which cues people with TBI make most use of when interpreting emotions in everyday situations. However, identifying commercial film clips that manipulate each of these extraneous factors to varying degrees poses a significant challenge. Instead, future research might focus on creating high quality film clips within various situational settings show characters engaged in meaningful interactions. These clips could be controlled for the presence and absence of contextual cues, postures, gestures, and background music. In addition, the clips could be used to assess recognition of dynamic isolated facial expressions as described earlier, recognition of isolated vocal emotion expressions, these two cues combined, and then the addition of contextual information. This would provide much needed insight into how people with TBI perceive emotion and might allow for better generalization to real-life settings.
Other methodological considerations for future studies like the one presented here are the number of target emotions and the response format chosen. The current study included four primary emotions: happy, sad, angry, and fearful. Neutral was also evaluated in the film clips task. This selection of target emotions includes only one positive emotion (i.e., happy) making it difficult to compare response accuracy for positive versus negative emotions. Also, such a limited set of emotions makes it easier for participants to rely upon process of elimination when making their selections, a strategy that could not be reliably used in everyday interactions. Future studies should expand upon the number of emotions included and should also include isolated facial and vocal expressions portraying neutral. Further, angry film clips were not well identified by either group, suggesting that the film clips themselves may have been poorly chosen. The average identification rate of the angry film clips in the previous validation study by Zupan and BabbageZupan, B. &#38; Babbage, D. R. Emotion elicitation stimuli from film clips and narrative text. Manuscript submitted to J Soc Psyc was only 73% compared to 85 - 90% for the remaining four categories. Previous studies using film clips also report difficulty selecting angry clips33,44,45. It remains unclear why it is more challenging to select film clips that lead to high identification of angry. It may be because angry is a more complicated emotion that serves a range of social functions (e.g., expressing dislike versus asserting authority), requiring the perceiver to evaluate a situation in terms of its relevance to their own belief systems46.
In addition, future studies should consider whether the valence and intensity of the film clip shown influences participant responses, particularly for neutral clips. While the current study showed no effect of the order in which the 15 film clips were shown to participants, it is still possible that some clips may have led to greater feelings of empathy, resulting in a carry-over effect when the next clip was played. While we did have participants complete a questionnaire to evaluate their general capacity for empathy (see Neumann, Zupan, Malec and Hammond47) We did not specifically ask participants to indicate whether or not they emphasized with the characters, and if so, to what degree. These questions may provide greater insight into the why our results indicated participants with TBI were more likely to provide negative responses to neutral stimuli.
The current study used a forced-choice response format, a commonly used paradigm in emotion recognition studies. IT is possible that the limited options constrained participants to select a single emotion when they perceived the presence of multiple emotions48. Additionally, they may have selected an emotion they did not perceive rather than choose the alternative response (i.e., I don&#39;t know). Using a free-choice response format with people with severe traumatic brain injury could result in additional and/or more problematic issues (e.g., word generation difficulties; long streams of explanations versus a single word). Therefore, it might be best to first adopt a forced-choice format that allows an additional free-choice word or comment to be added before moving onto the next item.
The use of film clips in the current study was a novel approach to assessing emotion recognition by people with TBI. It was hypothesized that people with TBI would find the multimodal nature of the film clips more complex, and thus more challenging than isolated facial and vocal cues. However, results suggested that rather than increase processing demands, the presence of numerous affective cues, including contextual information, facilitated processing. Yet, in everyday situations where a multitude of cues are presumably present, emotion recognition is a significant issue for people with TBI. Thus, while the film clips are more ecologically valid than isolated emotion expressions, they are not fully representative of the cues available in everyday social interactions. It could be that acting as an observer versus a participant allows for better integration of the available emotion cues.

Traumatic brain injury (TBI) affects approximately 10 million people each year across the world 1. Following TBI, the ability to recognize emotion in others using nonverbal cues such as facial and vocal expressions (i.e., tone of voice), and contextual cues, is often significantly compromised 2-7. Since successful interpersonal interactions and quality relationships are at least partially dependent on accurate interpretation of others&#39; emotions7-10, it is not surprising that difficulty with emotion recognition has been reported to contribute to the poor social outcomes commonly reported following TBI3,8,11,12.
Studies investigating emotion recognition by people with TBI have tended to focus on isolated cues, particularly recognition of facial emotion expressions using static photographs2. While this work has been important in informing the development of treatment programs, it does not adequately represent one&#39;s ability to recognize and interpret nonverbal cues of emotion in everyday interactions. Not only do static images provide increased time to interpret the emotion portrayed, they also typically only portray the apex (i.e., maximum representation) of the expressions13-16. Additionally, static images lack the temporal cues that occur with movement12,13,15, and are not representative of the quickly changing facial expressions of emotion encountered in everyday situations. Further, there is evidence to indicate that static and dynamic visual expressions are processed in different areas of the brain, with more accurate responses to dynamic stimuli17-24.
Vocal emotion recognition by people with TBI has also been studied, both with and without meaningful verbal content. It has been suggested that the presence of verbal content increases perceptual demands because accurate interpretation requires simultaneous processing of the nonverbal tone of voice with the semantic content included in the verbal message25. While many studies have shown improved recognition of vocals affect in the absence of meaningful verbal content, Dimoska et al. 25 found no difference in performance. They argue that people with TBI have a bias toward semantic content in affective sentences; so even when the content is semantically neutral or eliminated, they continue to focus on the verbal information in the sentence. Thus, their results did not clearly show whether meaningful verbal content helps or hinders vocal emotion expression.
In addition to the meaningful verbal content that can accompany the nonverbal vocal cues, context is also provided through the social situation itself. Situational context provides background information about the circumstances under which the emotion expression was produced and can thus greatly influence the interpretation of the expression. When evaluating how someone is feeling, we use context to determine if the situation is consistent with that person&#39;s wants and expectations. This ultimately affects how someone feels, so knowledge of the situational context should result in more accurate inferencing of others&#39; emotion26. The ability to do this is referred to as theory of mind, a construct found to be significantly impaired following TBI6,27-31. Milders et al. 8,31 report that people with TBI have significant difficulty making accurate judgments about and understanding the intentions and feelings of characters in brief stories.
While these studies have highlighted deficits in nonverbal and situational emotion processing, it is difficult to generalize the results to everyday social interactions where these cues occur alongside one another and within a social context. To better understand how people with TBI interpret emotion in everyday social interactions, we need to use stimuli that are multimodal in nature. Only two previous studies have included multimodal stimuli as part of their investigation into how people with TBI process emotion cues. McDonald and Saunders15 and Williams and Wood16 extracted stimuli from the Emotion Evaluation Test (EET), which is part of The&#160;Awareness of Social Inference Test26. The EET consists of videotaped vignettes that show male and female actors portraying nonverbal emotion cues while engaging in conversation. The verbal content in the conversations is semantically neutral; cues regarding how the actors are feeling and contextual information are not provided. Thus, the need to process and integrate meaningful verbal content while simultaneously doing this with the nonverbal facial and vocal cues was eliminated. Results of these studies indicated that people with TBI were significantly less accurate than controls in their ability to identify emotion from multimodal stimuli. Since neither study considered the influence of semantic or contextual information on interpretation of emotion expressions, it remains unclear whether the addition of this type of information would facilitate processing because of increased intersensory redundancy or negatively affect perception because of increased cognitive demands.
This article outlines a set of tasks used to compare perception of facial and vocal emotional cues in isolation, and perception of these cues occurring simultaneously within meaningful situational context. The isolations tasks are part of a larger test of nonverbal emotion recognition &#8212; The Diagnostic Analysis of Nonverbal Accuracy 2 (DANVA2)32. In this protocol, we used the Adult Faces (DANVA-Faces) and Adult Paralanguage (DANVA-Voices) subtests. Each of these subtests includes 24 stimuli, depicting six representations each of happy, sad, angry, and fearful emotions. The DANVA2 requires participants to identify the emotion portrayed using a forced-choice format (4 choices). Here, a fifth option (I don&#39;t know) is provided for both subtests. The amount of time each facial expression was shown was increased from two seconds to five seconds to ensure that we were assessing affect recognition and not speed of processing. We did not alter the adult paralanguage subtest beyond adding the additional choice in the forced-choice response format. Participants heard each sentence one time only. Feedback was not provided in either task.
Film clips were used to assess perception of facial and vocal emotion cues occurring simultaneously within situational context. While the verbal content within these clips did not explicitly state what the characters in the clips were feeling, it was meaningful to the situational context. Fifteen clips extracted from commercial movies and television series were included in the study, each ranging from 45 to 103 (mean = 71.87) seconds. Feedback was not provided to participants during this task. There were three clips representing each of the following emotion categories: happy, sad, angry, fearful, neutral. These clips were chosen based on results of a study conducted with young adults (n = 70)&#160;Zupan, B. &#38; Babbage, D. R. Emotion elicitation stimuli from film clips and narrative text. Manuscript submitted to J Soc Psyc. In that study, participants were presented with six film clips for each of the target emotion categories, and six neutral film clips, ranging from 24 to 127 seconds (mean = 77.3 seconds). While the goal was to include relatively short film clips, each clip chosen for inclusion needed to have sufficient contextual information that it could be understood by viewers without having additional knowledge of the film from which it was taken33.
The clips selected for the current study had been correctly identified in the Zupan and Babbage study Emotion elicitation stimuli from film clips and narrative text, cited above as the target emotion between 83 and 89% for happy, 86 - 93% for sad, 63 - 93% for angry, 77 - 96% for fearful, and 81 - 87% for neutral. While having the same number of exemplars of each emotion category as was in the DANVA2 tasks (n = 6) would have been ideal, we opted for only three due to the increased length of the film clip stimuli compared to stimuli in the DANVA-Faces and DANVA-Voices tasks.
Data from these tasks were collected from 60 participants with TBI and 60 age and gender matched controls. Participants were seen either individually or in small groups (max = 3) and the order of the three tasks was randomized across testing sessions.

<table><tbody><tr><td>Diagnostic Analysis of Nonverbal Accuracy-2</td><td>Department of Pychology, Emory University. Atlanta, GA</td><td></td><td>DANVA-Faces subtest, DANVA-Voices subtest</td></tr><tr><td>Computer</td><td>Apple iMac Desktop, 27&quot; display</td><td></td><td></td></tr><tr><td>Statistical Analysis Software</td><td>SPSS</td><td></td><td>University Licensed software for data analysis</td></tr><tr><td>Happy Film Clip 1</td><td>Sweet Home Alabama, D&amp;amp;D Films, 2002, Director: A Tennant</td><td></td><td>A man surprises his girlfriend by proposing in a jewelry store</td></tr><tr><td>Happy Film Clip 2</td><td>Wedding Crashers, New Line Cinema, 2005, Director: D. Dobkin</td><td></td><td>A couple is sitting on the beach and flirting while playing a hand game</td></tr><tr><td>Happy Film Clip 3</td><td>Reme-mber the Titans, Jerry Bruckheimer Films, 2000, Director: B. Yakin</td><td></td><td>An African American football coach and his family are accepted by their community when the school team wins the football championship</td></tr><tr><td>Sad Film Clip 1</td><td>Grey&#039;s Anatomy, ABC, 2006, Director: P. Horton</td><td></td><td>A father is only able to communicate with his family using his eyes.&amp;nbsp;</td></tr><tr><td>Sad Film Clip 2</td><td>Armageddon, Touchstone Pictures, 1998, Director: M. Bay</td><td></td><td>A daughter is saying goodbye to her father who is in space on a dangerous mission</td></tr><tr><td>Sad Film Clip 3</td><td>Grey&#039;s Anatomy, ABC, 2006, Director: M. Tinker</td><td></td><td>A woman is heartbroken her fiance has died</td></tr><tr><td>Angry Film Clip 1</td><td>Anne of Green Gables, Canadian Broadcast Corporation, 1985, Director: K. Sullivan</td><td></td><td>An older woman speaks openly about a child&#039;s physical appearance in front of her</td></tr><tr><td>Angry Film Clip 2</td><td>Enough, Columbia Pictures, 2002, Director: M. Apted</td><td></td><td>A wife confronts her husband about an affair when she smells another woman&#039;s perfume on his clothing</td></tr><tr><td>Angry Film Clip 3</td><td>Pretty Woman, Touchstone Pictures, 1990, Director: G. Marshall</td><td></td><td>A call girl attempts to purchase clothing in an expensive boutique and is turned away</td></tr><tr><td>Fearful Film Clip 1</td><td>Blood Diamond, Warner Bros Pictures, 2006, Director: E. Zwick</td><td></td><td>Numerous vehicles carrying militia approach a man and his son while they are out walking</td></tr><tr><td>Fearful Film Clip 2</td><td>The Life Before Her Eyes, 2929 Productions, 2007, Directors: V. Perelman</td><td></td><td>Two teenaged gilrs are talking in the school bathroom when they hear gunshots that continue to get closer</td></tr><tr><td>Fearful Film Clip 3</td><td>The Sixth Sense, Barry Mendel Productions, 1999, Director: M. N. Shyamalan</td><td></td><td>A young boy enters the kitchen in the middle of the night and finds his mother behaving very atypically</td></tr><tr><td>Neutral Film Clip 1</td><td>The Other Sister, Mandeville Films, 1999, Director: G. Marshall</td><td></td><td>A mother and daughter are discussing an art book while sitting in their living room</td></tr><tr><td>Neutral Film Clip 2</td><td>The Game, Polygram Filmed Entertainment, 1999, Director: D. Fincher</td><td></td><td>One gentleman is explaining the rules of a game to another</td></tr><tr><td>Neutral Film Clip 3</td><td>The Notebook, New Line Cinema, 2004, Director: N. Cassavetes</td><td></td><td>An older gentleman and his doctor are conversing during a routine check-up</td></tr></tbody></table>

exploring the use of isolated expressions and film clips to evaluate emotion recognition by people with traumatic brain injury

The current study presented 60 people with traumatic brain injury (TBI) and 60 controls with isolated facial emotion expressions, isolated vocal emotion expressions, and multimodal (i.e., film clips) stimuli that included contextual cues. All stimuli were presented via computer. Participants were required to indicate how the person in each stimulus was feeling using a forced-choice format. Additionally, for the film clips, participants had to indicate how they felt in response to the stimulus, and the level of intensity with which they experienced that emotion.

This task battery was used to compare emotion recognition for isolated emotion expressions (i.e., face-only; voice-only) and combined emotion expressions (i.e., face and voice) that occur within a situational context. A total of 60 (37 males and 23 females) participants with moderate to severe TBI between the ages of 21 and 63 years (mean = 40.98) and 60 (38 males and 22 females) age -matched Controls (range = 18 to 63; mean = 40.64) completed the three tasks. To participate in the study, participants with a TBI needed to have either a Glasgow coma scale score, posttraumatic amnesia or loss of conscious that was indicative of a moderate to severe TBI. The average time since injury was 13.68 years and the most common cause of injury was a motor vehicle accident (72%). Participants with a TBI had completed an average of 14.43 years of education and control participants 15.72 years of education. Participants with TBI were excluded from the study if they had a developmental affective disorder, acquired neurological disorder, major psychiatric disorder or impaired vision or hearing that would inhibit full participation in study tasks. Control participants were excluded if they reported any history of TBI of any severity, including concussion resulting in postconcussive syndrome. All participants spoke English as their primary language.
To ensure there were no task order effects, we assigned participants to one of three restricted randomized orders of film clips and one of six task orders (see Figure 1). Film and task order were pre-determined using an on-line computer randomized number generator. Neither film nor task order were shown to effect performance and thus were not used as covariates in analyses.
Figure 2 includes a pictorial example of a stimulus from the DANVA-Faces task and the Film Clips task. The benefits and challenges of using the DANVA2 and Film Clip stimuli are examined in the discussion.
An ANOVA examining group differences across the three affect recognition tasks found significant main effects for group [F(1, 100) = 9.32; p = 0.003] and task [F(2, 200) = 11.05, p &#60;0.001]. As hypothesized, people with TBI were significantly less accurate than Controls for performance on the DANVA-Faces task, F(1, 101) = 6.77; p = 0.005, and the DANVA-Voices task, F(1, 100) = 9.64, p = 0.002 (see Figure 3). It was surprising to find that recognition of emotion in the film clips task did not significantly differ between the two groups, F(1, 118) = &#160;2.12, p&#160;= 0.15. We had anticipated that the multimodal nature of the film clips would increase the cognitive demand of the task, making it more difficult for people with TBI to recognize emotion in the film clips compared to Controls. This was not the case. It appears that the multimodal cues increased redundancy, allowing for people with TBI to recognize emotion more easily than they could the isolated expressions.
Similar to previous work in this area, people with TBI found it significantly more challenging to recognize negative emotion expressions than positive ones for both the DANVA-Faces [F(1, 101) = 6.8, p&#160;= 0.01] and the DANVA-Voices [DANVA-Voices; F(1,100) = 10.85, p&#160;= 0.001] tasks (see Figure 4).
While studies in this area have reported valence effects, few studies have examined responses to the target emotions. We analyzed responses for happy, sad, fearful, and angry stimuli for all three tasks (see Figure 5) and additionally analyzed responses to the neutral film clips (see Figure 6). Although the valence results indicated that people with TBI generally found negatively valenced stimuli more challenging to identify than people without TBI, there was a desire to investigate whether a particular emotion (e.g., angry) was driving this result. Since happy was our only positively valenced emotion, our results here mirrored those for valence. When examining responses to the negatively valenced emotions (sad, angry, fearful), results showed that people with TBI found fearful to be the most difficult emotion to identify. In fact, this was the only between group response difference found in the Film Clip task. Identifying neutral was significantly more difficult for people with TBI than Controls, F(1, 118) = 6.68, p&#160;= 0.001. When neutral was incorrectly identified, people with TBI were significantly more likely to provide a negatively valenced response than Controls, F(2,236) = 4.75, p&#160;= 0.009.
<img alt="Figure 1" src="/files/ftp_upload/53774/53774fig1.jpg" /> 
Figure 1. Schematic representation of protocol. Prior to testing, participants were assigned to one of three presentation orders for the film clips. There were also assigned to one of six randomized orders of task completion. On the day of testing, participants provided preliminary information before completing the experimental tasks [Film clips, DANVA-Faces (Faces), and DANVA-Voices (Voices)]. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/53774/53774fig1large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 2" src="/files/ftp_upload/53774/53774fig2.jpg" /> 
Figure 2. Examples of Stimuli Used.&#160;&#160;A)&#160;An example of the isolated facial stimuli in the DANVA-Faces task. All stimuli in this task are static photographs with the same background. B) A screenshot of one film clip in the Film Clips task. All film clips were taken from commercial movies and television and ranged from 45 - 103 milliseconds in length. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/53774/53774fig2large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 3" src="/files/ftp_upload/53774/53774fig3.jpg" /> 
Figure 3. Responses across Media, by Group. Responses for each task type by people with and without TBI. Participants with TBI were significantly less accurate than Controls for recognition of static facial expressions of emotion [DANVA-Faces; F(1, 101) = 6.77; p = 0.005], and vocal emotion expressions [DANVA-Voices; F(1, 100) = 9.64, p = 0.002]. The difference of recognition of emotion for the context-enriched film clips was not significant [F(1, 118) = &#160;2.12, p = 0.15]. Error bars show standard error of the mean (SEM). <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/53774/53774fig3large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 4" src="/files/ftp_upload/53774/53774fig4.jpg" /> 
Figure 4. Valence Responses across Media, by Group. Responses to positive versus negative stimuli across tasks. People with TBI were able to recognize positive emotion expressions similarly to Controls but had significantly more difficulty recognizing negative facial emotion expressions [DANVA-Faces; F(1, 101) = 6.8, p = 0.01] and negative vocal emotion expressions [DANVA-Voices; F(1,100) = 10.85, p = 0.001]. Error bars show standard error of the mean (SEM). <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/53774/53774fig4large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 5" src="/files/ftp_upload/53774/53774fig5.jpg" /> 
Figure 5. Responses for Each Emotion Category across Media, by Group. This figure shows responses to each of the target emotions for people with TBI and Controls for the DANVA-Faces, DANVA-Voices, and Film Clips tasks. People with TBI performed similarly to Controls for Happy, regardless of the stimulus type. For sad, angry, and fearful, people with TBI were significantly less accurate than Controls for DANVA-Faces and DANVA-Voices tasks. People with TBI also had significantly more difficulty identifying fearful than Controls in Film Clips. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/53774/53774fig5large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 6" src="/files/ftp_upload/53774/53774fig6.jpg" /> 
Figure 6. Responses to Neutral Stimuli in the Film Clips Task, by Group. People with TBI had significantly more difficulty labeling neutral film clips as neutral than Controls, F(1, 118) = 6.68, p = 0.001. When neutral was incorrectly identified, people with TBI were also significantly more likely to provide a negatively valenced response instead than Controls, F(2,236) = 4.75, p = 0.009. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/53774/53774fig6large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>Emotion</td>
			<td>Television Show/Commercial Film</td>
			<td>Scene</td>
			<td>Length (sec)</td>
		</tr>
		<tr>
			<td rowspan="2">Happy</td>
			<td>Sweet Home Alabama, D &#38; D Films, 2002</td>
			<td rowspan="2">A man surprises his girlfriend with a marriage proposal at a jewelry store.</td>
			<td rowspan="2">76</td>
		</tr>
		<tr>
			<td>Director: A. Tennant</td>
		</tr>
		<tr>
			<td rowspan="2">Happy</td>
			<td>Wedding Crashers, New Line Cinema, 2005</td>
			<td rowspan="2">A young couple is sitting on the beach playing flirtatious games.&#160;</td>
			<td rowspan="2">62</td>
		</tr>
		<tr>
			<td>Director: D. Dobkin</td>
		</tr>
		<tr>
			<td rowspan="2">Happy</td>
			<td>Remember the Titans, Jerry Bruckheimer Films, 2000</td>
			<td rowspan="2">A black family is accepted into a white community after the high school football team wins the championship game.&#160;</td>
			<td rowspan="2">65</td>
		</tr>
		<tr>
			<td>Director:B. Yakin</td>
		</tr>
		<tr>
			<td rowspan="2">Sad</td>
			<td>Grey&#8217;s Anatomy, ABC, 2006</td>
			<td rowspan="2">A father is going into surgery and is only able to communicate with his family by blinking his eyes.</td>
			<td rowspan="2">98</td>
		</tr>
		<tr>
			<td>Director: P. Horton</td>
		</tr>
		<tr>
			<td rowspan="2">Sad</td>
			<td>Armageddon, Touchstone Pictures, 1998</td>
			<td rowspan="2">A daughter must say goodbye to her father while he is on a dangerous mission.&#160;</td>
			<td rowspan="2">75</td>
		</tr>
		<tr>
			<td>Director: M. Bay</td>
		</tr>
		<tr>
			<td rowspan="2">Sad</td>
			<td>Grey&#8217;s Anatomy, ABC, 2006</td>
			<td rowspan="2">A woman is heart broken after her fianc&#233; has passed away.</td>
			<td rowspan="2">87</td>
		</tr>
		<tr>
			<td>Director: M. Tinker</td>
		</tr>
		<tr>
			<td rowspan="3">Angry</td>
			<td>Anne of Green Gables, Canadian Broadcast Corporation, 1985&#160;</td>
			<td rowspan="3">An older woman speaks openly about a child&#8217;s physical flaws in front of her.</td>
			<td rowspan="3">62</td>
		</tr>
		<tr>
			<td>Director: K. Sullivan</td>
		</tr>
		<tr>
			<td>Canada: Anne of Green Gables Productions</td>
		</tr>
		<tr>
			<td rowspan="3">Angry</td>
			<td>Enough, Columbia Pictures, 2002</td>
			<td rowspan="3">A wife confronts her husband about infidelity and he responds by exerting physical and emotional dominance.</td>
			<td rowspan="3">103</td>
		</tr>
		<tr>
			<td>Director: M. Apted</td>
		</tr>
		<tr>
			<td>United States: Columbia Pictures</td>
		</tr>
		<tr>
			<td rowspan="2">Angry</td>
			<td>Pretty Woman, Touchstone Pictures, 1990</td>
			<td rowspan="2">A call girl attempts to buy clothing from an expensive store and the store clerks turn her away.</td>
			<td rowspan="2">65</td>
		</tr>
		<tr>
			<td>Director: G. Marshall</td>
		</tr>
		<tr>
			<td rowspan="2">Fearful</td>
			<td>Blood Diamond, Warner Bros Pictures, 2006</td>
			<td rowspan="2">A father and son are walking together when numerous vehicles carrying militia approach.</td>
			<td rowspan="2">50</td>
		</tr>
		<tr>
			<td>Director: E. Zwick</td>
		</tr>
		<tr>
			<td rowspan="2">Fearful</td>
			<td>The Life Before Her Eyes, 2929 Productions, 2007</td>
			<td rowspan="2">Two teenage girls are talking in a highschool bathroom when they hear gunshots that continue to get closer.</td>
			<td rowspan="2">79</td>
		</tr>
		<tr>
			<td>Director: V. Perelman</td>
		</tr>
		<tr>
			<td>Fearful</td>
			<td>The Sixth Sense, Barry Mendel Productions, 1999</td>
			<td>A young boy wakes from his sleep and enters the kitchen to find his mother making dinner.</td>
			<td>70</td>
		</tr>
		<tr>
			<td>Neutral</td>
			<td>The Other Sister, Mandeville Films, 1999</td>
			<td>A mother and daughter are sitting and discussing an art book.&#160;</td>
			<td>45</td>
		</tr>
		<tr>
			<td>Neutral</td>
			<td>The Game, Polygram Filmed Entertainment, 1999</td>
			<td>One gentleman is explaining the rules of a game to another gentleman.&#160;</td>
			<td>85</td>
		</tr>
		<tr>
			<td rowspan="2">Neutral</td>
			<td>The Notebook, New Line Cinema, 2004</td>
			<td rowspan="2">An older gentlemen and his doctor are conversing during a routine check-up.</td>
			<td rowspan="2">56</td>
		</tr>
		<tr>
			<td>Director: N. Cassavetes</td>
		</tr>
	</tbody>
</table>
Table 1. Description of Film Clips Used.

Watch this Scientific Journal Video about Exploring the Use of Isolated Expressions and Film Clips to Evaluate Emotion Recognition by People with Traumatic Brain Injury at JoVE.com

Exploring the Use of Isolated Expressions and Film Clips to Evaluate Emotion Recognition by People with Traumatic Brain Injury

This paper describes how to implement a battery of behavioral tasks to examine emotion recognition of isolated facial and vocal emotion expressions, and a novel task using commercial television and film clips to assess multimodal emotion recognition that includes contextual cues.

The current study presented 60 people with traumatic brain injury (TBI) and 60 controls with isolated facial emotion expressions, isolated vocal emotion ...

exploring-use-isolated-expressions-film-clips-to-evaluate-emotion

Nanyang Technological University

Research

JoVE Journal

Behavior

9.0K Views.  Brock University.  Dit document beschrijft hoe u een batterij van gedrags-taken uit te voeren om herkenning van emoties van geïsoleerde het gezicht en vocale emotie te onderzoeken en een nieuwe taak met behulp van commerciële televisie en film clips multimodale herkenning van emoties die contextuele signalen omvat beoordelen. 

Video: Exploring the Use of Isolated Expressions and Film Clips to Evaluate Emotion Recognition by People with Traumatic Brain Injury

Brain Imaging Investigation of the Neural Correlates of Emotional Autobiographical Recollection

Recollection of emotional autobiographical memories (AMs) is important to healthy cognitive and affective functioning 1 - remembering positive AMs is associated with increased personal well-being and self-esteem 2, whereas remembering and ruminating on negative AMs may lead to affective disorders 3. Although significant progress has been made in understanding the brain mechanisms underlying AM retrieval in general (reviewed in 4, 5), less is known about the effect of emotion on the subjective re-experience of AMs and the associated neural correlates. This is in part due to the fact that, unlike the investigations of the emotion effect on memory for laboratory-based microevents (reviewed in 6, 7-9), often times AM studies do not have a clear focus on the emotional aspects of remembering personal events (but see 10). Here, we present a protocol that allows investigation of the neural correlates of recollecting emotional AMs using functional magnetic resonance imaging (fMRI). Cues for these memories are collected prior to scanning by means of an autobiographical memory questionnaire (AMQ), therefore allowing for proper selection of emotional AMs based on their phenomenological properties (i.e., intensity, vividness, personal significance). This protocol can be used in healthy and clinical populations alike.

We present a protocol that allows investigation of the neural correlates of recollecting emotional autobiographical memories, using functional magnetic resonance imaging. This protocol can be used with both healthy and clinical participants.

Recollection of emotional autobiographical memories (AMs) is important to healthy cognitive and affective functioning 1 - remembering positive AMs is ...

Neuroscience

Brain Imaging Investigation of the Neural Correlates of Emotion Regulation

The ability to control/regulate emotions is an important coping mechanism in the face of emotionally stressful situations. Although significant progress has been made in understanding conscious/deliberate emotion regulation (ER), less is known about non-conscious/automatic ER and the associated neural correlates. This is in part due to the problems inherent in the unitary concepts of automatic and conscious processing1. Here, we present a protocol that allows investigation of the neural correlates of both deliberate and automatic ER using functional magnetic resonance imaging (fMRI). This protocol allows new avenues of inquiry into various aspects of ER. For instance, the experimental design allows manipulation of the goal to regulate emotion (conscious vs. non-conscious), as well as the intensity of the emotional challenge (high vs. low). Moreover, it allows investigation of both immediate (emotion perception) and long-term effects (emotional memory) of ER strategies on emotion processing. Therefore, this protocol may contribute to better understanding of the neural mechanisms of emotion regulation in healthy behaviour, and to gaining insight into possible causes of deficits in depression and anxiety disorders in which emotion dysregulation is often among the core debilitating features.

We present a protocol that allows investigation of the neural correlates of deliberate and automatic emotion regulation, using functional magnetic resonance imaging. This protocol can be used in healthy participants, both young and older, as well as in clinical patients.

The ability to control/regulate emotions is an important coping mechanism in the face of emotionally stressful situations. Although significant progress ...

Brain Imaging Investigation of the Memory-Enhancing Effect of Emotion

Emotional events tend to be better remembered than non-emotional events1,2. One goal of cognitive and affective neuroscientists is 
to understand the neural mechanisms underlying this enhancing effect of emotion on memory. A method that has proven particularly influential in the 
investigation of the memory-enhancing effect of emotion is the so-called subsequent memory paradigm (SMP). This method was originally used to investigate the 
neural correlates of non-emotional memories3, and more recently we and others also applied it successfully to studies of emotional memory (reviewed in4, 5-7). 
Here, we describe a protocol that allows investigation of the neural correlates of the memory-enhancing effect of emotion using the SMP in conjunction with 
event-related functional magnetic resonance imaging (fMRI). An important feature of the SMP is that it allows separation of brain activity specifically 
associated with memory from more general activity associated with perception. Moreover, in the context of investigating the impact of emotional stimuli, 
SMP allows identification of brain regions whose activity is susceptible to emotional modulation of both general/perceptual and memory-specific processing. 
This protocol can be used in healthy subjects8-15, as well as in clinical patients where there are alterations in the neural correlates of emotion perception 
and biases in remembering emotional events, such as those suffering from depression and post-traumatic stress disorder (PTSD)16, 17.

We present a protocol that uses functional magnetic resonance imaging to investigate the neural correlates of the memory-enhancing effect of emotion. This protocol allows identification of brain activity specifically linked to memory-related processing, contrary to more general perceptual processing, and can be used with healthy and clinical populations.

Emotional events tend to be better remembered than non-emotional events1,2. One goal of cognitive and affective neuroscientists is 
to understand the ...

Brain Imaging Investigation of the Impairing Effect of Emotion on Cognition

Emotions can impact cognition by exerting both enhancing (e.g., better memory for emotional events) and impairing (e.g., increased emotional distractibility) effects (reviewed in 1). Complementing our recent protocol 2 describing a method that allows investigation of the neural correlates of the memory-enhancing effect of emotion (see also 1, 3-5), here we present a protocol that allows investigation of the neural correlates of the detrimental impact of emotion on cognition. The main feature of this method is that it allows identification of reciprocal modulations between activity in a ventral neural system, involved in 'hot' emotion processing (HotEmo system), and a dorsal system, involved in higher-level 'cold' cognitive/executive processing (ColdEx system), which are linked to cognitive performance and to individual variations in behavior (reviewed in 1). Since its initial introduction 6, this design has proven particularly versatile and influential in the elucidation of various aspects concerning the neural correlates of the detrimental impact of emotional distraction on cognition, with a focus on working memory (WM), and of coping with such distraction 7,11, in both healthy 8-11 and clinical participants 12-14.

We present a protocol that allows investigation of the neural mechanisms mediating the detrimental impact of emotion on cognition, using functional magnetic resonance imaging. This protocol can be used with both healthy and clinical participants.

Emotions can impact cognition by exerting both enhancing (e.g., better memory for emotional events) and impairing (e.g., increased emotional ...

Conscious and Non-conscious Representations of Emotional Faces in Asperger's Syndrome

Several neuroimaging studies have suggested that the low spatial frequency content in an emotional face mainly activates the amygdala, pulvinar, and superior colliculus especially with fearful faces1-3. These regions constitute the limbic structure in non-conscious perception of emotions and modulate cortical activity either directly or indirectly2. In contrast, the conscious representation of emotions is more pronounced in the anterior cingulate, prefrontal cortex, and somatosensory cortex for directing voluntary attention to details in faces3,4. Asperger&#39;s syndrome (AS)5,6 represents an atypical mental disturbance that affects sensory, affective and communicative abilities, without interfering with normal linguistic skills and intellectual ability. Several studies have found that functional deficits in the neural circuitry important for facial emotion recognition can partly explain social communication failure in patients with AS7-9. In order to clarify the interplay between conscious and non-conscious representations of emotional faces in AS, an EEG experimental protocol is designed with two tasks involving emotionality evaluation of either photograph or line-drawing faces. A pilot study is introduced for selecting face stimuli that minimize the differences in reaction times and scores assigned to facial emotions between the pretested patients with AS and IQ/gender-matched healthy controls. Information from the pretested patients was used to develop the scoring system used for the emotionality evaluation. Research into facial emotions and visual stimuli with different spatial frequency contents has reached discrepant findings depending on the demographic characteristics of participants and task demands2. The experimental protocol is intended to clarify deficits in patients with AS in processing emotional faces when compared with healthy controls by controlling for factors unrelated to recognition of facial emotions, such as task difficulty, IQ and gender.

An EEG experimental protocol is designed to clarify the interplay between conscious and non-conscious representations of emotional faces in patients with Asperger&#39;s syndrome. The technique suggests that patients with Asperger&#39;s syndrome have deficits in non-conscious representation of emotional faces, but have comparable performance in conscious representation with healthy controls.

Several neuroimaging studies have suggested that the low spatial frequency content in an emotional face mainly activates the amygdala, pulvinar, and ...

Psychophysiological Assessment of the Effectiveness of Emotion Regulation Strategies in Childhood

Effective regulation of emotion is one of the most important skills that develops in childhood. Research interest in this area is expanding, but empirical work has been limited by predominantly correlational investigations of children's skills. Relatedly, a key conceptual challenge for emotion scientists is to distinguish between emotion responding and emotion regulatory processes. This paper presents a novel method to address these conceptual and methodological issues in child samples. An experimental paradigm that assesses the effectiveness with which children regulate emotion is described. Children are randomly assigned to use specific emotion regulation strategies, negative emotions are elicited with film clips, and changes in subsequent psychophysiology index the extent to which emotion regulation is effective. Children are instructed to simply watch the emotion-eliciting film (control), distract themselves from negative emotions (cognitive distraction), or reframe the situation in a way that downplays the importance of the emotional event (cognitive reappraisal). Cardiac physiology, continuously acquired before and during the emotional task, serves as an objective measure of children's unfolding emotional responding while viewing evocative films. Key comparisons in patterns of obtained physiological reactivity are between the control and emotion regulation strategy conditions. Representative results from this approach are described, and discussion focuses on the contribution of this methodological approach to developmental science.

A detailed protocol for an experimental paradigm to assess the effectiveness with which children regulate emotion is described. Children are randomly assigned to use specific emotion regulation strategies, negative emotions are elicited with film clips, and changes in subsequent psychophysiology index the extent to which emotion regulation is effective.

Effective regulation of emotion is one of the most important skills that develops in childhood. Research interest in this area is expanding, but empirical ...

Use of a Psychophysiological Script-driven Imagery Experiment to Study Trauma-related Dissociation in Borderline Personality Disorder

This protocol offers a detailed description of a psychophysiological experiment using script-driven trauma-related imagery and standardized clinical instruments within a comparative design assessing physiological and psychopathological features of individuals with BPD. This method aims at studying the psychological and physiological effects of trauma-related dissociation. Since the psychodiagnostic classification of trauma-related disorders relies on the observation of evolutionarily determined responses to life-threat, an integrated assessment paradigm for the study of reactions to traumatic memories proposes a very appropriate methodological approach.
The employed script-driven imagery paradigm uses individual recall instructions to activate trauma-related memory networks and prompt associated emotional and physiological responses. These responses are measured by means of self-rating scales and physiological assessments. During the individual recall, participants are asked to vividly imagine traumatic and everyday experiences and other situations triggered by short personalized verbal scripts they authored beforehand together with the experimenters. A wide range of affective reactions and different physiological parameters can be measured. We used this paradigm to investigate dissociative states in BPD and to find physiological and affective correlates of dissociative states. Some of the participants were having severe traumatic antecedents.
To investigate different reaction patterns within the same diagnostic group, participants with different levels of traumatic histories, symptom severities, and co-morbidities should be included. By using short verbal scripts, the level of stress induced to participants is held as low as possible without affecting the validity of the object of investigation.

We present a protocol of personalized script-driven trauma-related imagery and clinical assessments within a comparison design for investigating peritraumatic dissociation (PD), psychophysiological reactions, i.e. heart rate (HR) and skin conductance (SC), and psychological features of often severely traumatized individuals with borderline personality disorder (BPD).

This protocol offers a detailed description of a psychophysiological experiment using script-driven trauma-related imagery and standardized clinical ...

The Adventures of Fundi Intervention Based on the Cognitive and Emotional Processing in Attention Deficit Hyperactive Disorder Patients

Many children with attention deficit hyperactive disorder (ADHD) are known to have executive dysfunction, which weakens their abilities to learn and behave in daily living. This protocol describes the methodology that is required for the intervention (psychotherapy) based on planning, attention, successive, and simultaneous (PASS theory) cognitive processing and fear emotional processing. It provides guiding principles and practical recommendations. A disproportionately high level of fear (dysregulation) increases the vulnerability for dysfunction in learning and behavior. We explain the interplay between emotion and cognition at the neurological level.
A go/no go task (The Adventures of Fundi), which involves decision making, is administered in a PC- mode to a sample of 66 ADHD subjects. The Adventures of Fundi, a computer program, was constructed to induce successive or simultaneous processing when involving the training of planning and selective attention. It aims to improve the executive function with planning and selective attention. If executive function improves, learning improves, and behavior ameliorates. After intervention over 6 months, remission was achieved in 70% of subjects.
The instructor encourages the use of appropriate strategies and points out the ways in which the strategies can be useful in finding the solution to the problem (go/no go). The emphasis is not on rehearsing and adult instructed verbal sequence. The verbalization may reveal the conscious verbalized strategy to solve a task that is not really the strategy being unconsciously used in that case. A self-verbal report is unreliable. This is an inductive learning rather than deductive rule-learning approach central to cognitive PASS training. This inductive training has proved to produce not only near transfer but also far transfer.
Noncognitive factors (emotional factors) must be considered to maximize the benefit of cognitive training. Indirect and metaphorical communication considers the emotional factor.

This protocol illustrates a remedial therapy based on inductive learning and indirect communication (Ericksonian metaphorical hypnosis) that can be applied to children with Attention Deficit Hyperactive Disorder (ADHD). Particularly, this protocol is intended to test the effectiveness of a go/no-go decision task. Neuroscientific rationale is presented throughout the text.

Many children with attention deficit hyperactive disorder (ADHD) are known to have executive dysfunction, which weakens their abilities to learn and ...

Assessment and Communication for People with Disorders of Consciousness

In this experiment, we demonstrate a suite of hybrid Brain-Computer Interface (BCI)-based paradigms that are designed for two applications: assessing the level of consciousness of people unable to provide motor response and, in a second stage, establishing a communication channel for these people that enables them to answer questions with either &#39;yes&#39; or &#39;no&#39;. The suite of paradigms is designed to test basic responses in the first step and to continue to more comprehensive tasks if the first tests are successful. The latter tasks require more cognitive functions, but they could provide communication, which is not possible with the basic tests. All assessment tests produce accuracy plots that show whether the algorithms were able to detect the patient&#39;s brain&#39;s response to the given tasks. If the accuracy level is beyond the significance level, we assume that the subject understood the task and was able to follow the sequence of commands presented via earphones to the subject. The tasks require users to concentrate on certain stimuli or to imagine moving either the left or right hand. All tasks are designed around the assumption that the user is unable to use the visual modality, and thus, all stimuli presented to the user (including instructions, cues, and feedback) are auditory or tactile.

With this experiment, one might be able to detect consciousness in people with disorders of consciousness. Furthermore, the approach can create a simple communication channel that enables people to give simple YES/NO answers to questions.

In this experiment, we demonstrate a suite of hybrid Brain-Computer Interface (BCI)-based paradigms that are designed for two applications: assessing the ...

Physiological Correlates of Emotion Recognition

Source: Laboratories of Jonas T. Kaplan and Sarah I. Gimbel&#8212;University of Southern California
The autonomic nervous system (ANS) controls the activity of the body's internal organs and regulates changes in their activity depending on the current environment. The vagus nerve, which innervates many of the internal organs, is an important part of the system. When our brain senses danger, vagal tone is inhibited, leading to a set of changes in the body designed to make us more prepared to fight or flee; for example, our heart rate increases, our pupils dilate, and we breath more quickly. Conversely, when the vagal system is activated, these physiological responses are inhibited, leading to a calmer state. The vagus nerve, then, acts as a kind of "brake" on our arousal. One interesting consequence of this calmer state is that it tends to promote social interaction-when we are not tensed and afraid of our immediate environment we are instead receptive to interacting with others. Poor functioning of this regulatory mechanism, therefore, may be associated with difficulties in social behavior.
One index of autonomic regulation is heart rate variability (HRV). HRV is a measure of how much the gap between one beat and the next varies over time. High HRV means there are continual fluctuations in the heart rate over time, a reflection of successful autonomic regulation. Low HRV means there is consistency of the heart rate over time, a condition associated with poor autonomic regulation. 
In this study we will test the hypothesis that increased HRV is associated with more accurate categorization of emotional stimuli.1,2 Following a study by Park et al., we will measure HRV and test its association on a task that measures skill at perceiving facial emotions.3

Correlating Heart Rate Variability with Emotion Recognition

Source: Laboratories of Jonas T. Kaplan and Sarah I. Gimbel&#8212;University of Southern California
The autonomic nervous system (ANS) controls the ...

Exploring the Use of Isolated Expressions and Film Clips to Evaluate Emotion Recognition by People with Traumatic Brain Injury

Samenvatting

Meer video's verkennen

Hoofdstukken in deze video