Name: using the race model inequality to quantify behavioral multisensory integration effects
Uploaded: 2019-05-10T14:45:00.000+00:00
Duration: 8 min 13 s
Description: Watch this Scientific Journal Video about Using the Race Model Inequality to Quantify Behavioral Multisensory Integration Effects at JoVE.com

The current body of work is supported by the National Institute on Aging at the National Institute of Health (K01AG049813 to JRM). Supplementary funding was provided by the Resnick Gerontology Center of the Albert Einstein College of Medicine. Special thanks to all the volunteers and research staff for exceptional support with this project.

All participants provided written informed consent to the experimental procedures, which were approved by the institutional review board of the Albert Einstein College of Medicine.
1. Participant Recruitment, Inclusion Criteria, and Consent
<ol>
	<li>Recruit a relatively large cohort of English-speaking individuals who can ambulate independently and are free of significant sensory loss; active neurological or psychiatric disorders that interfere with experimental evaluations; and current/future medical procedures that affect mobility.</li>
	<li>Ensure that each participant can successfully complete a sensory screening exam, where visual, auditory, and somatosensory acuity are formally tested to confirm study appropriateness.
	<ol>
		<li>Use the Snellen eye chart to ensure that bilateral visual acuity is better than or equal to 20/100.</li>
		<li>Use a tone-emitting otoscope to ensure that participants are at a minimum able to hear a 2,000 Hz tone at 25 dB30.</li>
		<li>Determine whether participants maintain a diagnosis of clinical neuropathy and whether it interferes with the ability to feel the experimental somatosensory stimulation21,28,29.</li>
		<li>If the participant is unable to meet these minimum sensory requirements, do not include them in the study.</li>
	</ol>
	</li>
	<li>Exclude older adults with dementia by implementing cut-scores from reliable screening instruments such as the AD8 Dementia Screening Interview cutoff score &#8805;2 31,32; and the Memory Impairment Screen MIS; cutoff score &#60; 533.</li>
	<li>Have participants provide written informed consent to the experimental procedures (approved by a local institutional review board) if willing to participate.</li>
</ol>
2. Experimental Design
<ol>
	<li>Use stimulus presentation software to program a simple reaction time experiment with three experimental conditions: visual (V) alone, somatosensory (S) alone, and simultaneous visual-somatosensory (VS). Inform participants to respond to each sensory stimulus, regardless of the condition, as quickly as possible. See supplementary files for an example of a VS simple RT task (Supplementary File 1).
	<ol>
		<li>Use a stimulus generator with three control boxes (30.48 mm &#215; 20.32 mm &#215; 12.70 mm) and plastic housing for stimulators. The left and right control boxes contain bilateral blue light emitting diodes (LEDs; 15.88 cm diameter) that illuminate for visual stimulation and bilateral motors with 0.8 G vibration amplitude that vibrate for somatosensory stimulation (equivalent to a cell-phone vibration)22,23,28.</li>
		<li>Ensure that stimulus generators provide both unisensory (visual OR somatosensory alone), as well as multisensory (simultaneous visual AND somatosensory) stimulation. Place a center dummy control box equidistant (28 cm) from the left and right control boxes described in 2.1.1. and affix a visual target sticker (central circle of 0.4 cm diameter) to serve as the fixation point.</li>
		<li>Connect the stimulus generator to the experimental computer via the parallel port which allows the direct control for each stimulator.</li>
		<li>Program the stimulus presentation software to send transistor-transistor-logic (TTL, 5 V) pulses to the trigger stimulus generators on and off directly via the parallel port. Set the stimulus presentation time to 100 ms in duration.</li>
	</ol>
	</li>
	<li>In the stimulus presentation software, program a minimum of 3 experimental blocks each consisting of 45 trials (15 trials of each stimulus condition presented in random order) for a total of 135 stimulus presentations for this simple reaction time experiment.</li>
	<li>Vary the inter-stimulus-interval randomly between 1 and 3 s to prevent anticipatory effects. Alternatively, insert catch trials where the stimulus parameters are the same as above, but the TTL pulse is not sent, thus no visual or somatosensory stimulation occurs and, therefore, no response is expected.</li>
	<li>Allow participants up to 2,000 ms to respond to any given stimulus condition. If no response is detected within the 2,000 ms response period, ensure that the stimulus presentation software advances to the next trial automatically. 
	NOTE: This response window cut-off is arbitrary but necessary to keep the total experimental time to a minimum; note that longer RTs will be set to infinity regardless.</li>
	<li>Separate the three experimental blocks by programming 20-s breaks in the stimulus presentation software to reduce potential fatigue and increase concentration. Ensure each subsequent block starts immediately after the 20-s break concludes.</li>
	<li>Program written instructions to appear on the visual display (monitor of the experimental computer). The exact instructions are provided in the supplemental material. Ask the participant to start the experiment by pushing the response pad with their right foot when ready to commence. Once the stimulus parameters are programmed, the stimulus presentation software creates a script that is to be run on each participant.</li>
	<li>Provide participant ID and session number in order to run the experimental script. Once the experiment is completed, a unique behavioral data log is produced for each participant (see Supplementary File 2 for a sample Eprime 2.0 output file).</li>
</ol>
3. Apparatus &#38; Task
<ol>
	<li>Have participants sit upright and comfortably rest hands upon the left and right control boxes.
	<ol>
		<li>Strategically place index fingers over the vibratory motors mounted to the back of the control box, and thumbs on the front of the control box, under the LED to not block the light&#160;(see Figure 1).</li>
		<li>Ensure that the somatosensory stimuli are inaudible by providing participants with headphones over which continuous white noise is played at a comfortable level (typically 65-75 dBs).</li>
	</ol>
	</li>
	<li>Instruct participants to respond to all stimuli as quickly as possible.
	<ol>
		<li>Ask participants to use a foot-pedal located under the right foot as the response pad since fingers will be accepting somatosensory stimulation&#160;(see Figure 1).</li>
	</ol>
	</li>
	<li>Calculate performance accuracy by stimulus condition.
	<ol>
		<li>Instruct participants to respond to each of the experimental stimuli (45 per condition) as quickly as possible.</li>
		<li>Divide the number of accurately detected stimuli per condition over 45 (total number of trials per condition) to obtain measures of performance accuracy for visual, somatosensory, and VS conditions, respectively.</li>
	</ol>
	</li>
</ol>
4. Race Model Inequality Data Preparation (Individual Level)
<ol>
	<li>Determine whether an individual&#8217;s behavioral performance is valid.
	<ol>
		<li>Exclude participants that are not able to attain an accuracy of 70% correct or greater on any one stimulus condition As the participant&#8217;s performance accuracy on a simple reaction time task decreases, so does the reliability of the individual&#8217;s data.</li>
		<li>Consider trials inaccurate (omitted) if a participant fails to respond to a stimulus within the set response time period and set corresponding RT to infinity rather than excluding the trial from the analysis25,28. 
		NOTE: In previous studies, the group-averaged (n=289) stimulus detection was 96% across all conditions, and over 90 % of the population had detection rates above 90% for all conditions28.</li>
		<li>Do not employ data-trimming procedures that delete very slow RTs as this will bias the distribution of RT data.25 Ensure RTs that are clearly outliers are set to infinity. See supplementary file depicting alterations in CDFs based on data-trimming procedures and inclusion of slow RTs (Supplementary File 3).</li>
	</ol>
	</li>
	<li>Organize the RT Data.
	<ol>
		<li>Sort RT data in ascending order by the experimental condition. Place visual, somatosensory, and VS conditions in separate columns of sorted RT data. Ensure each row represents one trial and each cell represents the actual RT (or infinity in the case of omitted or slow trials).</li>
	</ol>
	</li>
	<li>Bin the RT Data.
	<ol>
		<li>Identify the fastest RT (to whichever condition- orange ellipse) and the slowest RT (to whichever condition-red ellipse). Subtract the slowest RT from the fastest (e.g., 740 ms &#8211; 237 ms) in order to calculate the individual&#8217;s RT range (503ms; blue ellipse) across ALL test conditions. Table 1&#160;demonstrates how to calculate an individual&#39;s RT Range and depicts the various color elipses.</li>
		<li>Bin RT data from the 0% (fastest RT = 237 in this example) to the 100% (or slowest RT = 740 in this example) in 5% increments by taking the fastest RT and gradually adding 5% of the RT range identified in 4.3.1 until 100% of the RT data is accounted for (see Table 2). This will result in 21-time bins. 
		NOTE: In Table 2 - 1%ile is only included in worksheet just for illustrative purposes.</li>
	</ol>
	</li>
	<li>Calculate the Cumulative Distribution Frequency (CDF) for the experimental conditions.
	<ol>
		<li>Using spreadsheet software, use a &#8220;FREQUENCY&#8221; function where array1 equals the actual RTs for one of the experimental conditions and array2 equals the 21 quantized RTs bins calculated in Step 4.3, divided by the total number of trials (45) per condition. This is illustrated in Figure 2a.</li>
		<li>Repeat this function for the other two experimental conditions (Figure&#160;2b-2c) so as to populate frequencies (or probability (P)) of an RT occurring within each of the 21 quantized time bins, for each of the three experimental conditions.</li>
		<li>Next, create the cumulative distribution frequency (CDF) by summing the running total of probabilities across the quantized bins (0%, 0+5%, 0+5+10%, 0+5+10+15%, etc.) for each of three experimental conditions. For example, in the cumulative probability column for the Soma condition (column AE), the cumulative probability for the 95%ile range (cell AE22) is the summation of the probability values in cells Z3:Z23 (see Figure 3).</li>
	</ol>
	</li>
	<li>Actual vs. Predicted CDFs.
	<ol>
		<li>Ensure that the CDF of the multisensory condition represents the actual CDF (see Figure 4 column AF and plotted purple trace). To calculate the predicted CDF (column AG), sum the two unisensory CDFs (with an upper limit set to 1)&#160;across each of the 21 quantized time bins (see Figure 5). Start at the 0th percentile (bin 1) and continue all the way down to the 100th percentile (bin 21).</li>
	</ol>
	</li>
	<li>Conduct the Test of the Race Model Inequality (RMI).
	<ol>
		<li>Subtract the predicted CDF (calculated in 4.5.2.) from the actual CDF for each of the 21 quantized time bins to obtain the difference values (column AH; see Figure 6).</li>
		<li>Plot these 21 values as a line graph, where the x-axis represents each one of the quantized time bins (column AC) and the y-axis represents the probability difference between the actual and predicted CDFs (column AH; Figure 7 (black trace).</li>
		<li>Check for positive values at any latency (i.e., quantiles) which indicate the integration of the unisensory stimuli and reflect a violation of the RMI (see the green highlighted portion of difference wave from 0.00 &#8211; 0.10 in Figure 7).</li>
	</ol>
	</li>
</ol>
5. Quantification of the Multisensory Effect (Group Level).
<ol>
	<li>Group-average the individual RMI data (differences between predicted CDF and the actual CDF for each of the 21-time bins; step 4.6.1- column AH) across all participants. Use a spreadsheet software to assign&#160;individuals to rows and time bins as columns. In a new spreadsheet, place the 21 values calculated in 4.6.1 in individual rows (1 row per participant), and average values within time bins to create one group-averaged difference waveform.</li>
	<li>Plot the group average 21 values as a line graph, where the x-axis represents each one of the quantized time bins and the y-axis represents the probability difference between CDFs.</li>
	<li>Visually inspect and document the violated portion of the group-averaged difference wave (i.e., positive values).</li>
	<li>Run Gondan&#8217;s RMI permutation test (R script available as a free download)26 to determine whether there is a statistically significant violation of the RMI over the positive values identified in step 5.3.
	<ol>
		<li>Organize the data in one text file where the first column is named &#8220;Obs&#8221; for Observer (e.g., participant ID), the second column is named &#8220;Cond&#8221; for stimulus condition (V, S, or VS) and the third column is named &#8220;RT&#8221; for actual RT or &#8220;Inf&#8221; if set to infinity.</li>
		<li>Open the software, identify which time bins are to be tested (based on the positive time bins identified in 5.3), and enter the text file name created in 5.4.1.</li>
		<li>Run the test by calling up the script. The results will provide a tmax value, 95% criterion, and p-value which will be instrumental in determining whether a significant violation of the Race Model exists across the entire study sample.</li>
	</ol>
	</li>
	<li>Calculate the area-under-the-curve (AUC) for each individual after establishing the significantly violated percentile bins in step 5.3. AUC will serve as the magnitude of multisensory integration (or the independent variable). To calculate AUC use participant 1&#8217;s data as an example, for percentile bins 0.00 - 0.15 depicted in Figure&#160;8a-d).
	<ol>
		<li>Sum the CDF difference value at time bin 1 (1st time positive value) with the CDF difference value of time bin 2 (next positive value) and then divide by two (see Figure 8a). Repeat step 5.3.1. for each consecutive pair of time bins containing positive values (see Figure 8b-8c).</li>
		<li>Sum the results obtained from steps 5.5.1 - 5.5.2. to generate the total AUC of the CDF difference wave during the violated percentile range (e.g., 0.00 &#8211; 0.15 in Figure 8d). 
		NOTE: AUC is a continuous measure and one AUC value is present for each individual for the violated portion of the RMI (Figure 8d red ellipse = participant 1&#8217;s AUC = 0.13). AUC can be used as an independent variable representing &#8216;magnitude of VS integration&#8217; which can later be tested to predict important clinical outcome measures (see also28,29).</li>
	</ol>
	</li>
	<li>Assign multisensory integration classification groups based on the number of violated percentile bins (values greater than zero highlighted in gray in Table 3) during the significantly violated percentile range identified above in step 5.3. Looking at Table 3 (percentile bins 0.00 &#8211; 0.15): Participant 1 has positive values for 2 out of 4 bins; Participant 2 has positive values for 4 out of 4 bins; and Participant 3 has positive values for 0 out of 4 bins.
	<ol>
		<li>Operationalize a classification system based on the number of violated percentile bins (values greater than zero for 0, 1, 2, or 3 bins) during the 0-10th percentile.&#160;</li>
		<li>Figure 9&#160;depicts one potential classification definition which is adapted from recently published data presented by Mahoney and Verghese29.</li>
	</ol>
	</li>
</ol>

<ol>
	<li>Foxe, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Auditory-somatosensory+multisensory+processing+in+auditory+association+cortex:+an+fMRI+study.">Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study.</a> Journal of Neurophysiology. 88 (1), 540-543 (2002).</li><li>Molholm, S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multisensory+auditory-visual+interactions+during+early+sensory+processing+in+humans:+a+high-density+electrical+mapping+study.">Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study.</a> Brain Research: Cognitive Brain Research. 14 (1), 115-128 (2002).</li><li>Murray, M. M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Grabbing+your+ear:+rapid+auditory-somatosensory+multisensory+interactions+in+low-level+sensory+cortices+are+not+constrained+by+stimulus+alignment.">Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment.</a> Cerebral Cortex. 15 (7), 963-974 (2005).</li><li>Molholm, S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Audio-visual+multisensory+integration+in+superior+parietal+lobule+revealed+by+human+intracranial+recordings.">Audio-visual multisensory integration in superior parietal lobule revealed by human intracranial recordings.</a> Journal of Neurophysiology. 96 (2), 721-729 (2006).</li><li>Peiffer, A. M., Mozolic, J. L., Hugenschmidt, C. E., Laurienti, P. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Age-related+multisensory+enhancement+in+a+simple+audiovisual+detection+task.">Age-related multisensory enhancement in a simple audiovisual detection task.</a> Neuroreport. 18 (10), 1077-1081 (2007).</li><li>Brandwein, A. B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+development+of+audiovisual+multisensory+integration+across+childhood+and+early+adolescence:+a+high-density+electrical+mapping+study.">The development of audiovisual multisensory integration across childhood and early adolescence: a high-density electrical mapping study.</a> Cerebral Cortex. 21 (5), 1042-1055 (2011).</li><li>Girard, S., Collignon, O., Lepore, F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multisensory+gain+within+and+across+hemispaces+in+simple+and+choice+reaction+time+paradigms.">Multisensory gain within and across hemispaces in simple and choice reaction time paradigms.</a> Experimental Brain Research. 214 (1), 1-8 (2011).</li><li>Mahoney, J. R., Li, P. C., Oh-Park, M., Verghese, J., Holtzer, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multisensory+integration+across+the+senses+in+young+and+old+adults.">Multisensory integration across the senses in young and old adults.</a> Brain Research. 1426, 43-53 (2011).</li><li>Foxe, J. J., Ross, L. A., Molholm, S., Stein, B. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Ch.+38.">Ch. 38.</a> The New Handbook of Multisensory Processing. , 691-706 (2012).</li><li>Kinchla, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Detecting+target+elements+in+multielement+arrays:+A+confusability+model.">Detecting target elements in multielement arrays: A confusability model.</a> Perception and Psychophysics. 15, 149-158 (1974).</li><li>Miller, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Divided+attention:+Evidence+for+coactivation+with+redundant+signals.">Divided attention: Evidence for coactivation with redundant signals.</a> Cognitive Psychology. 14 (2), 247-279 (1982).</li><li>Eriksen, C. W., Goettl, B., St James, J. D., Fournier, L. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Processing+redundant+signals:+coactivation,+divided+attention,+or+what?.">Processing redundant signals: coactivation, divided attention, or what?.</a> Perception and Psychophysics. 45 (4), 356-370 (1989).</li><li>Mordkoff, J. T., Yantis, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+interactive+race+model+of+divided+attention.">An interactive race model of divided attention.</a> Journal of Experimental Psychology: Human Perception and Performance. 17 (2), 520-538 (1991).</li><li>Miller, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Timecourse+of+coactivation+in+bimodal+divided+attention.">Timecourse of coactivation in bimodal divided attention.</a> Perception and Psychophysics. 40 (5), 331-343 (1986).</li><li>Gondan, M., Lange, K., Rosler, F., Roder, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+redundant+target+effect+is+affected+by+modality+switch+costs.">The redundant target effect is affected by modality switch costs.</a> Psychonomic Bulletin Review. 11 (2), 307-313 (2004).</li><li>Colonius, H., Diederich, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+race+model+inequality:+interpreting+a+geometric+measure+of+the+amount+of+violation.">The race model inequality: interpreting a geometric measure of the amount of violation.</a> Psychological Review. 113 (1), 148-154 (2006).</li><li>Maris, G., Maris, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Testing+the+race+model+inequality:+A+nonparametric+approach.">Testing the race model inequality: A nonparametric approach.</a> Journal of Mathematical Psychology. 47 (5-6), 507-514 (2003).</li><li>Clark, J. J., Yuille, A. L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Data Fusion for Sensory Information Processing Systems. , (1990).</li><li>Ernst, M. O., Banks, M. S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Humans+integrate+visual+and+haptic+information+in+a+statistically+optimal+fashion.">Humans integrate visual and haptic information in a statistically optimal fashion.</a> Nature. 415 (6870), 429-433 (2002).</li><li>Mahoney, J. R., Verghese, J., Dumas, K., Wang, C., Holtzer, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+effect+of+multisensory+cues+on+attention+in+aging.">The effect of multisensory cues on attention in aging.</a> Brain Research. 1472, 63-73 (2012).</li><li>Mahoney, J. R., Holtzer, R., Verghese, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Visual-somatosensory+integration+and+balance:+evidence+for+psychophysical+integrative+differences+in+aging.">Visual-somatosensory integration and balance: evidence for psychophysical integrative differences in aging.</a> Multisensory Research. 27 (1), 17-42 (2014).</li><li>Mahoney, J. R., Dumas, K., Holtzer, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Visual-Somatosensory+Integration+is+linked+to+Physical+Activity+Level+in+Older+Adults.">Visual-Somatosensory Integration is linked to Physical Activity Level in Older Adults.</a> Multisensory Research. 28 (1-2), 11-29 (2015).</li><li>Dumas, K., Holtzer, R., Mahoney, J. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Visual-Somatosensory+Integration+in+Older+Adults:+Links+to+Sensory+Functioning.">Visual-Somatosensory Integration in Older Adults: Links to Sensory Functioning.</a> Multisensory Research. 29 (4-5), 397-420 (2016).</li><li>Couth, S., Gowen, E., Poliakoff, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Using+race+model+violation+to+explore+multisensory+responses+in+older+adults:+Enhanced+multisensory+integration+or+slower+unisensory+processing.">Using race model violation to explore multisensory responses in older adults: Enhanced multisensory integration or slower unisensory processing.</a> Multisensory Research. 31 (3-4), 151-174 (2017).</li><li>Gondan, M., Minakata, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+tutorial+on+testing+the+race+model+inequality.">A tutorial on testing the race model inequality.</a> Attention, Perception &amp; Psychophysics. 78 (3), 723-735 (2016).</li><li>Gondan, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+permutation+test+for+the+race+model+inequality.">A permutation test for the race model inequality.</a> Behavior Research Methods. 42 (1), 23-28 (2010).</li><li>Kiesel, A., Miller, J., Ulrich, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Systematic+biases+and+Type+I+error+accumulation+in+tests+of+the+race+model+inequality.">Systematic biases and Type I error accumulation in tests of the race model inequality.</a> Behavior Research Methods. 39 (3), 539-551 (2007).</li><li>Mahoney, J., Cotton, K., Verghese, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multisensory+Integration+Predicts+Balance+and+Falls+in+Older+Adults.">Multisensory Integration Predicts Balance and Falls in Older Adults.</a> Journal of Gerontology: Medical Sciences. , (2018).</li><li>Mahoney, J. R., Verghese, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Visual-Somatosensory+Integration+and+Quantitative+Gait+Performance+in+Aging.">Visual-Somatosensory Integration and Quantitative Gait Performance in Aging.</a> Frontiers in Aging Neuroscience. 10, 377 (2018).</li><li>Yueh, B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Long-term+effectiveness+of+screening+for+hearing+loss:+the+screening+for+auditory+impairment--which+hearing+assessment+test+(SAI-WHAT)+randomized+trial.">Long-term effectiveness of screening for hearing loss: the screening for auditory impairment--which hearing assessment test (SAI-WHAT) randomized trial.</a> Journal of the American Geriatrics Society. 58 (3), 427-434 (2010).</li><li>Galvin, J. E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+AD8:+a+brief+informant+interview+to+detect+dementia.">The AD8: a brief informant interview to detect dementia.</a> Neurology. 65 (4), 559-564 (2005).</li><li>Galvin, J. E., Roe, C. M., Xiong, C., Morris, J. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Validity+and+reliability+of+the+AD8+informant+interview+in+dementia.">Validity and reliability of the AD8 informant interview in dementia.</a> Neurology. 67 (11), 1942-1948 (2006).</li><li>Buschke, H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Screening+for+dementia+with+the+memory+impairment+screen.">Screening for dementia with the memory impairment screen.</a> Neurology. 52 (2), 231-238 (1999).</li></ol>

There are no conflicts of interest to report and the authors have nothing to disclose.

The goal of the current study was to detail the process behind the establishment of a robust multisensory integration phenotype. Here, we provide the necessary and critical steps required to acquire multisensory integration effects that can be utilized to predict important cognitive and motor outcomes relying on similar neural circuitry. Our overall objective was to provide a step-by-step tutorial for calculating the magnitude of multisensory integration in an effort to facilitate innovative and novel translational multisensory studies across diverse clinical populations and age-ranges.
As stated above and outlined by Gondan and colleagues, it is very important to preserve the individual&#8217;s RT dataset25,28. That is, avoid data trimming procedures that omit very slow RTs given its inherent bias on the RT distribution;25 instead, set omitted and slow RTs to infinity. This step is critical and failure to abide by these simple rules will lead to the development of inaccurate multisensory integration results. Additionally, race model significance testing should only be tested over group-averaged violated portions of the RMI identified in the study cohort (i.e., not a priori specified windows).
In terms of limitations, the current experimental design was based on data from a simple reaction time task to bilateral stimuli that were presented to the same location and at precisely the same time. We recognize that several adaptations to the current experimental design can be made depending upon various hypotheses that researchers are interested in examining. We utilize this study as a launching pad towards documenting robust MSI effects in aging but recognize that implementation of various experimental adaptations (e.g., different bi- and even tri-sensory combinations, varied stimulus presentation onset&#160;times, and differential magnitude of stimulus intensity) will provide a wealth of incremental information regarding this multisensory phenomenon.
We have implemented the above approach to demonstrate significant associations between the magnitude of visual-somatosensory integration with balance28 and incident falls28, where older adults with greater multisensory integration abilities manifest better balance performance and less&#160;incident falls. Similarly, we demonstrate that the magnitude of visual-somatosensory integration was a strong predictor of spatial aspects of gait29, where individuals with worse visual-somatosensory integration demonstrated slower gait speed, shorter strides, and increased double support. In the future, this methodology should be used to uncover the relationship of MSI with other important clinical outcomes like cognitive status, and aid in the identification of critical functional and structural multisensory integrative neural networks in aging and other clinical populations.

Interactions across sensory systems are essential for everyday functions. While multisensory integration effects are measured across a wide array of populations using assorted sensory combinations and different neuroscience approaches [including but not limited to the psychophysical, electrophysiological, and neuroimaging methodologies]1,2,3,4,5,6,7,8,9, currently a gold standard for quantifying multisensory integration is lacking. Given that multisensory experiments typically contain&#160;a behavioral component, reaction time (RT) data is often examined to determine the existence of a well-known phenomenon called the redundant signals effect10. As its name suggests, simultaneous sensory signals provide redundant information, which typically yield quicker RTs. Race and co-activation models are used to explain the above mentioned redundant signals effect11. Under race models, the unisensory signal that is processed the fastest is the winner of the race and is responsible for producing the behavioral response. However, evidence for co-activation occurs when responses to multisensory stimuli are quicker than what race models predict.
Earlier versions of the race model are inherently controversial12,13 as they are referred to by some as overly conservative14,15 and purportedly contain limitations regarding the independence between the constituent unisensory detection times inherent in the multisensory condition16. In an effort to address some of these limitations, Colonius &#38; Diederich16 developed a more conventional race model test:
<img alt="Equation 1" src="/files/ftp_upload/59575/59575eq1.jpg" />,
where the cumulative distribution frequencies (CDFs) of the unisensory conditions (e.g., A &#38; B; with an upper limit of one) are compared to the CDF of the simultaneous multisensory condition (e.g., AB) for any given latency (t)11,16,17. In general, a CDF determines how often an RT occurs, within a given range of RTs, divided by the total number of stimulus presentations (i.e., trials). If the CDF of the actual multisensory condition <img alt="Equation 2" src="/files/ftp_upload/59575/59575eq2.jpg" /> is less than or equal to the predicted CDF derived from the unisensory conditions
<img alt="Equation 3" src="/files/ftp_upload/59575/59575eq3.jpg" />,
then the race model is accepted and there is no evidence for sensory integration. However, when the multisensory CDF is greater than the predicted CDF derived from the unisensory conditions, the race model is rejected. Rejection of the race model indicates that multisensory interactions from redundant sensory sources combine&#160;in a non-linear manner, resulting in a speeding up of RTs (e.g., RT facilitation) to multisensory stimuli.
One main hurdle that multisensory researchers face is how to best quantify integration effects. For instance, in the case of the most basic behavioral multisensory paradigm, where participants are asked to perform a simple reaction time task, information regarding accuracy and speed is collected. Such multisensory data can be used at the face-value or manipulated using various mathematical applications including but not limited to Maximum Likelihood Estimation18,19, CDFs11, and various other statistical approaches. The majority of our previous multisensory studies employed both quantitative and probabilistic approaches where multisensory integrative effects were calculated by 1) subtracting the mean reaction time (RT) to a multisensory event from the mean reaction time (RT) to the shortest unisensory event, and 2) by employing CDFs to determine whether RT facilitation resulted from synergistic interactions facilitated by redundant sensory information8,20,21,22,23. However, the former methodology was likely not sensitive to the individual differences in integrative processes and researchers have since posited that the later methodology (i.e., CDFs) may provide a better proxy for quantifying multisensory integrative effects24.
Gondan and Minakata recently published a tutorial on how to accurately test the Race Model Inequality (RMI) since researchers all too often make countless errors during the acquisition and pre-processing stages of RT data collection and preparation25. First, the authors posit that is unfavorable to apply data trimming procedures where certain a priori minimum and maximum RT limits are set. They recommend that slow and omitted responses be set to infinity, rather than excluded. Second, given that the RMI may be violated at any latency, multiple t-tests are often used to test the RMI at different time points (i.e., quantiles); unfortunately, this practice leads to the increased Type I error and substantially reduced statistical power. To avoid these issues, it is recommended that the RMI be tested over one specific time range. Some researchers have suggested that it makes sense to test the fastest quartile of responses (0-25%)26 or some pre-identified windows (i.e., 10-25%)24,27 as multisensory integration effects are typically observed during that time interval; however, we argue that the percentile range to be tested must be dictated by the actual dataset (see Protocol Section 5). The problem with relying on published data from young adults or computer simulations is that older adults manifest very different RT distributions, likely due to the age-related declines in sensory systems. Race model significance testing should only be tested over violated portions (positive values) of group-averaged difference wave between actual and predicted CDFs from the study cohort.
To this end, a protective effect of multisensory integration in healthy older adults using the conventional test of the race model16 and the principles set forth by Gondan and colleagues25 has been demonstrated. In fact, greater magnitude of visual-somatosensory RMI (a proxy for multisensory integration) was found to be linked to better balance performance, lower probability of incident falls and increased spatial gait performance28,29.
The objective of the current experiment is to provide researchers with a step-by-step tutorial to calculate the magnitude of multisensory integration effects using the RMI, to facilitate the increased production of diverse translational research studies across many different clinical populations. Note that data presented in the current study are from recently published visual-somatosensory experiments conducted on healthy older adults28,29, but this methodology can be applied to various cohorts across many different experimental designs, utilizing a wide-array of multisensory combinations.

<table><tbody><tr><td>stimulus generator</td><td>Zenometrics, LLC; Peekskill, NY, USA</td><td>n/a</td><td>custom-built</td></tr><tr><td>Excel</td><td>Microsoft Corporation</td><td></td><td>spreadsheet program</td></tr><tr><td>Eprime</td><td>Psychology Software Tools (PST)</td><td></td><td>stimulus presentation software</td></tr></tbody></table>

using the race model inequality to quantify behavioral multisensory integration effects

Multisensory integration research investigates how the brain processes simultaneous sensory information. Research on animals (mainly cats and primates) and humans reveal that intact multisensory integration is crucial for functioning in the real world, including both cognitive and physical activities. Much of the research conducted over the past several decades documents multisensory integration effects using diverse psychophysical, electrophysiological, and neuroimaging techniques. While its presence has been reported, the methods used to determine the magnitude of multisensory integration effects varies and typically faces much criticism. In what follows, limitations of previous behavioral studies are outlined and a step-by-step tutorial for calculating the magnitude of multisensory integration effects using robust probability models is provided.

The purpose of this study was to provide a step-by-step tutorial of a methodical approach to quantify the magnitude of VS integration effects, to foster the publication of new multisensory studies using similar experimental designs and setups (see Figure 1). Screenshots of each step and calculation needed to derive magnitude of multisensory integration effects, as measured by RMI AUC, are delineated above and illustrated in Figures&#160;2-8.
Figure 9 demonstrates a group-averaged violation (dashed trace) occurring over the 0-10% percentile range for a sample of 333 older adults (see also29). Here, the total number of positive values (0, 1, 2, or 3) for those 3 quantiles (0.00 - 0.10) determines which multisensory classification group a person is assigned (deficient, poor, good, or superior) respectively.&#160;
As depicted in Figure 9, group-averaged results demonstrate a significant race model violation over the fastest tenth of all response times26. While this group-averaged difference waveform suggests that on average older adults demonstrate significant race model violation (i.e., multisensory integration effects), we argue that this is not a one size fits all model. Rather, the individual&#8217;s AUC under the violated time period (0-10%ile) provides a better proxy for&#160;assessing the individual&#8217;s magnitude of VS integration, as differential integration patterns have been documented 20-23, 28,29. Once calculated, the individual magnitude of VS integration can serve as a continuous predictor of important outcomes in various clinical populations.
We recommend implementing a classification system, perhaps based on the number of violated percentile bins (values greater than zero) during the group-averaged RMI violation period, as a means of depicting inherent differential integration patterns. Classification of data in this manner will reveal a clear degradation of race model violation by multisensory integration classification group.
<img alt="Figure 1" class="xfigimg" src="/files/ftp_upload/59575/59575fig1.jpg" /> 
Figure 1: Experimental apparatus. Using a foot pedal located under the right foot as a response pad, participants were asked to respond to unisensory and multisensory stimuli as quickly as possible. This figure has been reprinted with permission22,28,29. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59575/59575fig1large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 2" class="xfigimg" src="/files/ftp_upload/59575/59575fig2.jpg" /> 
Figure 2: Calculating the frequency of an RT occurring within a specified range of RTs for each experimental condition. a) Visual (V); b) Somatosensory (S); and c) Visual-Somatosensory (VS). <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59575/59575fig2v2large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 3" class="xfigimg" src="/files/ftp_upload/59575/59575fig3.jpg" /> 
Figure 3: Creating the cumulative distribution frequency for the experimental conditions. This figure depicts the summation of the cumulative probability at the 95%ile bin for the Soma (S) condition. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59575/59575fig3v2large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 4" class="xfigimg" src="/files/ftp_upload/59575/59575fig4.jpg" /> 
Figure 4: Plotting the Actual CDF (VS condition; purple trace) as a function of quantile. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59575/59575fig4v2large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 5" class="xfigimg" src="/files/ftp_upload/59575/59575fig5.jpg" /> 
Figure 5: Calculating the Predicted CDF. Sum the CDFs of the two unisensory CDFs while including an upper limit = 1 for each of the quantiles from 0.00 to 1.00. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59575/59575fig5v2large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 6" class="xfigimg" src="/files/ftp_upload/59575/59575fig6.jpg" /> 
Figure 6: Create the Race Model Inequality (RMI). Subtract the CDF of the predicted CDF from the actual CDF for each quantile. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59575/59575fig6v2large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 7" class="xfigimg" src="/files/ftp_upload/59575/59575fig7.jpg" /> 
Figure 7: Plot the individual RMI values. The x-axis represents each of the 21 quantiles (column AC) and the y-axis represents the probability difference between CDFs (column AH). The green highlighted portion of the RMI depicts the positive or violated portion of the waveform, indicative of multisensory integration. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59575/59575fig7v2large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 8" class="xfigimg" src="/files/ftp_upload/59575/59575fig8.jpg" /> 
Figure 8: Calculating an individual&#8217;s Area-Under-the-Curve (AUC).&#160;a) Sum the CDF difference value at quantile 1 (0.00) with the CDF difference value of quantile 2 (0.05) and then divide by two to create a measure of AUC from 0.00 - 0.05. b-c) Repeat step a) for each consecutive pair of quantiles (e.g., 0.05 - 0.10 and 0.10 - 0.15) to attain the AUC for each quantile range. d) Sum the AUC for each time bin range to obtain the total AUC for the entire time bin window identified in 5.3. Note this example includes a wider quantile range (0.00 - 0.15) for illustrative purposes only. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59575/59575fig8large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 9" class="xfigimg" src="/files/ftp_upload/59575/59575fig9.jpg" /> 
Figure 9: Race Model Inequality: Overall and by Group Classification. The group-averaged difference between actual and predicted CDFs over the trajectory of all quantiles is represented by the dashed trace. The solid traces represent each of the four multisensory integration classifications defined above based on the number of violated quantile bins. This adapted figure has been reprinted with permission29. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59575/59575fig9v2large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
Supplementary File 1: Sample Simple Reaction Time Paradigm programmed in Eprime 2.0. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59575/S1_RACE_Model_Examples_using_Sample_RT_data.xlsx">Please click here to download this file.</a>
Supplementary File 2: Sample RT data behavioral data output from Eprime 2.0. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59575/S2_Sample_rt_data_from_eprime.xlsx">Please click here to download this file.</a>
Supplementary File 3: Sample RMI data with and without outliers and omitted trials. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59575/sample_eprime_behave_data.edat2">Please click here to download this file.</a>
Table 1. Individual Descriptive Statistics by Condition and Calculation of RT Range. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59575/Table_1.xlsx">Please click here to download this file.</a>
Table 2. Example of how to bin RT data based on RT range. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59575/Table_2.xlsx">Please click here to download this file.</a>
Table 3. Example of AUC calculation &#38; Identification of # of violated quantiles (grey shaded area). <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59575/Table_3.xlsx">Please click here to download this file.</a>

Watch this Scientific Journal Video about Using the Race Model Inequality to Quantify Behavioral Multisensory Integration Effects at JoVE.com

Using the Race Model Inequality to Quantify Behavioral Multisensory Integration Effects

The current study aims to provide a step-by-step tutorial for calculating the magnitude of multisensory integration effects in an effort to facilitate the production of translational research studies across diverse clinical populations.

Multisensory integration research investigates how the brain processes simultaneous sensory information. Research on animals (mainly cats and primates) ...

using-race-model-inequality-to-quantify-behavioral-multisensory

Nanyang Technological University

Research

JoVE Journal

Behavior

6.2K Views.  Albert Einstein College of Medicine. Het vaststellen van een reproduceerbare methode voor de berekening van de omvang van multisensorische integratie-effecten is aanzienlijk, omdat het zal helpen bij het vergemakkelijken van toekomstig translationeel onderzoek tussen verschillende klinische populaties. Het belangrijkste voordeel van onze techniek is dat we in staat zijn om een robuust fenotype van multisensorische integratie te kwantificeren die vervolgens wordt geassocieerd met belangrijke cognitieve en motorische resultaten bi...

Video: Using the Race Model Inequality to Quantify Behavioral Multisensory Integration Effects

Perceptual and Category Processing of the Uncanny Valley Hypothesis' Dimension of Human Likeness: Some Methodological Issues

Mori's Uncanny Valley Hypothesis1,2 proposes that the perception of humanlike characters such as robots and, by extension, avatars (computer-generated characters) can evoke negative or positive affect (valence) depending on the object's degree of visual and behavioral realism along a dimension of human likeness (DHL) (Figure 1). But studies of affective valence of subjective responses to variously realistic non-human characters have produced inconsistent findings 3, 4, 5, 6. One of a number of reasons for this is that human likeness is not perceived as the hypothesis assumes. While the DHL can be defined following Mori's description as a smooth linear change in the degree of physical humanlike similarity, subjective perception of objects along the DHL can be understood in terms of the psychological effects of categorical perception (CP) 7. Further behavioral and neuroimaging investigations of category processing and CP along the DHL and of the potential influence of the dimension's underlying category structure on affective experience are needed. This protocol therefore focuses on the DHL and allows examination of CP. Based on the protocol presented in the video as an example, issues surrounding the methodology in the protocol and the use in "uncanny" research of stimuli drawn from morph continua to represent the DHL are discussed in the article that accompanies the video. The use of neuroimaging and morph stimuli to represent the DHL in order to disentangle brain regions neurally responsive to physical human-like similarity from those responsive to category change and category processing is briefly illustrated.

Investigation of the Uncanny Valley Hypothesis and affective experience requires an understanding of the hypothesis' dimension of human likeness (DHL). This protocol allows representation of the DHL and examination of categorical perception. Use of the same stimuli and fMRI to distinguish brain regions responsive to physical and category change is illustrated.

Mori's Uncanny Valley Hypothesis1,2 proposes that the perception of humanlike characters such as robots and, by extension, avatars (computer-generated ...

The Crossmodal Congruency Task as a Means to Obtain an Objective Behavioral Measure in the Rubber Hand Illusion Paradigm

The rubber hand illusion (RHI) is a popular experimental paradigm. Participants view touch on an artificial rubber hand while the participants' own hidden hand is touched. If the viewed and felt touches are given at the same time then this is sufficient to induce the compelling experience that the rubber hand is one's own hand. The RHI can be used to investigate exactly how the brain constructs distinct body representations for one's own body. Such representations are crucial for successful interactions with the external world. To obtain a subjective measure of the RHI, researchers typically ask participants to rate statements such as "I felt as if the rubber hand were my hand". Here we demonstrate how the crossmodal congruency task can be used to obtain an objective behavioral measure within this paradigm.
The variant of the crossmodal congruency task we employ involves the presentation of tactile targets and visual distractors. Targets and distractors are spatially congruent (i.e. same finger) on some trials and incongruent (i.e. different finger) on others. The difference in performance between incongruent and congruent trials - the crossmodal congruency effect (CCE) - indexes multisensory interactions. Importantly, the CCE is modulated both by viewing a hand as well as the synchrony of viewed and felt touch which are both crucial factors for the RHI.
The use of the crossmodal congruency task within the RHI paradigm has several advantages. It is a simple behavioral measure which can be repeated many times and which can be obtained during the illusion while participants view the artificial hand. Furthermore, this measure is not susceptible to observer and experimenter biases. The combination of the RHI paradigm with the crossmodal congruency task allows in particular for the investigation of multisensory processes which are critical for modulations of body representations as in the RHI.

We demonstrate how an objective measure can be employed in the widely employed rubber hand illusion paradigm. This measure is obtained by modifying the well-established crossmodal congruency task. This task allows the investigation of multisensory processes which are critical for modulations of body representations as in the rubber hand illusion.

The rubber hand illusion (RHI) is a popular experimental paradigm. Participants view touch on an artificial rubber hand while the participants' own hidden ...

Testing Sensory and Multisensory Function in Children with Autism Spectrum Disorder

In addition to impairments in social communication and the presence of restricted interests and repetitive behaviors, deficits in sensory processing are now recognized as a core symptom in autism spectrum disorder (ASD). Our ability to perceive and interact with the external world is rooted in sensory processing. For example, listening to a conversation entails processing the auditory cues coming from the speaker (speech content, prosody, syntax) as well as the associated visual information (facial expressions, gestures). Collectively, the &#8220;integration&#8221; of these multisensory (i.e., combined audiovisual) pieces of information results in better comprehension. Such multisensory integration has been shown to be strongly dependent upon the temporal relationship of the paired stimuli. Thus, stimuli that occur in close temporal proximity are highly likely to result in behavioral and perceptual benefits &#8211; gains believed to be reflective of the perceptual system&#39;s judgment of the likelihood that these two stimuli came from the same source. Changes in this temporal integration are expected to strongly alter perceptual processes, and are likely to diminish the ability to accurately perceive and interact with our world. Here, a battery of tasks designed to characterize various aspects of sensory and multisensory temporal processing in children with ASD is described. In addition to its utility in autism, this battery has great potential for characterizing changes in sensory function in other clinical populations, as well as being used to examine changes in these processes across the lifespan.

We describe how to implement a battery of behavioral tasks to examine the processing and integration of sensory stimuli in children with ASD. The goal is to characterize individual differences in temporal processing of simple auditory and visual stimuli and relate these to higher order perceptual skills like speech perception.

In addition to impairments in social communication and the presence of restricted interests and repetitive behaviors, deficits in sensory processing are ...

An Operant Intra-/Extra-dimensional Set-shift Task for Mice

Alterations in executive control and cognitive flexibility, such as attentional set-shifting abilities, are core features of several neuropsychiatric diseases. The most widely used neuropsychological tests for the evaluation of attentional set-shifting in human subjects are the Wisconsin Card Sorting Test (WCST) and the CANTAB Intra-/Extra-dimensional set shift task (ID/ED). These tasks have proven clinical relevance and have been modified and successfully adapted for research in animal models. However, currently available tasks for rodents present several limitations, mainly due to their manual-based testing procedures, which are hampering translational advances in psychiatric medicine. To overcome these limitations and to better mimic the original version in primates, we present the development of a novel operant-based two-chamber ID/ED "Operon" task for rodents. We demonstrated the effectiveness of this novel task to measure different facets of cognitive flexibility in mice including attentional set formation and shifting, and reversal learning. Moreover, we show the high flexibility of this task in which three different perceptual dimensions can be manipulated with a high number of stimuli cues for each dimension. This novel ID/ED Operon task can be an effective preclinical tool for drug testing and/or large genetic screening relevant to the study of executive dysfunction and cognitive symptoms found in psychiatric disorders.

Attentional set-shifting cognitive abnormalities are a core feature of many psychiatric diseases. Here, we present the development of a novel automatic system for the effective study of attentional set-shifting abilities, executive functions and other cognitive abilities in mice with high translational validity to human studies.

Alterations in executive control and cognitive flexibility, such as attentional set-shifting abilities, are core features of several neuropsychiatric ...

Measuring Attention and Visual Processing Speed by Model-based Analysis of Temporal-order Judgments

This protocol describes how to conduct temporal-order experiments to measure visual processing speed and the attentional resource distribution. The proposed method is based on a new and synergistic combination of three components: the temporal-order judgments (TOJ) paradigm, Bundesen&#39;s Theory of Visual Attention (TVA), and a hierarchical Bayesian estimation framework. The method provides readily interpretable parameters, which are supported by the theoretical and neurophysiological underpinnings of TVA. Using TOJs, TVA-based estimates can be obtained for a broad range of stimuli, whereas traditional paradigms used with TVA are mainly limited to letters and digits. Finally, the meaningful parameters of the proposed model allow for the establishment of a hierarchical Bayesian model. Such a statistical model allows assessing results in one coherent analysis both on the subject and the group level.
To demonstrate the feasibility and versatility of this new approach, three experiments are reported with attention manipulations in synthetic pop-out displays, natural images, and a cued letter-report paradigm.

Temporal-order judgments can be used to estimate processing speed parameters and attentional weights and thereby to infer the mechanisms of attentional processing. This methodology can be applied to a wide range of visual stimuli and works with many attention manipulations.

This protocol describes how to conduct temporal-order experiments to measure visual processing speed and the attentional resource distribution. The ...

Cross-Modal Multivariate Pattern Analysis

Multivariate pattern analysis (MVPA) is an increasingly popular method of analyzing functional magnetic resonance imaging (fMRI) data1-4. Typically, the method is used to identify a subject's perceptual experience from neural activity in certain regions of the brain. For instance, it has been employed to predict the orientation of visual gratings a subject perceives from activity in early visual cortices5 or, analogously, the content of speech from activity in early auditory cortices6.
Here, we present an extension of the classical MVPA paradigm, according to which perceptual stimuli are not predicted within, but across sensory systems. Specifically, the method we describe addresses the question of whether stimuli that evoke memory associations in modalities other than the one through which they are presented induce content-specific activity patterns in the sensory cortices of those other modalities. For instance, seeing a muted video clip of a glass vase shattering on the ground automatically triggers in most observers an auditory image of the associated sound; is the experience of this image in the "mind's ear" correlated with a specific neural activity pattern in early auditory cortices? Furthermore, is this activity pattern distinct from the pattern that could be observed if the subject were, instead, watching a video clip of a howling dog?
In two previous studies7,8, we were able to predict sound- and touch-implying video clips based on neural activity in early auditory and somatosensory cortices, respectively. Our results are in line with a neuroarchitectural framework proposed by Damasio9,10, according to which the experience of mental images that are based on memories - such as hearing the shattering sound of a vase in the "mind's ear" upon seeing the corresponding video clip - is supported by the re-construction of content-specific neural activity patterns in early sensory cortices.

Classical multivariate pattern analysis predicts sensory stimuli a subject perceives from neural activity in the corresponding cortices (e.g. visual stimuli from activity in visual cortex). Here, we apply pattern analysis cross-modally and show that sound- and touch-implying visual stimuli can be predicted from activity in auditory and somatosensory cortices, respectively.

Multivariate pattern analysis (MVPA) is an increasingly popular method of analyzing functional magnetic resonance imaging (fMRI) data1-4. Typically, the ...

Neuroscience

MPI CyberMotion Simulator: Implementation of a Novel Motion Simulator to Investigate Multisensory Path Integration in Three Dimensions

Path integration is a process in which self-motion is integrated over time to obtain an estimate of one's current position relative to a starting point 1. Humans can do path integration based exclusively on visual 2-3, auditory 4, or inertial cues 5. However, with multiple cues present, inertial cues - particularly kinaesthetic - seem to dominate 6-7. In the absence of vision, humans tend to overestimate short distances (&lt;5 m) and turning angles (&lt;30&deg;), but underestimate longer ones 5. Movement through physical space therefore does not seem to be accurately represented by the brain.
Extensive work has been done on evaluating path integration in the horizontal plane, but little is known about vertical movement (see 3 for virtual movement from vision alone). One reason for this is that traditional motion simulators have a small range of motion restricted mainly to the horizontal plane. Here we take advantage of a motion simulator 8-9 with a large range of motion to assess whether path integration is similar between horizontal and vertical planes. The relative contributions of inertial and visual cues for path navigation were also assessed.
16 observers sat upright in a seat mounted to the flange of a modified KUKA anthropomorphic robot arm. Sensory information was manipulated by providing visual (optic flow, limited lifetime star field), vestibular-kinaesthetic (passive self motion with eyes closed), or visual and vestibular-kinaesthetic motion cues. Movement trajectories in the horizontal, sagittal and frontal planes consisted of two segment lengths (1st: 0.4 m, 2nd: 1 m; &plusmn;0.24 m/s2 peak acceleration). The angle of the two segments was either 45&deg; or 90&deg;. Observers pointed back to their origin by moving an arrow that was superimposed on an avatar presented on the screen.
Observers were more likely to underestimate angle size for movement in the horizontal plane compared to the vertical planes. In the frontal plane observers were more likely to overestimate angle size while there was no such bias in the sagittal plane. Finally, observers responded slower when answering based on vestibular-kinaesthetic information alone. Human path integration based on vestibular-kinaesthetic information alone thus takes longer than when visual information is present. That pointing is consistent with underestimating and overestimating the angle one has moved through in the horizontal and vertical planes respectively, suggests that the neural representation of self-motion through space is non-symmetrical which may relate to the fact that humans experience movement mostly within the horizontal plane.

An efficient way to gain insight into how humans navigate themselves in three dimensions is described. The method takes advantage of a motion simulator capable of moving observers in ways unattainable by traditional simulators. Results confirm that movement in the horizontal plane is underestimated, while vertical movement is overestimated.

Path integration is a process in which self-motion is integrated over time to obtain an estimate of one's current position relative to a starting point 1. ...

A Two-interval Forced-choice Task for Multisensory Comparisons

We provide a procedure for a psychophysics experiment in humans based on a previously described paradigm aimed to characterize the perceptual duration of intervals within the range of milliseconds of visual, acoustic, and audiovisual aperiodic trains of six pulses. In this task, each of the trials consists of two consecutive intramodal intervals where the participants press the upward arrow key to report that the second stimulus lasted longer than the reference, or the downward arrow key to indicate otherwise. The analysis of the behavior results in psychometric functions of the probability of estimating the comparison stimulus to be longer than the reference, as a function of the comparison intervals. In conclusion, we advance a way of implementing standard programming software to create visual, acoustic, and audiovisual stimuli, and to generate a two-interval forced-choice (2IFC) task by delivering stimuli through noise-blocking headphones and a computer&#39;s monitor.

Psychophysics is essential for studying perception phenomena through sensory information. Here we present a protocol to perform a two-interval forced-choice task as implemented in a previous report on human psychophysics where participants estimated the duration of visual, auditory, or audiovisual intervals of aperiodic trains of pulses.

We provide a procedure for a psychophysics experiment in humans based on a previously described paradigm aimed to characterize the perceptual duration of ...

Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios

In medicine or industry, the analysis of high-dimensional data sets is increasingly required. However, available technical solutions are often complex to use. Therefore, new approaches like immersive analytics are welcome. Immersive analytics promise to experience high-dimensional data sets in a convenient manner for various user groups and data sets. Technically, virtual-reality devices are used to enable immersive analytics. In Industry 4.0, for example, scenarios like the identification of outliers or anomalies in high-dimensional data sets are pursued goals of immersive analytics. In this context, two important questions should be addressed for any developed technical solution on immersive analytics: First, is the technical solutions being helpful or not? Second, is the bodily experience of the technical solution positive or negative? The first question aims at the general feasibility of a technical solution, while the second one aims at the wearing comfort. Extant studies and protocols, which systematically address these questions are still rare. In this work, a study protocol is presented, which mainly investigates the usability for immersive analytics in Industry 4.0 scenarios. Specifically, the protocol is based on four pillars. First, it categorizes users based on previous experiences. Second, tasks are presented, which can be used to evaluate the feasibility of the technical solution. Third, measures are presented, which quantify the learning effect of a user. Fourth, a questionnaire&#160;evaluates the stress level when performing tasks. Based on these pillars, a technical setting was implemented that uses mixed reality smartglasses to apply the study protocol. The results of the conducted study show the applicability of the protocol on the one hand and the feasibility of immersive analytics in Industry 4.0 scenarios on the other. The presented protocol includes a discussion of discovered limitations.

This protocol delineates the technical setting of a developed mixed reality application that is used for immersive analytics. Based on this, measures are presented, which were used in a study to gain insights into usability aspects of the developed technical solution.

In medicine or industry, the analysis of high-dimensional data sets is increasingly required. However, available technical solutions are often complex to ...

Engineering

Behavioral Assays for Autism Spectrum Disorder in Mouse Models

Strategies for Assessing Autistic-Like Behaviors in Mice

Autism spectrum disorder (ASD) is a neurobiologically complex condition with a heterogeneous genetic etiology. Clinically, ASD is diagnosed by social communication impairments and restrictive or repetitive behaviors, such as hand flapping or lining up objects. These behavioral patterns can be reliably observed in mouse models with ASD-linked genetic mutations, making them highly useful tools for studying the underlying cellular and molecular mechanisms in ASD. Understanding how genetic changes affect the neurobiology and behaviors observed in ASD will facilitate the development of novel targeted therapeutic compounds to ameliorate core behavioral impairments. Our lab has employed several protocols encompassing well-described training and testing procedures that reflect a wide range of behavioral deficits related to ASD. Here, we detail two assays to study the core features of ASD in mouse models: self-grooming (a measure of repetitive behavior) and the three-chamber social interaction test (a measure of social interaction approach and preference for social novelty).

Rodent models are valuable tools for studying core behaviors related to autism spectrum disorder (ASD). In this article, we expound on two behavioral tests for modeling the core features of ASD in mice: self-grooming, which assesses repetitive behavior, and the three-chamber social interaction test, which documents social impairments.

Autism spectrum disorder (ASD) is a neurobiologically complex condition with a heterogeneous genetic etiology. Clinically, ASD is diagnosed by social ...

Using the Race Model Inequality to Quantify Behavioral Multisensory Integration Effects

Samenvatting

Meer video's verkennen

Hoofdstukken in deze video