Name: evaluating usability aspects of a mixed reality solution for immersive analytics in industry 4.0 scenarios
Uploaded: 2020-10-06T13:00:00.000+00:00
Duration: 6 min 2 s
Description: Watch this Scientific Journal Video about Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios at JoVE.com

The authors have nothing to acknowledge.

All materials and methods were approved by the Ethics Committee of Ulm University, and were carried out in accordance with the approved guidelines. All participants gave their written informed consent.
1. Establish Appropriate Study Environment
NOTE: The study was conducted in a controlled environment&#160;to cope with the complex hardware setting. The used mixed reality smartglasses (see Table of Materials) and the laptop for the 2D application were&#160;explained to the study participants.
<ol>
	<li>Check the technical solution before each participant; set in default mode. Prepare the questionnaires and place next to a participant.</li>
	<li>Let participants solve tasks from the use cases outlier detection and cluster recognition in one session (i.e., average time was 43 min).</li>
	<li>Start the study by welcoming the participants and introducing the goal of the study, as well as the overall procedure.</li>
	<li>Participants using the skin conductance measurement device (see Table of Materials) must adhere to a short resting phase, to receive a baseline measurement. Only half of the participants used this device.</li>
	<li>All participants have to fill out the State-Trait Anxiety Inventory (STAI) questionnaire31, prior to the start of the experiment.
	<ol>
		<li>Next,&#160;participants have to perform the mental rotation test (see Figure 4, this test evaluated the spatial imagination abilities), which was the basis to distinguish high from low performers (high performers are advanced users, while low performers are novices), followed by the spatial sound test to measure spatial hearing abilities of a participant. 
		NOTE: A median split of the test scores in the mental rotation test32 was used to distinguish low from high performers.</li>
	</ol>
	</li>
	<li>Randomly separate participants into two groups;&#160;either start with the task on outlier detection or cluster recognition, while continuing with the other use case afterwards. For the cluster recognition task, half of the participants firstly started with the used mixed reality smartglasses (see Table of Materials), and then used the 2D application, while the other half firstly started with the 2D application, and then used the mixed reality smartglasses (see Table of Materials).&#160;For the outlier&#160;detection task, randomly select one group which receives&#160;sound support, while the other part of the group receives&#160;no sound support.&#160;</li>
	<li>Conclude&#160;the session,&#160;participants have to answer the State-Trait Anxiety Inventory (STAI) questionnaire31 again, as well as the self-developed, and a demographic questionnaire.&#160;</li>
	<li>Store the generated data, which was automatically recorded by each developed application, on the laptop&#39;s storage after the session was accomplished.</li>
</ol>
2. Study Protocol for Participants
<ol>
	<li>Prepare the experiment (see Figure 2 for the room of the experiment) for each participant. Present the desktop PC, the used mixed reality smartglasses, and hand out the questionnaires.</li>
	<li>Inform the participants that the experiment will take 40 to 50 minutes, and that half of them start after the pretests (see Points 3-6 of Study Protocol) firstly with the outlier detection test (see Point 7 of Study Protocol), followed by the cluster recognition test (see Point 8 of Study Protocol), while the others accomplish these two tests vice versa (i.e., Point 8 of the Study Protocol before Point 7).</li>
	<li>Decide randomly whether a skin conductance measurement is done. In case of yes, prepare the skin conductance measurement device33 and inform the participant to put on the device. Request a short resting phase from participants to receive a baseline measurement for their stress level.</li>
	<li>Request participants to fill out the State-Trait Anxiety Inventory (STAI) questionnaire31 and inform them that it measures the current perceived stress before&#160;the experiment.</li>
	<li>Conduct a mental rotation test.
	<ol>
		<li>Inform participants that their mental rotation capabilities are evaluated and usher them in front of a desktop computer. Inform participants about the test procedure. Note that they had to identify similar objects that had different positions in a simulated 3D space.</li>
		<li>Inform participants that only two of the five shown objects are similar and that they will have 2 minutes for the entire test. Inform participants that seven tasks could be accomplished within the given 2 minutes and tell them that performance measures are recorded for each accomplished task.</li>
	</ol>
	</li>
	<li>Evaluate spatial sound abilities.
	<ol>
		<li>Inform participants that their spatial sound abilities are evaluated and usher them in front of a desktop computer. Inform participants about the test procedure. Explain to participants that six sound samples must be detected, which will be played for 13 seconds each.</li>
		<li>Inform participants that they have to detect the direction (analogously to the four compass directions) of which the sound is coming from.</li>
	</ol>
	</li>
	<li>Evaluate outlier detection skills.
	<ol>
		<li>Request participants to put on the mixed reality smartglasses. Explain to them that outliers must be found within the world created for the mixed reality smartglasses.</li>
		<li>Further inform them that an outlier is a red-marked point, all other points are white-marked. Explain then to them that they must direct their gaze to the red-colored point to detect it.</li>
		<li>Further inform the participants that not only visual help is provided, additionally environmental sounds support them to find outliers. Provide the information to the participants that they have to accomplish 8 outlier tasks, meaning that 8 times within the virtual world, the red-colored point has to be found.&#160;For each participant, 4 tasks are sound-supported, while 4 tasks are sound-unsupported. For each participant, it is randomly selected whether they start a task sound-supported or not. Then, dependent from the first task, it changes from task to task whether sound support is provided or not.</li>
		<li>Tell participants which information will be recorded: required time for each task, length of walking, and how their final moving position is looking like related to their starting position. Finally tell participants that the red-marked point changes to green if it was detected (see Figure 3).</li>
	</ol>
	</li>
	<li>Evaluate cluster recognition skills.
	<ol>
		<li>Randomly decide for the&#160;participant whether firstly to use the mixed reality smartglasses or to usher the participant to a desktop computer.&#160;In the following, only the procedure for the mixed reality setting is described. If a participant firstly starts with the desktop computer, the procedure is the same in changed order and except the voice commands, they are only provided when using the mixed reality solution.</li>
		<li>For participants using mixed reality:&#160;Request participants to put on the mixed reality smartglasses. Inform participants how to find clusters within the world created with the used mixed reality smartglasses. Emphasize to the participants that they had to distinguish between overlapping clusters by moving around them.</li>
		<li>For participants using mixed reality:&#160;Explain to participants that they can navigate in the virtual world and around the clusters using voice commands. Finally tell participants that they had to detect six clusters.</li>
		<li>For participants using mixed reality:&#160;Request participants to remove the used mixed reality smartglasses. Usher participants to a desktop computer and tell them to use the software shown on the screen of the desktop computer. Inform them that the same type of clusters like shown in the used mixed reality smartglasses had to be detected using the software on the desktop computer (see Figure 7 and Figure 8).</li>
	</ol>
	</li>
	<li>Request participants to fill out three questionnaires, namely the State-Trait Anxiety Inventory (STAI) questionnaire31, a self-developed questionnaire to gather subjective feedback, and a demographic questionnaire to gather information about them.</li>
	<li>Request participants to remove the skin conductance measurement device33 if they were requested in the beginning to put it on.</li>
	<li>Relieve participants from the experiment by saying thanks for the participation.</li>
</ol>

<ol>
	<li>Korinth, M., Sommer-Dittrich, T., Reichert, M., Pryss, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Design+and+Evaluation+of+a+Virtual+Reality-Based+Car+Configuration+Concept.">Design and Evaluation of a Virtual Reality-Based Car Configuration Concept.</a> Science and Information Conference. , 169-189 (2019).</li><li>Whalen, T. E., No&#235;l, S., Stewart, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Measuring+the+human+side+of+virtual+reality.">Measuring the human side of virtual reality.</a> IEEE International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems, 2003. , 8-12 (2003).</li><li>Martens, M. A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=It+feels+real:+physiological+responses+to+a+stressful+virtual+reality+environment+and+its+impact+on+working+memory.">It feels real: physiological responses to a stressful virtual reality environment and its impact on working memory.</a> Journal of Psychopharmacology. 33 (10), 1264-1273 (2019).</li><li>Keim, D. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Information+visualization+and+visual+data+mining.">Information visualization and visual data mining.</a> IEEE transactions on Visualization and Computer Graphics. 8 (1), 1-8 (2002).</li><li>Dwyer, T., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Immersive+analytics:+An+introduction.">Immersive analytics: An introduction.</a> Immersive analytics. , 1-23 (2018).</li><li>Moloney, J., Spehar, B., Globa, A., Wang, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+affordance+of+virtual+reality+to+enable+the+sensory+representation+of+multi-dimensional+data+for+immersive+analytics:+from+experience+to+insight.">The affordance of virtual reality to enable the sensory representation of multi-dimensional data for immersive analytics: from experience to insight.</a> Journal of Big Data. 5 (1), 53 (2018).</li><li>Davis, F. D., Riedl, R., Vom Brocke, J., L&#233;ger, P. M., Randolph, A. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Information Systems and Neuroscience. , (2018).</li><li>Huckins, J. F., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Fusing+mobile+phone+sensing+and+brain+imaging+to+assess+depression+in+college+students.">Fusing mobile phone sensing and brain imaging to assess depression in college students.</a> Frontiers in Neuroscience. 13, 248 (2019).</li><li>Preece, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Human-computer interaction. , (1994).</li><li>Card, S. K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> The psychology of human-computer interaction. , (2018).</li><li>Pelayo, S., Senathirajah, Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Human+factors+and+sociotechnical+issues.">Human factors and sociotechnical issues.</a> Yearbook of Medical Informatics. 28 (01), 078-080 (2019).</li><li>Bangor, A., Kortum, P., Miller, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Determining+what+individual+SUS+scores+mean:+adding+an+adjective+rating+scale.">Determining what individual SUS scores mean: adding an adjective rating scale.</a> Journal of Usability Studies. 4 (3), 114-123 (2009).</li><li>Krahmer, E., Ummelen, N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Thinking+about+thinking+aloud:+A+comparison+of+two+verbal+protocols+for+usability+testing.">Thinking about thinking aloud: A comparison of two verbal protocols for usability testing.</a> IEEE Transactions on Professional Communication. 47 (2), 105-117 (2004).</li><li>Hornb&#230;k, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Current+practice+in+measuring+usability:+Challenges+to+usability+studies+and+research.">Current practice in measuring usability: Challenges to usability studies and research.</a> International Journal of Human-Computer Studies. 64 (2), 79-102 (2006).</li><li>Peppa, V., Lysikatos, S., Metaxas, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Human-Computer+interaction+and+usability+testing:+Application+adoption+on+B2C+websites.">Human-Computer interaction and usability testing: Application adoption on B2C websites.</a> Global Journal of Engineering Education. 14 (1), 112-118 (2012).</li><li>Alwashmi, M. F., Hawboldt, J., Davis, E., Fetters, M. D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+iterative+convergent+design+for+mobile+health+usability+testing:+mixed-methods+approach.">The iterative convergent design for mobile health usability testing: mixed-methods approach.</a> JMIR mHealth and uHealth. 7 (4), 11656 (2019).</li><li>System Usability Scale (SUS). Assistant Secretary for Public Affairs Available from: <a target="_blank" href="https://www.hhs.gov/about/agencies/aspa/how-to-and-tools/methods/system-usability-scale.html">https://www.hhs.gov/about/agencies/aspa/how-to-and-tools/methods/system-usability-scale.html</a> (2013)</li><li>Fang, Y. M., Lin, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+Usability+Testing+of+VR+Interface+for+Tourism+Apps.">The Usability Testing of VR Interface for Tourism Apps.</a> Applied Sciences. 9 (16), 3215 (2019).</li><li>Pryss, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Exploring+the+Time+Trend+of+Stress+Levels+While+Using+the+Crowdsensing+Mobile+Health+Platform,+TrackYourStress,+and+the+Influence+of+Perceived+Stress+Reactivity:+Ecological+Momentary+Assessment+Pilot+Study.">Exploring the Time Trend of Stress Levels While Using the Crowdsensing Mobile Health Platform, TrackYourStress, and the Influence of Perceived Stress Reactivity: Ecological Momentary Assessment Pilot Study.</a> JMIR mHealth and uHealth. 7 (10), 13978 (2019).</li><li>Zugal, S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Investigating+expressiveness+and+understandability+of+hierarchy+in+declarative+business+process+models.">Investigating expressiveness and understandability of hierarchy in declarative business process models.</a> Software &amp; Systems Modeling. 14 (3), 1081-1103 (2015).</li><li>Schobel, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Learnability+of+a+configurator+empowering+end+users+to+create+mobile+data+collection+instruments:+usability+study.">Learnability of a configurator empowering end users to create mobile data collection instruments: usability study.</a> JMIR mHealth and uHealth. 6 (6), 148 (2018).</li><li>Schobel, J., Probst, T., Reichert, M., Schickler, M., Pryss, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Enabling+Sophisticated+Lifecycle+Support+for+Mobile+Healthcare+Data+Collection+Applications.">Enabling Sophisticated Lifecycle Support for Mobile Healthcare Data Collection Applications.</a> IEEE Access. 7, 61204-61217 (2019).</li><li>Hoppenstedt, B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Dimensionality+Reduction+and+Subspace+Clustering+in+Mixed+Reality+for+Condition+Monitoring+of+High-Dimensional+Production+Data.">Dimensionality Reduction and Subspace Clustering in Mixed Reality for Condition Monitoring of High-Dimensional Production Data.</a> Sensors. 19 (18), 3903 (2019).</li><li>Winter, M., Pryss, R., Probst, T., Reichert, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Towards+the+Applicability+of+Measuring+the+Electrodermal+Activity+in+the+Context+of+Process+Model+Comprehension:+Feasibility+Study.">Towards the Applicability of Measuring the Electrodermal Activity in the Context of Process Model Comprehension: Feasibility Study.</a> Sensors. 20, 4561 (2020).</li><li>Butscher, S., Hubenschmid, S., M&#252;ller, J., Fuchs, J., Reiterer, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Clusters,+trends,+and+outliers:+How+immersive+technologies+can+facilitate+the+collaborative+analysis+of+multidimensional+data.">Clusters, trends, and outliers: How immersive technologies can facilitate the collaborative analysis of multidimensional data.</a> Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. , 1-12 (2018).</li><li>Wagner Filho, J. A., Rey, M. F., Freitas, C. M. D. S., Nedel, L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Immersive+analytics+of+dimensionally-reduced+data+scatterplots.">Immersive analytics of dimensionally-reduced data scatterplots.</a> 2nd Workshop on Immersive Analytics. , (2017).</li><li>Batch, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=There+is+no+spoon:+Evaluating+performance,+space+use,+and+presence+with+expert+domain+users+in+immersive+analytics.">There is no spoon: Evaluating performance, space use, and presence with expert domain users in immersive analytics.</a> IEEE Transactions on Visualization and Computer Graphics. 26 (1), 536-546 (2019).</li><li>Towards hmd-based immersive analytics. HAL Available from: <a target="_blank" href="https://hal.archives-ouvertes.fr/hal-01631306">https://hal.archives-ouvertes.fr/hal-01631306</a> (2017)</li><li>Luboschik, M., Berger, P., Staadt, O. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=On+spatial+perception+issues+in+augmented+reality+based+immersive+analytics.">On spatial perception issues in augmented reality based immersive analytics.</a> Proceedings of the 2016 ACM Companion on Interactive Surfaces and Spaces. , 47-53 (2016).</li><li>Fonnet, A., Pri&#233;, Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Survey+of+Immersive+Analytics.">Survey of Immersive Analytics.</a> IEEE Transactions on Visualization and Computer. , (2019).</li><li>Spielberger, C. D., Gorsuch, R. L., Lushene, R. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=STAI+Manual+for+the+Stait-Trait+Anxiety+Inventory+(self-evaluation+questionnaire).">STAI Manual for the Stait-Trait Anxiety Inventory (self-evaluation questionnaire).</a> Consulting Psychologist. 22, 1-24 (1970).</li><li>Vandenberg, S. G., Kuse, A. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Mental+rotations,+a+group+test+of+three-dimensional+spatial+visualization.">Mental rotations, a group test of three-dimensional spatial visualization.</a> Perceptual Motor Skills. 47 (2), 599-604 (1978).</li><li>H&#228;rtel, S., Gnam, J. P., L&#246;ffler, S., B&#246;s, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Estimation+of+energy+expenditure+using+accelerometers+and+activity-based+energy+models-Validation+of+a+new+device.">Estimation of energy expenditure using accelerometers and activity-based energy models-Validation of a new device.</a> European Review of Aging and Physical Activity. 8 (2), 109-114 (2011).</li><li>. RPY2: A Simple and Efficient Access to R from Python Available from: <a target="_blank" href="https://sourceforge.net/projects/rpy/">https://sourceforge.net/projects/rpy/</a> (2020)</li><li>Hoppenstedt, B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Applicability+of+immersive+analytics+in+mixed+reality:+Usability+study.">Applicability of immersive analytics in mixed reality: Usability study.</a> IEEE Access. 7, 71921-71932 (2019).</li><li>Hoppenstedt, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Applicability+of+Immersive+Analytics+in+Mixed+Reality:+Usability+Study.">Applicability of Immersive Analytics in Mixed Reality: Usability Study.</a> IEEE Dataport. , (2019).</li></ol>

The authors have nothing to disclose.

Regarding the developed mixed reality smartglasses (see Table of Materials) application, two aspects were particularly beneficial. The use of spatial sounds for the outlier&#8217;s detection task was positively perceived on one hand (see the results of RQ3). On the other, the use of voice commands was also perceived positively (see Figure 10).
Regarding the study participants, although the number of recruited participants was rather small for an empirical study, the number is competitive compared to many other works. Nevertheless, a larger-scale study is planned based on the shown protocol. However, as it showed its feasibility for 60 participants, more participants are expected to reveal no further challenges. It was discussed that the selection of participants could be broader (in the sense of the fields the participants are coming from) and that the number of baseline variables to distinguish between high and low performers could be higher. On the other, if these aspects are changed to higher numbers, the protocol itself has not to be changed profoundly.
In general, the revealed limitations do not affect the conduction of a study based on the protocol shown in this work, they only affect the recruitment and the used questions for the demographic questionnaire. However, one limitation of this study is nevertheless important: the overall required time to finish the experiment for one participant is high. On the other hand, as the participants did not complain about the wearing comfort, or that the test device is burdening them too much, the time of conducting the overall protocol for one participant can be considered to be acceptable. Finally, in a future experiment, several aspects have to be added to the protocol. In particular, the outlier detection task should also be evaluated in the 2D desktop application. Furthermore, other hardware devices like the used mixed reality smartglasses (see Table of Materials) must be also evaluated. However, the protocol seems to be beneficial in a broader sense.
The following major insights were gained for the presented protocol. First, it showed its feasibility for evaluating immersive analytics for a mixed-reality solution. Specifically, the used mixed reality smartglasses (see Table of Materials) revealed their feasibility to evaluate immersive analytics in a mixed-reality application for Industry 4.0 scenarios. Second, the comparison of the developed used mixed reality smartglasses (see Table of Materials) application with a 2D desktop application was helpful to investigate whether the mixed-reality solution can outperform an application that does not make use of VR techniques. Third, the measurement of physiological parameters or vital signs should be always considered in such experiments. In this work, stress was measured using a questionnaire and a skin conductance device. Although the latter worked technically properly, the authors revealed in another study with the same device that additional considerations are required24. Fourth, the spatial ability test and the separation of high and low performers was advantageous. In summary, although the presented protocol seems to be complex at a first glance (see Figure 9), it showed its usefulness technically. Regarding the results, it also revealed its usefulness.
As the detection of outliers and the recognition of clusters are typical tasks in the evaluation of many high-dimensional data sets in Industry 4.0 scenarios, their use in an empirical study is representative for this field of research. The protocol showed that these scenarios can be well-integrated in a usability study on immersive analytics. Therefore, the used setting can be recommended for other studies in this context.
As the outcome of the shown study showed that the use of a mixed-reality solution based on the utilized&#160;smartglasses (see Table of Materials) is useful to investigate immersive analytics for Industry 4.0 scenarios, the protocol might be used for other usability studies in the given context as well.

Virtual-reality solutions (VR solutions) are increasingly important in different fields. Often, with VR solutions (including Virtual Reality, Mixed Reality, and Augmented Reality), the accomplishment of many daily tasks and procedures shall be eased. For example, in the automotive domain, the configuration procedure of a car can be supported by the use of Virtual Reality1 (VR). Researchers and practitioners have investigated and developed a multitude of approaches and solutions in this context. However, studies that investigate usability aspects are still rare. In general, the aspects should be considered in the light of two major questions. First, it must be evaluated whether a VR solution actually outperforms an approach that does not make use of VR techniques. Second, as VR solutions are mainly relying on heavy and complex hardware devices, parameters like the wearing comfort and mental effort should be investigated more in-depth. In addition, the mentioned aspects should always be investigated with respect to the application field in question. Although many extant approaches see the needs to investigate these questions2, less studies exist that have presented results.
A research topic in the field of VR, which is currently important, is denoted with immersive analytics. It is derived from the research field of visual analytics, which tries to include the human perception into analytics tasks. This process is also well-known as visual data mining4. Immersive analytics includes topics from the fields of data visualization, visual analytics, virtual reality, computer graphics, and human-computer interaction5. Recent advantages in head-mounted displays (HMD) led to improved possibilities for exploring data in an immersive way. Along these trends, new challenges and research questions emerge, like the development of new interaction systems, the need to investigate user fatigue, or the development of sophisticated 3D visualizations6. In a previous publication6, important principles of immersive analytics are discussed. In the light of big data, methods like immersive analytics are more and more needed to enable a better analysis of complex data pools. Only a few studies exist that investigate usability aspects of immersive analytics solutions. Furthermore, the domain or field in question should also be considered in such studies. In this work, an immersive analytics prototype was developed, and based on that, a protocol, which investigates the developed solution for Industry 4.0 scenarios. The protocol thereby exploits the experience method2, which is based on subjective, performance, and physiological aspects. In the protocol at hand, the subjective aspects are measured through perceived stress of the study users. Performance, in turn, is measured through the required time and errors that are made to accomplish analysis tasks. Finally, a skin conductance sensor measured physiological parameters. The first two measures will be presented in this work, while the measured skin conductance requires further efforts to be evaluated.
The presented study involves several research fields, particularly including neuroscience aspects and information systems. Interestingly, considerations on neuroscience aspects of information systems have recently garnered attention of&#160;several research groups7,8, showing the demand to explore the use of IT systems also from a cognitive viewpoint. Another field that is relevant for this work constitutes the investigation of human factors of information systems9,10,11. In the field of human-computer interaction, instruments exist to investigate the usability of a solution. Note that the System Usability Scale is mainly used in this context12. Thinking Aloud Protocols13 are another widely used study technique to learn more about the use of information systems. Although many approaches exist to measure usability aspects of information systems, and some of them have been presented&#160;long ago14, still questions emerge that require to investigate new measures or study methods. Therefore, research in this field is very active12,15,16.
In the following, the reasons will be discussed why two prevalently used methods have not been considered in the current&#160;work. First, the System Usability Scale was not used. The scale is based on ten questions17 and its use can be found in several other VR studies18 as well. As this study mainly aims at the measurement of stress19, a stress-related questionnaire was more appropriate. Second, no Thinking Aloud Protocol20 was used. Although this protocol type has shown its usefulness in general13, it was not used here as the stress level of study users might increase only due to the fact that the think aloud session must be accomplished in parallel to the use of a heavy and complex VR device. Although these two techniques have not been used, results of other recent studies have been incorporated in the study at hand. For example, in previous works21,22, the authors distinguish between novices and experts in their studies. Based on the successful outcome of these studies, the&#160;protocol at hand utilizes this presented separation of study users. The stress measurement, in turn, is based on ideas of the following works15,19,21,22.
At first, for conducting the study, a suitable Industry 4.0 scenario must be found for accomplishing analytical&#160;tasks. Inspired by another work of the authors23, two scenarios (i.e., the analysis tasks) have been identified, (1) Detection of Outliers, and (2) Recognition of Clusters. Both scenarios are challenging, and are highly relevant in the context of the maintenance of high-throughput production machines. Based on this decision, six major considerations have driven the study protocol presented in this work:
<ol>
	<li>The solution developed for the study will be technically based on mixed reality smartglasses (see Table of Materials) and will be developed as a mixed reality application.</li>
	<li>A suitable test must be developed, which is able to distinguish novices from advanced users.</li>
	<li>Performance measures should&#160;consider&#160;time and errors.</li>
	<li>A desktop application must be developed, which can be compared to the immersive analytics solution.</li>
	<li>A measure&#160;must be applied to evaluate the perceived stress level.</li>
	<li>In addition to the latter point, features shall be developed to mitigate the stress level while a user accomplishes the procedure of the two mentioned analysis tasks&#160;(i.e., (1) Detection of Outliers, and (2) Recognition of Clusters).</li>
</ol>
Based on the six mentioned points, the study protocol incorporates the following procedure. Outlier Detection and Cluster Recognition Analysis tasks have to be accomplished in an immersive way using mixed reality smartglasses (see Table of Materials). Therefore, a new application was developed. Spatial sounds shall ease the performing of&#160;analysis tasks without increasing the mental effort. A voice feature shall ease the navigation used for&#160;the developed application of&#160;the mixed reality smartglasses (see Table of Materials). A mental rotation test shall be the basis to distinguish novices from advanced users. The stress level is measured based on a questionnaire. Performance, in turn, is evaluated based on the (1) time a user requires for the analysis tasks, and based on the (2) errors that were made by a user for the analysis tasks. The performance in mixed reality smartglass is compared with the accomplishment of the same tasks in a newly developed and comparable 2D desktop application. In addition, a skin conductance device is used to measure the skin conductance level as a possible indicator for stress. Results to this measurement are subject to further analysis and will not be discussed in this work. The authors revealed in another study with the same device that additional considerations are required24.
Based on this protocol, the following five research questions (RQs) are addressed:
RQ1: Do spatial imagination abilities of the participants affect the performance of tasks significantly? 
RQ2: Is there a significant change of task performance over time? 
RQ3: Is there a significant change of task performance when using spatial sounds in the immersive analytics solution? 
RQ4: Is the developed immersive analytics perceived stressful by the users? 
RQ5: Do users perform better when using an immersive analytics solution compared to an 2D approach?
Figure 1 summarizes the presented protocol with respect to two scales. It shows the developed and used measures and their novelty with respect to the level of interaction. As the interaction level constitutes an important aspect when developing features for a VR setting, Figure 1 shall better show the novelty of the entire protocol developed in this work. Although the evaluation of the aspects within the two used scales is subjective, their overall evaluation is based on the current related work and the following major considerations: One important principle constitutes the use of abstractions of an environment for a natural interaction, in which the user has become attuned to. With respect to the protocol at hand, the visualization of point clouds seems to be intuitive for users and the recognition of patterns in such clouds has been recognized as a manageable task in general. Another important principle constitutes to overlay affordances. Hereby, the use of spatial sounds as used in the protocol at hand is an example, as they correlate with the proximity of a searched object. The authors recommend to tune the representations in a way that most information is located in the intermediate zone, which is most important for human perception. The reason why the authors did not include this principle was to encourage the user to find the best spot by themselves as well as to try to orientate themselves in a data visualization space, which is too large to be shown at once. In the presented approach, no further considerations of the characteristics of the 3D data to be shown were made. For example, if a dimension is assumed to be temporal, scatterplots could have been shown. The authors consider this kind of visualization generally interesting in the context of Industry 4.0. However, it has to been focused on a reasonably small set of visualizations. Moreover, a previous publication already focused on the collaborative analysis of data. In this work, this research question was excluded due to complexity of the other addressed issues in this study. In the presented setup here, the user is able to explore the immersive space by walking around. Other approaches offer controllers to explore the virtual space. In this study, the focus is set on the usability by using the System Usability Scale (SUS). Another previous publication has conducted a study for economic experts, but with VR headsets. In general, and most importantly, this study complains about the limited field of view for other devices like the used mixed reality smartglasses in this work (see Table of Materials). Their findings show that beginners in the field of VR were able to use the analytic tool efficiently. This matches with the experiences of this study, although in this work beginners were not classified to have VR or gaming experiences. In contrast to most VR solutions, mixed reality is not fixed to a position as it allows to track the real environment. VR approaches such as mention the use of special chairs for a 360&#176; experience to free the user from his desktop. The authors of indicate that perception issues influence the performance of immersive analytics; for example, by using shadows. For the study at hand, this is not feasible, as the used mixed reality smartglasses (see table of materials) are not able to display shadows. A workaround could be a virtual floor, but such a setup was out of the scope of this study. A survey study in the field of immersive analytics identified 3D scatterplots as one of the most common representations of multi-dimensional data. Altogether, the aspects shown in Figure 1 cannot be found currently compiled to a protocol that investigates usability aspects of immersive analytics for Industry 4.0 scenarios.

<table><tbody><tr><td>edaMove</td><td>movisens</td><td></td><td></td></tr><tr><td>HoloLens</td><td>Microsoft</td><td></td><td></td></tr><tr><td>Matlab R2017a</td><td>MathWorks</td><td></td><td></td></tr><tr><td>RPY2</td><td>GNU General Public License v2 or later (GPLv2+) (GPLv2+)</td><td></td><td>https://pypi.org/project/rpy2/</td></tr><tr><td>SPSS 25.0</td><td>IBM</td><td></td><td></td></tr></tbody></table>

evaluating usability aspects of a mixed reality solution for immersive analytics in industry 4.0 scenarios

In medicine or industry, the analysis of high-dimensional data sets is increasingly required. However, available technical solutions are often complex to use. Therefore, new approaches like immersive analytics are welcome. Immersive analytics promise to experience high-dimensional data sets in a convenient manner for various user groups and data sets. Technically, virtual-reality devices are used to enable immersive analytics. In Industry 4.0, for example, scenarios like the identification of outliers or anomalies in high-dimensional data sets are pursued goals of immersive analytics. In this context, two important questions should be addressed for any developed technical solution on immersive analytics: First, is the technical solutions being helpful or not? Second, is the bodily experience of the technical solution positive or negative? The first question aims at the general feasibility of a technical solution, while the second one aims at the wearing comfort. Extant studies and protocols, which systematically address these questions are still rare. In this work, a study protocol is presented, which mainly investigates the usability for immersive analytics in Industry 4.0 scenarios. Specifically, the protocol is based on four pillars. First, it categorizes users based on previous experiences. Second, tasks are presented, which can be used to evaluate the feasibility of the technical solution. Third, measures are presented, which quantify the learning effect of a user. Fourth, a questionnaire&#160;evaluates the stress level when performing tasks. Based on these pillars, a technical setting was implemented that uses mixed reality smartglasses to apply the study protocol. The results of the conducted study show the applicability of the protocol on the one hand and the feasibility of immersive analytics in Industry 4.0 scenarios on the other. The presented protocol includes a discussion of discovered limitations.

Watch this Scientific Journal Video about Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios at JoVE.com

Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios

This protocol delineates the technical setting of a developed mixed reality application that is used for immersive analytics. Based on this, measures are presented, which were used in a study to gain insights into usability aspects of the developed technical solution.

In medicine or industry, the analysis of high-dimensional data sets is increasingly required. However, available technical solutions are often complex to ...

evaluating-usability-aspects-mixed-reality-solution-for-immersive

Nanyang Technological University

Research

JoVE Journal

Engineering

2.2K Views.  Ulm University. Dit protocol beschrijft de technische setting van een ontwikkelde mixed reality-applicatie die wordt gebruikt voor immersive analytics. Op basis hiervan worden metingen gepresenteerd, die in een onderzoek zijn gebruikt om inzicht te krijgen in bruikbaarheidsaspecten van de ontwikkelde technische oplossing.

Video: Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios

Novel 3D/VR Interactive Environment for MD Simulations, Visualization and Analysis

The increasing development of computing (hardware and software) in the last decades has impacted scientific research in many fields including materials science, biology, chemistry and physics among many others. A new computational system for the accurate and fast simulation and 3D/VR visualization of nanostructures is presented here, using the open-source molecular dynamics (MD) computer program LAMMPS. This alternative computational method uses modern graphics processors, NVIDIA CUDA technology and specialized scientific codes to overcome processing speed barriers common to traditional computing methods. In conjunction with a virtual reality system used to model materials, this enhancement allows the addition of accelerated MD simulation capability. The motivation is to provide a novel research environment which simultaneously allows visualization, simulation, modeling and analysis. The research goal is to investigate the structure and properties of inorganic nanostructures (e.g., silica glass nanosprings) under different conditions using this innovative computational system. The work presented outlines a description of the 3D/VR Visualization System and basic components, an overview of important considerations such as the physical environment, details on the setup and use of the novel system, a general procedure for the accelerated MD enhancement, technical information, and relevant remarks. The impact of this work is the creation of a unique computational system combining nanoscale materials simulation, visualization and interactivity in a virtual environment, which is both a research and teaching instrument at UC Merced.

A new computational system featuring GPU-accelerated molecular dynamics simulation and 3D/VR visualization, analysis and manipulation of nanostructures has been implemented, representing a novel approach to advance materials research and promote innovative investigation and alternative methods to learn about material structures with dimensions invisible to the human eye.

The increasing development of computing (hardware and software) in the last decades has impacted scientific research in many fields including materials ...

Using a Virtual Store As a Research Tool to Investigate Consumer In-store Behavior

People&#39;s responses to products and/or choice environments are crucial to understanding in-store consumer behaviors. Currently, there are various approaches (e.g., surveys or laboratory settings) to study in-store behaviors, but the external validity of these is limited by their poor capability to resemble realistic choice environments. In addition, building a real store to meet experimental conditions while controlling for undesirable effects is costly and highly difficult. A virtual store developed by virtual reality techniques potentially transcends these limitations by offering the simulation of a 3D virtual store environment in a realistic, flexible, and cost-efficient way. In particular, a virtual store interactively allows consumers (participants) to experience and interact with objects in a tightly controlled yet realistic setting. This paper presents the key elements of using a desktop virtual store to study in-store consumer behavior. Descriptions of the protocol steps to: 1) build the experimental store, 2) prepare the data management program, 3) run the virtual store experiment, and 4) organize and export data from the data management program are presented. The virtual store enables participants to navigate through the store, choose a product from alternatives, and select or return products. Moreover, consumer-related shopping behaviors (e.g., shopping time, walking speed, and number and type of products examined and bought) can also be collected. The protocol is illustrated with an example of a store layout experiment showing that shelf length and shelf orientation influence shopping- and movement-related behaviors. This demonstrates that the use of a virtual store facilitates the study of consumer responses. The virtual store can be especially helpful when examining factors that are costly or difficult to change in real life (e.g., overall store layout), products that are not presently available in the market, and routinized behaviors in familiar environments.

This paper describes the use of a desktop virtual store to create virtual shopping environments to investigate in-store consumer behavior. A description of the protocol to build and run experiments, example results from an experiment concerning store layout, and important considerations when conducting virtual store experiments are presented.

People&#39;s responses to products and/or choice environments are crucial to understanding in-store consumer behaviors. Currently, there are various ...

Behavior

A Networked Desktop Virtual Reality Setup for Decision Science and Navigation Experiments with Multiple Participants

Investigating the interactions among multiple participants is a challenge for researchers from various disciplines, including the decision sciences and spatial cognition. With a local area network and dedicated software platform, experimenters can efficiently monitor the behavior of the participants that are simultaneously immersed in a desktop virtual environment and digitalize the collected data. These capabilities allow for experimental designs in spatial cognition and navigation research that would be difficult (if not impossible) to conduct in the real world. Possible experimental variations include stress during an evacuation, cooperative and competitive search tasks, and other contextual factors that may influence emergent crowd behavior. However, such a laboratory requires maintenance and strict protocols for data collection in a controlled setting. While the external validity of laboratory studies with human participants is sometimes questioned, a number of recent papers suggest that the correspondence between real and virtual environments may be sufficient for studying social behavior in terms of trajectories, hesitations, and spatial decisions. In this article, we describe a method for conducting experiments on decision-making and navigation with up to 36 participants in a networked desktop virtual reality setup (i.e., the Decision Science Laboratory or DeSciL). This experiment protocol can be adapted and applied by other researchers in order to set up a networked desktop virtual reality laboratory.

This paper describes a method for conducting multi-user experiments on decision-making and navigation using a networked computer laboratory.

Investigating the interactions among multiple participants is a challenge for researchers from various disciplines, including the decision sciences and ...

Usability Evaluation of Augmented Reality: A Neuro-Information-Systems Study

This study introduces an experimental paradigm for a usability test of emerging technologies in a management information system (MIS). The usability test included both subjective and objective evaluations. For the subjective evaluation, a usability questionnaire and a NASA-TLX scale were adopted. For the objective evaluation, methods of Neuro-Information-Systems (NeuroIS) were used. From a NeuroIS perspective, this study used mobile fNIRS and eye tracking glasses for multimodal measurements, which solved the problem of ecological validity of cognitive neuroscience tools used in real-world behavior experiments. This study used Augmented Reality (AR) integrated into the Internet of Things (IoT) as an experimental object. Comparing the differences in the neuroimaging data, the physiological data, the usability questionnaire, and the NASA-TLX scale data between the two information-search modes (AR versus a website), information search with AR had a higher efficiency and a lower cognitive load compared with information search with a website during the process of consumption decision-making. The usability experiment results demonstrate that AR, as an emerging technology in retail, can effectively enhance consumer experiences and increase their purchase intention. The experimental paradigm, combining both subjective and objective evaluations in this study, could be applied to a usability test for emerging technologies, such as augmented reality, virtual reality, artificial intelligence, wearable technology, robotics, and big data. It provides a practical experimental solution for the user experience in human-computer-interactions with the adoption of emerging technologies.

This study presents an experimental paradigm for&#160;a&#160;usability test combining subjective and objective evaluations. The objective evaluation adopted Neuro-Information-Systems (NeuroIS) methods, and the subjective evaluation adopted a usability questionnaire and a NASA-Task Load Index (NASA-TLX) scale.

This study introduces an experimental paradigm for a usability test of emerging technologies in a management information system (MIS). The usability test ...

The Immersive Cleveland Clinic Virtual Reality Shopping Platform for the Assessment of Instrumental Activities of Daily Living

A decline in the performance of instrumental activities of daily living (IADLs) has been proposed as a prodromal marker of neurological disease. Existing clinical and performance-based IADL assessments are not feasible for integration into clinical medicine. Virtual reality (VR) is a powerful yet underutilized tool that could advance the diagnosis and treatment of neurological disease. An impediment to the adoption and scaling of VR in clinical neurology is VR-related sickness resulting from sensory inconsistencies between the visual and vestibular systems (i.e., locomotion problem).
The Cleveland Clinic Virtual Reality Shopping (CC-VRS) platform attempts to solve the locomotion problem by coupling an omnidirectional treadmill with high-resolution VR content, enabling the user to physically navigate a virtual grocery store to simulate shopping. The CC-VRS consists of Basic and Complex shopping experiences; both require walking 150 m and retrieving five items. The Complex experience has additional scenarios that increase the cognitive and motor demands of the task to better represent the continuum of activities associated with real-world shopping. The CC-VRS platform provides objective and quantitative biomechanical and cognitive outcomes related to the user&#39;s IADL performance. Initial data indicate that the CC-VRS results in minimal VR-sickness and is feasible and tolerable for older adults and patients with Parkinson&#39;s disease (PD). The considerations underlying the development, design, and hardware and software technology are reviewed, and initial models of integration into primary care and neurology are provided.

Virtual reality (VR) is a powerful yet underutilized approach to advance the diagnosis and treatment of neurological disease. The Cleveland Clinic Virtual Reality Shopping platform combines state-of-the-art VR content with an omnidirectional treadmill to quantify instrumental activities of daily living-a proposed prodromal marker of neurological disease.

A decline in the performance of instrumental activities of daily living (IADLs) has been proposed as a prodromal marker of neurological disease. Existing ...

Mixed Reality for Education (MRE) Implementation and Results in Online Classes for Engineering

The COVID-19 pandemic has changed many industries, empowering some sectors and causing many others to disappear. The education sector is not exempt from major changes; in some countries or cities, classes were taught 100% online for at least 1 year. However, some university careers need laboratory practices to complement learning, especially in engineering areas, and having only theoretical lessons online could affect their knowledge. For this reason, in this work, a mixed reality system called mixed reality for education (MRE) was developed to help students develop laboratory practices to complement online classes. An experiment was carried out with 30 students; 10 students did not use MRE, 10 used MRE, and 10 more used MRE with teacher feedback. With this, one can see the advantages of mixed reality in the education sector. The results show that using MRE helps to improve knowledge in engineering subjects; the students obtained qualifications with grades 10% to 20% better than those who did not use it. Above all, the results show the importance of feedback when using virtual reality systems.

Mixed Reality for Education MRE Implementation and Results in Online Classes for Engineering

In this work, a mixed reality system called MRE was developed to help students develop laboratory practices complementing online classes. An experiment was carried out with 30 students; 10 students did not use MRE, 10 used MRE, and 10 more used MRE with teacher feedback.

The COVID-19 pandemic has changed many industries, empowering some sectors and causing many others to disappear. The education sector is not exempt from ...

A Single-Channel and Non-Invasive Wearable Brain-Computer Interface for Industry and Healthcare

The present work focuses on how to build a wearable brain-computer interface (BCI). BCIs are a novel means of human-computer interaction that relies on direct measurements of brain signals to assist both people with disabilities and those who are able-bodied. Application examples include robotic control, industrial inspection, and neurorehabilitation. Notably, recent studies have shown that steady-state visually evoked potentials (SSVEPs) are particularly suited for communication and control applications, and efforts are currently being made to bring BCI technology into daily life. To achieve this aim, the final system must rely on wearable, portable, and low-cost instrumentation. In exploiting SSVEPs, a flickering visual stimulus with fixed frequencies is required. Thus, in considering daily-life constraints, the possibility to provide visual stimuli by means of smart glasses was explored in this study. Moreover, to detect the elicited potentials, a commercial device for electroencephalography (EEG) was considered. This consists of a single differential channel with dry electrodes (no conductive gel), thus achieving the utmost wearability and portability. In such a BCI, the user can interact with the smart glasses by merely staring at icons appearing on the display. Upon this simple principle, a user-friendly, low-cost BCI was built by integrating extended reality (XR) glasses with a commercially available EEG device. The functionality of this wearable XR-BCI was examined with an experimental campaign involving 20 subjects. The classification accuracy was between 80%-95% on average depending on the stimulation time. Given these results, the system can be used as a human-machine interface for industrial inspection but also for rehabilitation in ADHD and autism.

This paper discusses how to build a brain-computer interface by relying on consumer-grade equipment and steady-state visually evoked potentials. For this, a single-channel electroencephalograph exploiting dry electrodes was integrated with augmented reality glasses for stimuli presentation and output data visualization. The final system was non-invasive, wearable, and portable.

The present work focuses on how to build a wearable brain-computer interface (BCI). BCIs are a novel means of human-computer interaction that relies on ...

Motor Imagery Using VR Digital Avatars in a Brain-Computer Interface System

Motor Imagery Performance Through Embodied Digital Twins in a Virtual Reality-Enabled Brain-Computer Interface Environment

This study introduces an innovative framework for neurological rehabilitation by integrating brain-computer interfaces (BCI) and virtual reality (VR) technologies with the customization of three-dimensional (3D) avatars. Traditional approaches to rehabilitation often fail to fully engage patients, primarily due to their inability to provide a deeply immersive and interactive experience. This research endeavors to fill this gap by utilizing motor imagery (MI) techniques, where participants visualize physical movements without actual execution. This method capitalizes on the brain's neural mechanisms, activating areas involved in movement execution when imagining movements, thereby facilitating the recovery process. The integration of VR's immersive capabilities with the precision of electroencephalography (EEG) to capture and interpret brain activity associated with imagined movements forms the core of this system. Digital Twins in the form of personalized 3D avatars are employed to significantly enhance the sense of immersion within the virtual environment. This heightened sense of embodiment is crucial for effective rehabilitation, aiming to bolster the connection between the patient and their virtual counterpart. By doing so, the system not only aims to improve motor imagery performance but also seeks to provide a more engaging and efficacious rehabilitation experience. Through the real-time application of BCI, the system allows for the direct translation of imagined movements into virtual actions performed by the 3D avatar, offering immediate feedback to the user. This feedback loop is essential for reinforcing the neural pathways involved in motor control and recovery. The ultimate goal of the developed system is to significantly enhance the effectiveness of motor imagery exercises by making them more interactive and responsive to the user's cognitive processes, thereby paving a new path in the field of neurological rehabilitation.

Author Spotlight: Enhancing Neurorehabilitation Through EEG, Motor Imagery, and Virtual Reality

Motor imagery in a virtual reality environment has wide applications in brain-computer interface systems. This manuscript outlines the use of personalized digital avatars that resemble the participants performing movements imagined by the participant in a virtual reality environment to enhance immersion and a sense of body ownership.

This study introduces an innovative framework for neurological rehabilitation by integrating brain-computer interfaces (BCI) and virtual reality (VR) ...

Bioengineering

The Use of Mixed Reality in Custom-Made Revision Hip Arthroplasty: A First Case Report

The technology of 3D printing and visualization of anatomical structures is rapidly growing in various fields of medicine. A custom-made implant and mixed reality were used to perform complex revision hip arthroplasty in January 2019. The use of mixed reality allowed for a very good visualization of the structures and resulted in precise implant fixation. According to the authors' knowledge, this is the first described case report of the combined use of these two innovations. The diagnosis preceding the qualification for the procedure was the loosening of the left hip's acetabular component. Mixed reality headset and holograms prepared by engineers were used during the surgery. The operation was successful, and it was followed by early verticalization and patient rehabilitation. The team sees opportunities for technology development in joint arthroplasty, trauma, and orthopedic oncology.

A complex revision hip arthroplasty was performed using a custom-made implant and mixed reality technology. According to the authors&#39; knowledge, this is the first report of such procedure described in the literature.

The technology of 3D printing and visualization of anatomical structures is rapidly growing in various fields of medicine. A custom-made implant and mixed ...

Medicine

Mixed Reality Technology and Three-Dimensional Printing in Teaching: Heart Anatomy as an Example

Mixed reality technology and three-dimensional (3D) printing are becoming more and morecommon in the field of medicine. During the COVID-19 pandemic and immediately after the restrictions had beeneased, many innovations were implemented in the teaching of future doctors. There was also interest in immersive techniques and 3D printing technology in anatomy teaching. However, these are not common implementations. In 2023, 3D prints and holograms in mixed reality technology were prepared for classes focused on the structure of the heart. They were used to teach students, who, with the support of engineers, could learn about the detailed structure of the heart and familiarize themselves with the new technologies that support the traditional model of learning on human cadavers. Students findthis possibility to be highly valuable. The article presents the process of preparing materials for classes and further implementation possibilities. The authors see an opportunity for the development of the presented technologies in students'teaching at various levels of education and the justification for increasingly widespread implementation.

Here, we describe a protocol for constructing a heart model from scratch based on computed tomography and present it to medical students using three-dimensional (3D) printing and mixed reality technology to learn anatomy.

Mixed reality technology and three-dimensional (3D) printing are becoming more and morecommon in the field of medicine. During the COVID-19 pandemic and ...

Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios

Samenvatting

Meer video's verkennen

Hoofdstukken in deze video