A More Immersive Experience, Deeper Learning? - A Comparative Study of Multimodal Interaction Combinations and Cognitive Mechanisms in UE5 Digital Exhibition Halls
DOI:
https://doi.org/10.70088/sqcbct89Keywords:
Unreal Engine 5, Digital Exhibition Hall, Multimodal Interaction, Cognitive Load, Multiple Resource TheoryAbstract
Unreal Engine 5 (UE5) facilitates the creation of highly immersive digital exhibition halls. However, an overabundance of visual, auditory, and haptic cues might elevate cognitive load, thereby diminishing learning outcomes. By drawing upon Multiple Resource Theory and Cognitive Load Theory, this study posits that a greater number of modalities does not inherently equate to enhanced efficacy.etter and that the combination of visual and haptic interactions, denoted as V+H, could potentially offer a more favorable equilibrium between immersion and processing demand. A two-stage experimental approach is employed to compare the conditions of V+A, V+H, and V+A+H. Subsequently, cognitive load, attention allocation, and task self-efficacy are examined as mediating mechanisms. This study provides design guidance for achieving efficient multimodal interaction within UE5 exhibitions.References
"Effects of immersion level and sound type on user experience in mixed reality interactive exhibitions: Evidence from psychophysiology," Displays, Elsevier, 2026.
"Multimodal interaction enhancement of digital cultural heritage system: User behavior analysis and interface reconstruction of the heritage scanning library of the palace museum," Scientific Reports, Springer Nature, vol. 16, p. 10654, 2026.
L. Bu, W. Zhang, Y. Liu, S. Qiao, and H. Li, "Better cues for better pilots: Comparing multimodal combinations and exploring cognitive mechanisms in general aviation simulation," International Journal of Human–Computer Interaction, p. 124, 2026.
L. Bu, B. Cui, W. Pan, H. Chen, S. Xia, and H. Li, "User-centered multimodal interaction design for autonomous vehicles: A focus on cognitive load and accessibility for users with severe visual impairment," Universal Access in the Information Society, vol. 24, no. 4, pp. 3431–3448, 2025.
J. Sweller, "Cognitive load during problem solving: Effects on learning," Cognitive Science, vol. 12, no. 2, pp. 257–285, 1988.
C. D. Wickens, "Multiple resources and mental workload," Human Factors, vol. 50, no. 3, pp. 449–455, 2008.
A. Bandura, Self-efficacy: The exercise of control. Macmillan, 1997.
H. Li, Y. Hou, W. Pan, H. Chen, J. Gai, and L. Bu, "Exploring the impact of anthropomorphism in AI navigation systems on elderly users' efficiency, trust, satisfaction, and cognitive load," International Journal of Human–Computer Interaction, vol. 42, no. 3, pp. 1503–1522, 2026.
M.-F. Lee, G.-S. Chen, H.-C. Chen, and J.-Z. Chen, "The intersection of digital wellbeing and collection exhibition: A study on the impact of AR interactive display models on visitor experience," Computer Science and Information Systems, vol. 22, no. 4, p. 1637, 2025.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Xuzhe Sun (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
