A Study on the Meaning Construction of Digital Narrative Discourse of Learners of English in Universities: Taking Holiday-Themed Multimodal Videos as an Example

Xuan Wang; Mengmeng Ji

doi:10.70088/g8smg019

Authors

Xuan Wang Ningxia University of Foreign Languages, Yinchuan, China Author
Mengmeng Ji Ningxia University of Foreign Languages, Yinchuan, China Author

DOI:

https://doi.org/10.70088/g8smg019

Keywords:

digital storytelling, multimodal discourse, visual grammar, english teaching, discourse analysis

Abstract

Learner digital storytelling transforms given narrative content into dynamic multimedia works, a pedagogical approach which has demonstrated a highly positive impact on contemporary English language teaching. Drawing extensively on the Visual Grammar theoretical framework proposed by Kress and van Leeuwen (2020), this empirical study systematically examines foreign language videos published on the official WeChat account of a university located in Northwest China. It conducts a rigorous multimodal discourse analysis across three primary dimensions—representational meaning, interactive meaning, and compositional meaning—to comprehensively explore how university English learners construct complex discourse meaning specifically at the visual level. The analytical findings reveal three key insights: 1) At the level of representational meaning, the utilized images frequently lack sufficient narrative strength; 2) At the level of interactive meaning, the overall meaning construction tends heavily toward the objective transmission of factual information; 3) In terms of compositional meaning construction, visual salience is notably insufficient across the sampled media. Therefore, while the visual modality in the short videos created by these learners generally serves an adequate communicative function, it functions primarily as a supplementary tool to the dominant textual modality, meaning the holistic construction of visual meaning remains incomplete. By analyzing bilingual videos focused on traditional Chinese festivals, this study explores the nuanced characteristics of learners' multimodal expression, offering valuable pedagogical insights for language teachers. Furthermore, it helps learners gain a deeper understanding of Chinese history, customs, and cultural heritage, ultimately laying a robust foundation for effective cross-cultural dissemination.

References

C. Jewitt, Ed., The Routledge handbook of multimodal analysis, vol. 1. London: Routledge, 2009.

K. O’Halloran, "Mathematical and scientific forms of knowledge: A systemic functional multimodal grammatical approach," in Language, Knowledge and Pedagogy: Functional Linguistic and Sociological Perspectives, London: Continuum, 2007, pp. 205–236.

S. Norris, "Multiparty interaction: A multimodal perspective on relevance," Discourse Studies, vol. 8, no. 3, pp. 401–421, 2006.

J. Lemke, "Multimodal genres and transmedia traversals: Social semiotics and the political economy of the sign," Semiotica, vol. 173, no. 1, pp. 283–297, 2009.

D. Lindenberg, "Modes and intersemiotic cohesion in student presentations performed online: An SF-informed multimodal discourse analysis," English for Specific Purposes, vol. 69, pp. 67–79, 2023.

R. Barthes, Elements of semiology. Macmillan, 1977.

E. M. Mestre-Mestre, "The construction of meaning in the Second Language Classroom. A multimodal discourse analysis," Procedia-Social and Behavioral Sciences, vol. 173, pp. 228–233, 2015.

E. Bernad-Mechó, "A multimodal discourse analysis of linking metadiscursive elements in two opencourseware lectures (MOOCs)," Procedia-Social and Behavioral Sciences, vol. 212, pp. 61–66, 2015.

G. Kress and T. Van Leeuwen, Reading images: The grammar of visual design. Routledge, 2020.

A. S. Urbieta and E. A. Peñalver, "Multimodal discourse in digital storytelling: An assessment tool proposal," Computer-Assisted Language Learning Electronic Journal, vol. 22, no. 2, pp. 14–25, 2021.

Y. F. D. Yang, "Multimodal composing in digital storytelling," Computers and Composition, vol. 29, no. 3, pp. 221–238, 2012.

S. Meier, "Digital storytelling: A didactic approach to multimodal coherence," Frontiers in Communication, vol. 7, Art. no. 906268, 2022.

C. G. Signes, "Digital storytelling and multimodal literacy in education," Porta Linguarum: Revista Internacional de Didáctica de las Lenguas Extranjeras, no. 22, pp. 237–250, 2014.

M. Y. Liang, "Beyond elocution: Multimodal narrative discourse analysis of L2 storytelling," ReCALL, vol. 31, no. 1, pp. 56–74, 2019.

Z. M. Chemodurova, "The art of storytelling in the digital age: A multimodal perspective," Vestnik Volgogradskogo Gosudarstvennogo Universiteta. Seriya 2. Yazykoznanie, vol. 21, no. 6, p. 110, 2022.

I. Alonso, S. Molina, and M. D. Porto, "Multimodal digital storytelling," Multimodality and Cognitive Linguistics, vol. 78, p. 147, 2015.

P. Bou-Franch, "Multimodal discourse strategies of factuality and subjectivity in educational digital storytelling," Digital Education Review, vol. 22, pp. 80–91, 2012.

I. Alonso, S. Molina, and M. D. P. Requejo, "Multimodal digital storytelling: Integrating information, emotion and social cognition," Review of Cognitive Linguistics. Published under the auspices of the Spanish Cognitive Linguistics Association, vol. 11, no. 2, pp. 369–387, 2013.