Development of a Generative Art App for Individuals with Limb Loss: Exploring Mirror Visual Feedback and Tactile Substitution as Complementary Therapeutic Modalities

Abstract

Individuals with limb loss may experience altered sensorimotor perception, phantom limb sensations, and changes in body image, creating a need for accessible complementary forms of interaction. This study develops a generative art app integrating digitally reconstructed mirror visual feedback with vibrotactile substitution. Seven representative movement states were selected from NinaPro DB3, and ten consistently available surface electromyography channels were used as input. A lightweight one-dimensional convolutional neural network recognized movement states and drove a mirrored digital-limb representation. A constrained generative engine mapped movement class, intensity, and confidence to particle, color, trajectory, and line parameters, while a tactile encoder converted movement events into distinguishable vibration patterns. Subject-independent evaluation compared the proposed classifier with support vector machine and random forest baselines. Prototype testing further examined visual and haptic latency, frame rate, memory consumption, and tactile trigger accuracy across four feedback configurations. The evaluation indicates that movement recognition, mirror animation, generative graphics, and vibration commands can be coordinated within a responsive mobile framework. The findings concern technical feasibility and cross-modal synchronization rather than clinical efficacy. Future studies should recruit individuals with limb loss to assess usability, sustained engagement, body-image experience, and clinically supervised outcomes.