REFLECT

REFLECT is a VR application based on almost 5 years of PhD research that converts upper-limb movement into real-time MIDI interaction. Originally developed for stroke rehabilitation, it combines immersive interaction, music, and visual feedback to create motivating and accessible movement experiences.
REFLECT has two main scenes, Guided Mode and Freeplay Mode.
Guided Mode comprises ten song-exercise pairs where the user will move to the selected MIDI track and hear a chord matched to a MIDI instrument when their hand is in the correct position. The user has control of which MIDI channels they hear, and can also build the song up from the drum channel, adding an instrument after each bar if their hand is in the correct position. Following calibration, the guided movements will automatically adapt to the user’s functional range of motion, enabling personalisation.
Freeplay Mode provides users with a different set of 10 non-MIDI songs, with the ability to layer notes from a MIDI instrument of their choosing. The notes are pre-selected based on the pentatonic scale that matches the song’s key, enabling meaningful and pleasing musical exploration without prior musical experience.
A longer 25-minute walkthrough demonstrating all features and pilot-trial procedures is available on request.

REFLECT applies MIDI in a novel biomedical context, combining music, game development, and neurorehabilitation to transform rehabilitation movements into expressive musical interaction. REFLECT links hand position to real-time tri-modal feedback (visual, auditory, and haptic) to provide a fully immersive experience.
REFLECT also facilitates the rapid creation of customised rehabilitation movement sequences, enabling a therapist to prescribe a tailored exercise plan that can be completed independently in the participant’s home environment, at a time and pace that suits them.
The combination of MIDI feedback based on hand position, mirror therapy, and rehabilitation demonstrates how MIDI can support accessibility, motivation, and real-time human-computer interaction far beyond music production.

REFLECT supported multiple stroke survivors in demonstrating a clinically meaningful change in arm function after two months of independent, home-based use. REFLECT enables people with upper-limb impairment to use movement as a creative form of rehabilitation and an accessible way to create and explore music.
REFLECT was co-designed with 14 stroke survivors and 12 professionals to ensure it was fit for purpose and captured the diversity in requirements between all stakeholders.
REFLECT has been received well by a wide range of stroke survivor ages, from early 40s to late 80s, demonstrating that age is not a barrier to accessing music-supported rehabilitation technology.