Hot Licks

Categories: Software Prototypes/Non-Commercial Products

Submitted by:Nishal Silva and Luca Turchet, Creative, Intelligent & Multisensory Interactions Laboratory, University of Trento.

24 May 2023

Elevator Pitch

Hot Licks

A MIDI enabled VST plugin to incorporate a multitude of audiovisual devices into a musical performance.

Product Description

Hot Licks

Hot Licks is a VST plugin that allows a musician to use their performance to trigger external devices such as stage lights and smoke machines. The plugin allows the user to record several musical patterns they would use in the performance and assign control messages to each pattern to trigger an external device. Upon completing this preliminary step, a neural network is trained to detect the recorded patterns. The plugin will receive its input via MIDI, which can then be passed on to synthesize sound. With every note played, the plugin will check if the sequence of notes ending with the current note is similar to one of the recorded musical patterns. If there is a match, Hot Licks will send the MIDI control message associated with that pattern to a DMX controller to trigger any peripheral device. The detection is done in milliseconds, and devices could be triggered instantaneously, upon playing a pattern. Using this plugin, a musician can incorporate external peripheral devices into their performance by using musical phrases played on their MIDI-capable instrument as triggers to switch them on or off, change patches, change parameters, etc.

How It's Innovative

Hot Licks will allow a musician to use their instrument and the phrases they play on it, as triggers for external devices such as lights and smoke machines. It will give the musician an extended ability through the control of other devices, and it will allow the musician to include these devices as part of their performance.

See MIDI Innovation In Action

Most Inspiring Use Cases

Stage devices could be triggered even during an improvisation, without the use of a lighting control engineer. A musician could play a musical phrase to set off a smoke machine, to control stage lights, to switch on a spotlight, even to activate a flame thrower. We hope that musicians will be able to use Hot Licks as a creative and artistic output which acts as a visual complement to their music.

Expansion Plans

Currently, the plugin is able to detect monophonic phrases. We plan to extend this study to work with polyphony, and also multi-instrument patterns. We will plan to develop the plugin to run on embedded devices that are self-contained within a musical instrument, where model training will be performed on a cloud server, inference will be performed on the embedded device itself, and control messages will be transmitted wirelessly. The research will act as a fulfillment of the research field of smart musical instruments, and the internet of musical things.


We plan to release this plugin as an open source project upon publication of the research paper outlining its development.