Pose AI Synth

Usage:

(Anything you do will be saved locally, so please don't use this app in incognito until the app can export configs)

TL;DR:
Trigger channel is for external gates.
Output channel is where values are sent.
Sending more than one note is sending a chord.
Quantize your values!!!

This App uses your webcam with AI to track poses and convert them into MIDI messages.

Every frame on the browser is tracked and the results of each visible landmark are stored individually.

To send a landmark's values, you need to set up an output channel (this is the channel you need to use in your app consuming MIDI) and a trigger channel. This is where you send your clock/gate or any "noteon" message that will trigger the process of sending notes to the output channel. "noteoff" messages will try to stop the previous notes that were sent so they don't get stuck.

Notes on the X axis are sent from left (0) to right (127), notes on the Y axis are sent from top (0) to bottom (127). Any channel that receives more than a single message (either [x, y] or multiple landmarks) will send them as a chord, meaning you need to enable poliphony when parsing the received notes. The app also tries to eliminate note duplicates, so those probably won't be sent.

Scale quantization is recommended because the default scale will always be chromatic.

MIDI Options