Sax Tracking Project
Overview:
I was inspired to apply the principles from my computer vision class to control parameters in Ableton Live. There are certain instinctive gestures I’ll make when playing the saxophone, such as when a line gets louder or higher in pitch, I’ll move the bell of the saxophone up. I wanted to use these natural movements to control other sounds to augment my sax playing.
Writing the script:
I used the code from this video as a jumping off point. They use MediaPipe to control Ableton parameters with hand gestures, but I wanted to rework it to use OpenCV to track red stickers I could attach to my soprano saxophone. After a conversation with my professor, an unpaid consultation from a friend majoring in CS, and ChatGPT debugging, I had a script that could reliably track the angle between two stickers.
Attaching the stickers:
I could easily attach a sticker to the wider end of the saxophone, but I felt it would be a crime to leave sticker residue on my beloved soprano saxophone. I bought a pack of stickers that claimed to be “easily removable”, and thankfully found they were true to their name and left no residue.
However, I couldn’t attach the sticker to narrow end of the saxophone the same way; there just wasn’t enough space. I decided to take this as an opportunity to sharpen my CAD skills and 3D print a custom part to attach the sticker:
Final Demo:
In this video, I’m using the angle of the stickers mounted on my soprano to control the waveform shape in Ableton’s wavetable synth. As I raise my soprano, the tone of the synth pad gets brighter as it transitions to square waves, and as I lower my soprano the sound gets more mellow.
I wouldn’t feel confident using this in a live setting with changing light, as I noticed the program having trouble detecting the stickers when my overhead light was on. But I’m excited to explore what other control applications this might have.
Figure 2: Final 3D printed part
Figure 1: 3D models of sticker attachment part in Autodesk Fusion
