Archive 2017: Controlling Virtual Musical instruments with hand gestures

Leap Motion

For me, it is all about adding the layer of creativity on top of technology. VR and AR are the perfect technologies to add that creative layer. In my previous blog post, I’m using VR to look around and listening to what happens in 360 space. I didn’t want to use the 360 space for now. The focus, in this case, is interaction with hand gestures.

Hand presence is real

More and more people are understanding that being able to reach into a virtual world greatly increases the sense of being there.

Get up close

The brain has special nuclei for objects that are within arms reach, and VR is able to turn those on. So came this challenge came to live:

– create old musical instruments in VR space

– and make sure you can control them with hand gestures

– trigger sounds

– what can we control? volume, pitch, frequency

The 3 test builds are based on 3 existing musical instruments. Drum machine Roland TR909, Roland TB303, and Roland Alpha Juno

Test build 1: Drum machine, Roland TR909

– 3D modeling, UV map
– trigger sounds
– control with hand gestures

Bassline synthesizer: Roland TB 303

– 3D modeling, UV map
– create exploded view
– ad UI
– to control frequency

Synthesizer: Roland Alpha-Juno-1

– Note on and note off control (compare it with key down and key up)
– Pitch