So, what I’m currently working on is a website for my prototype 2, a wiimote gestural MIDI/OSC controller made in the scope of my master’s thesis. The site will feature some explanation about the concept and source code for anyone interested in using it/developing it further.
Microsoft has announced a peripheral for their xbox, Project Natal, that scans 3D space and lets players control their system with their bodies, without a physical controller attached. Of course, you’re still working with a screen-based setup, so the applications have some limitations. But I can see the technology evolving to something much bigger than this. A big step towards ubiquitous computing that this concept of free body movement controlling is introduced to the larger public.
Take a look at the youtube movies, the 3D tracking seems to work quite well.
Hooked up four wiimotes to a java application today and they all have their own set of gestures and actions. Now I just have to set the “learn gesture” and “detect gesture” buttons to only one of the four wiimotes, and create a usable GUI for it.
I edited the Wiigee demo application to send out OSC messages. These messages get formatted by Pure Data to MIDI messages for Ableton Live. For now it just says the gestures out loud, but ultimately, a set running in ableton live can be completely controlled by these gestures. Triggering notes or sequences that are or are not quantized to the beat.
Anyway, here’s a video where I show these 5 quick test gestures; “Circle”, “Square”, “Z”, “Triangle” and “Tennis”. A musician will be able to choose his own gestures and his own events connected to those gestures so he can control his music without having to hide behind his computer.
The next step is using 4 wiimotes for gesture recognition. Then the movements made will hopefully have a higher rate of recognition. Now it has a 90% success rate or so.
Hi everyone, it’s been a while. I’ve mostly been writing these last few days but now I wanted to work on my gesturing prototype again so I did. I compiled the demo-application that comes with WiiGee, and it works quite well.
Training of the gestures works smoothly, and recognition is fast, but it’s sometimes a bit off when you’ve overtrained a gesture or when it was not precisely trained. Then it will recognize every movement as that particular gesture. But overall it works very fast and I’m curious to how it will perform with multiple wiimotes.
I sent an email to Damien from the minim library with a recording of the glitch, and here’s what he said:
“Ugh, that is a weird glitch. I’d agree that it’s not buffer underrun. It
probably has something to do with the particular version of Java you are
using on your particular macbook. My understanding is that Apple has not
been very good about keeping the JVM for macs in good shape.”
Also, I got some proper sound out of it after I started up Ableton Live to record the glitch. But as mysteriously as it got fixed, it got broken again. Maybe I should switch to a windows system for this prototype. Or Pure Data, I haven’t decided yet.