iPinchVU - Preview
Posted on May 22nd, 2010 by admin
Here is a short preview of the upcoming version of iPinchMe, called iPinchVU. It uses the handvu library for hand tracking:
Drawing a box with your hand opens iTunes, moving left/right triggers play/pause or switches to the next track. I really hope to release this version soon, but I am incredibly busy right now. Anyways, thanks to everyone who is reading this and commenting. Without knowing that there are people out there appreciating this work, I would have stopped a long time ago.
Two questions: Will we be able to use pinches and other gestures or will it be limited to drawing gestures? Of course, that’s still very useful. Also, how will we setup the gesturing triggers? I don’t imagine it’d be as simple as it is now—just performing actions according to variable states like “blue” and “left”—if we have hundreds of different states to account for while we manipulate the preinstalled script.
Either way, this is far better than anything I’ve seen available to the average person, so thanks for that. All the other webcam gesture software I’ve seen requires you to install multiple libraries like OpenCV. I love how this comes with everything you need; no hunting around or waiting for hours while half of the platform installs.
Originally I wanted to use pinches and other similar gestures. However, this approach wasn’t very robust with respect to different lighting conditions, skin colors, so i have decided to simply use HandVU’s hand tracking algorithm which is able to deliver quite robust x/y positions of the hand. It does contain some Haarcascades to recognize different hand postures but they do not seem to be working well enough.
For the upcoming release the following is planned:
-Let users define their own gestures in a config file by specifying a sequence of directions like “left-right-up-down”
-Each of these user defined gestures will get an Id that will be sent as a parameter to the applescript when the gesture is performed by the user.
The whole state thing will be dropped for now unless some people would like to keep that.
I hope this answers your questions. Feel free to ask more questions or submit suggestions for possible applications or functionality, etc.
It’s always great to see that people are interested in this stuff even though Project Natal will hit the stores soon…
Actually, yes, just a few things: Is there an archive of the older versions? I just think it’d be fun to play around with the version you used to control Google Earth. Speaking of, what are the possibilities of controlling the mouse with the new app? Will we be able to use both hands—perhaps drawing two squares closes iTunes?
At first thought I couldn’t see the use of keeping the states either but it could be useful for giving the user more control—a square in “blue” does something different than it does in “yellow”.
Due to more or less nonexistent versioning control there is no real archive unfortunately…sorry!
I experimented a bit with controlling the mouse and similar things but stopped after a while because my arm got sore and started to hurt
This phenomenon is called “Gorilla Arm” (see http://en.wikipedia.org/wiki/Gesture_recognition).
Controlling the mouse pointer itself is (technically) no problem, yet finding the right “click” gesture, which may alse robustly be recognized, is not so easy. Hence, I haved stopped going into that direction and focused more on short term specialist use for iPinchMe.
Thanks for your input on the states thing, maybe I’ll reconsider.
I am not totally sure about both hands, but I think that HandVU supports this feature. Hope that I find some time to investigate further.
i think getting rid of states is a smarter choice then to keep.
1. Having States leaves room for accidental gestures and so forth.
2. I think states in previous versions allowed for more than just accidental gestures but also not enough room for free movement.
If you want more than input more gestures
ex: circle=____
triangle=_____
I think this would be a better solution.
Have any thoughts on when the release is?
Thanks a lot for your input, I hope that I can have a release ready within one or two months.
Hi anderl2k/admin, greetings from Melbourne Australia.
Pretty amazing work you have done with HandVU. I am writing to be another voice to encourage and thank you for taking on this project, with hopes that you will release it (source-wise) sometime soon (for us mortals who are interested in experimenting with it).
Also, if you are looking for help, please let me/us know. I would love to contribute to iPinchVU if possible.
Regards, Steve
Hi Steve,
Thanks for your encouragement! As soon as I find some time, I will try to release everything.
Also, thanks for your offer to contribute. There are probably lots of things that one could contribute to this project. If you would like to contribute programming-wise, you should have some knowledge of Cocoa and Objective-C (pure C/C++, OpenCV would be advantageous as well).
Anyways, I am quite busy with my PhD right now and there’s also the wife to attend to
Hope to have some news soon.
regards,
Andreas
Hi Andreas, thanks for your reply.
I do have some experience with Cocoa and Objective-C (as well as applescript and the like), so can help out where you want. Also, there are CompSci/Software Engineering students here at Melbourne University who are interested in contributing too.
Looking forward to ‘news’ and best of success with your Ph.D. (what’s the topic/title of your research?)
Cheers, Steve