Download: mactheclapper.zip

Howto: Download the application and unzip. Start the program. A small “MC” icon will appear in your system menu. Click on the icon and choose “Settings”. A dialog will popup with wich you may configure MacTheClapper. Everything should be mostly self explanatory. Be sure to quit MacTheClapper and restart in case you changed the AppleScript command. Once you have configured MacTheClapper,  click “Start” in the MacTheClapper menu.  Clapping twice loud enough should now trigger the AppleScript command.

That’s it for now. Drop a comment if you liked the program or have some trouble using it.

For all of you who don’t know what the “clapper” is:

So, since I do not really know what I want to do next with iPinchMe, I am asking all of you (in case there actually are  people reading this):

• How would you like to control applications with hand gestures?
• What is the interface of your dreams?
• Will gesture based input be a viable alternative to keyboard and mouse one day or just be confined to certain niches?
• What’s a killer-app for hand gesture based interfaces?
• …

Tell me what you think!

P.S: To get your juices flowing, you may want check out this overview article at Venturebeat.

Enjoy!

I decided to stop using background differencing and use a skin histogram based approach combined with the camshift algorithm. Since everybody was screaming Fluidtunes, Coverflow, etc. I am testing my current development version mainly with iTunes:

I am proud to announce that the first beta version of iPinchMe is now available for download:

Link: iPinchMe - Beta

Requirements: Mac OS X 10.5 or later, Intel based Mac with builtin iSight.

Disclaimer:
This software is provided “as is” and in a HIGHLY EXPERIMENTAL state. I take no responsibility for whatever happens when you use it

Here is a short introduction on how to use it:

1. Download and unpack the file.
2. Start the program.
3. While the prgram is initializing, step away from the camera, so that iPinchMe can learn the background. You can have it relearn the background, if you press the “Reinitialize” button (sometimes you have to press it a few times).
4. Step  back in front of the camera. The prgram should now recognize you as the foreground and draw a contour around you.
5. Perform a pinch gesture (as shown in the iPhoto videos). Upon releasing the pinch, Expose should be triggered.

General Notes:
You should be in front of a static and more or less homogeneous background with as few dark areas as possible. Be sure to have enough illumination in your room. Otherwise iPinchMe will recognize false pinches or none at all, wildly triggering AppleScripts.Remember: iPinchMe will not work like magic! Computer Vision is a tough and computationally expensive problem…

How to bind other Apple Scripts to the pinches:

Upon the first start, iPinchMe will create the following file: ~/Library/Application Support/iPinchMe/ipinchme.script which contains an Apple Script function that is called from iPinchMe.

It will look like this:

on handle_event(event_msg)

if event_msg is equal to “one_pinch_release” then
tell application “System Events” to key code 101
end if

end handle_event

Depending on what gesture you perform, the string event_msg will have a different content. Currently supported are the following gestures:

1. event_msg = “one_pinch_down” (pinching with one hand)
2. event_msg = “two_pinch_down” (pinching with two hands)
3. event_msg = “one_pinch_release” (stop pinching with one hand)
4. event_msg = “two_pinch_release” (stop pinching with both hands)
5. event_msg = “one_pinch_release_left” (stop pinching with one hand on the left side of the captured camera image)
6. event_msg = “one_pinch_release_right” (stop pinching with one hand on the right side of the captured camera image)

To bind one of these gestures to an Apple Script, for instance if you would like iTunes to play the next song when you perform the one_pinch_down gesture, you will have to add something like this into the body of the handle_event method:

if event_msg is equal to “one_pinch_down” then
tell application “iTunes” to next track
end if

Apart from binding apple scripts to gestures, iPinchMe can also send keyboard events to other applications. This feature is however in a quite experimental state. So you should be really careful when to activate it!!! Only use it when you know what you are doing!!!!

Controlling Google Earth: Tick the checkbox “Send Keys” and then make sure that Google Earth is running in the foreground. In the same way you can also control TuxRacer or other apps that react on the left/right/up/down arrow keys. Look at the videos below, in order to get an impression on how move your hands.

I hope that you will have some fun with this small tool. In case you have questions, comments, bug reports, etc., just use the comments section. Ideas for future versions are always welcome.