Entries Tagged as ''
iPinchMe Development Continues - CamShift, Skin Histograms and Cover Flow
Edit: New Version available.
I decided to stop using background differencing and use a skin histogram based approach combined with the camshift algorithm. Since everybody was screaming Fluidtunes, Coverflow, etc. I am testing my current development version mainly with iTunes:
Coverflow can now be controlled by a sort of waving motion. Songs are selected with the pinch gesture. Hopefully, this version of iPinchMe will soon be available for download.iPinchMe Beta available
Edit: New Version available.
I am proud to announce that the first beta version of iPinchMe is now available for download:
Link: iPinchMe - Beta
Requirements: Mac OS X 10.5 or later, Intel based Mac with builtin iSight.
Disclaimer:
This software is provided “as is” and in a HIGHLY EXPERIMENTAL state. I take no responsibility for whatever happens when you use it
Here is a short introduction on how to use it:
- Download and unpack the file.
- Start the program.
- While the prgram is initializing, step away from the camera, so that iPinchMe can learn the background. You can have it relearn the background, if you press the “Reinitialize” button (sometimes you have to press it a few times).
- Step back in front of the camera. The prgram should now recognize you as the foreground and draw a contour around you.
- Perform a pinch gesture (as shown in the iPhoto videos). Upon releasing the pinch, Expose should be triggered.
General Notes:
You should be in front of a static and more or less homogeneous background with as few dark areas as possible. Be sure to have enough illumination in your room. Otherwise iPinchMe will recognize false pinches or none at all, wildly triggering AppleScripts.Remember: iPinchMe will not work like magic! Computer Vision is a tough and computationally expensive problem…
How to bind other Apple Scripts to the pinches:
Upon the first start, iPinchMe will create the following file: ~/Library/Application Support/iPinchMe/ipinchme.script which contains an Apple Script function that is called from iPinchMe.
It will look like this:
on handle_event(event_msg)
if event_msg is equal to “one_pinch_release” then
tell application “System Events” to key code 101
end ifend handle_event
Depending on what gesture you perform, the string event_msg will have a different content. Currently supported are the following gestures:
- event_msg = “one_pinch_down” (pinching with one hand)
- event_msg = “two_pinch_down” (pinching with two hands)
- event_msg = “one_pinch_release” (stop pinching with one hand)
- event_msg = “two_pinch_release” (stop pinching with both hands)
- event_msg = “one_pinch_release_left” (stop pinching with one hand on the left side of the captured camera image)
- event_msg = “one_pinch_release_right” (stop pinching with one hand on the right side of the captured camera image)
To bind one of these gestures to an Apple Script, for instance if you would like iTunes to play the next song when you perform the one_pinch_down gesture, you will have to add something like this into the body of the handle_event method:
if event_msg is equal to “one_pinch_down” then
tell application “iTunes” to next track
end if
Apart from binding apple scripts to gestures, iPinchMe can also send keyboard events to other applications. This feature is however in a quite experimental state. So you should be really careful when to activate it!!! Only use it when you know what you are doing!!!!
Controlling Google Earth: Tick the checkbox “Send Keys” and then make sure that Google Earth is running in the foreground. In the same way you can also control TuxRacer or other apps that react on the left/right/up/down arrow keys. Look at the videos below, in order to get an impression on how move your hands.
I hope that you will have some fun with this small tool. In case you have questions, comments, bug reports, etc., just use the comments section. Ideas for future versions are always welcome.
TuxRacer and iPinchMe - Promises to be Fun
Another possible application for iPinchMe:
(I sure hope to find serious applications as well:-)iPinchMe and Google Earth
Seems that iPinchMe is able to control Google Earth:
iSightmaster now called iPinchMe - Video Preview
Here’s a preview of the upcoming tool “iPinchMe”, it will allow you to bind applescripts to certain pinch gestures. Check out the video for a short demonstration. In the demo, a single pinch is bound to an AppleScript which send the “leftarrow key down” command to the currently active application, in this cae iPhoto. The result is a kind of hands free image browsing solution. If anyone likes this sort off stuff and/or has some ideas that could be incorporated in the final app: Just drop me a comment.
Controlling iSight Autoexposure programmatically
One of the things that make computer vision with Apple’s builtin iSight quite cumbersome, are the extremely annoying auto-exposure and auto-whitebalance features. Also, Apple does not seem to care one bit about providing any kind of documentation on how to deactivate these features (and/or control other features) of your iSight. Anyways, after a long time of googling/research I found an Objective-C class at phoboslab.org, which allows to control UVC compliant USB webcams.
With the help of this class and a few tweaks to its code I was finally able to deactivate auto-exposure and auto-whitebalance of my builtin iSight.
Here are my steps:
1. Get the UVCCameraControl class and include it in your project.
2. Instantiate this class after you started your QTCapture session with something like this:
“cameraControl = [[UVCCameraControl alloc] initWithVendorID:0×05ac productID:0×8507];”
(You might have to adjust the parameters, USB Prober is your friend)
3. In the file “UVCCameraControl.h” change the line “#define UVC_PROCESSING_UNIT_ID 0×02″ to
“#define UVC_PROCESSING_UNIT_ID 0×03″
4. In the method “- (BOOL)sendControlRequest:(IOUSBDevRequest)controlRequest” remove the calls to USBInterfaceOpen() and USBInterfaceClose().
That’s it! At least this is how it worked out for me on my MacBook Pro (latest Generation). If you still have some problems, just drop a comment.
Note: This does not seem to work, if you access your iSight via the old sequence grabber component. Be sure to use the newer Quicktime API.
