OpenCV X Input
July 18, 2008More fun with the marvelous OpenCV library. While red is still a convenient color to track, I’m no longer using real vegetation. Instead, I found an old pair of gloves that do the job quite well.
How it works
(Edit: rather embarrassingly, I seem to have accidentally deleted the source code for this project. However, you should be able to figure out everything you need from camshiftdemo.cpp in the samples/ folder. I apologize for the inconvenience.) Found it! Go to this directory and download redglove.cpp for examples.
A brief explanation: I’m no longer using the library’s object-tracking algorithms. Instead, the mouse is just tied to the position of the “average” red pixel, with some trivial logic for throwing away outliers. Once the bounding box of the glove drops below a certain width, a click event is triggered.
(Edit: I simulate clicks and mouse motion with the XTest library.)
(The interaction is not normally this laggy, but istanbul clobbers performance when compositing is enabled. Disable compositing, you say? I need my exposé: the window buttons on a taskbar would be too small to use reliably. The system’s not perfect. Yet.)
Some thoughts:
The click interaction really needs to be improved. There’s just enough “wobble” between the camera and an unsteady hand to make it very difficult to click without dragging, and perspective makes the rotate gesture almost impossible to execute when not dead center.
In the same vein, it would be good if I could get the program to even roughly judge the distance of the glove to set the threshold for click events dynamically.
Once I get the kinks ironed out of the current setup I’m going to try moving up to two gloves, but I think I have a way to go first. Glove-in-air interaction will never be a good model for desktop use, but I could see this being useful in applications (games in particular) that require only gross input.
Awesome now you just need to map the right mouse button to your right eye left mouse button to the left eye and mouse movement to your head overall haha :O
TAylor Nelson — July 29, 2008 @ 7:21 pm
Also gnome looks really nice, I will have to give it a try.
TAylor Nelson — July 29, 2008 @ 7:48 pm
Also gnome looks really nice, I will have to give it a try.
P.S.: Since comcast+firefox2=crash here’s an interesting website I found a little while ago.
tony5m17h.net
TAylor Nelson — July 29, 2008 @ 7:49 pm
What do you use to change the co-ordinates of the mouse cursor, any insite into this would be a great help!!
MM — November 30, 2008 @ 7:16 am
@MM: mouse interaction is accomplished with the XTest library.
Jacob — November 30, 2008 @ 8:53 am
Hi man, i have the same question as the above comment “What do you use to change the co-ordinates of the mouse cursor?”, and i can not find your camshiftdemo.cpp in the website, can you tell me where i can find it. thanks!!!
Ricky — February 3, 2009 @ 9:18 pm
I’ve added a link to the code at the top of the page.
Jacob — February 4, 2009 @ 6:49 am
Thank you very much!
Ricky — February 4, 2009 @ 4:45 pm
hi
thanks for this nice programm.
works great, with a little red-ball on my table.
regards
steven
steven morf — May 4, 2009 @ 12:26 am
compiling redglove.cpp
/tmp/ccNTcIT1.o: In function `paint_red(_IplImage*, _XDisplay*)’:
/home/boo/Desktop/examples/redglove.cpp:131: undefined reference to `XTestFakeMotionEvent’
collect2: ld returned 1 exit status
Anny suggestions???
Running Ubuntu 8.04…
RockAmadeus — May 25, 2009 @ 7:22 am
@RockAmadeus:
Do you have XTest installed?
Jacob — May 30, 2009 @ 11:44 pm
@Jacob:
Sure do… I get the error even when i unlink the library so I think there is some problem with ubuntu and xtest… Any Ideas?
RockAmadeus — June 1, 2009 @ 6:37 am
Hy, very interesting code
a part that I don’t know where I can buy
a red glove.
Have you got to work the X input with a chinese red laser pointer?
I think that can work great.
Can your code works well with a red laser pointer?
How works the tracking with opencv?
Thanks Bye
Pietro — June 9, 2009 @ 1:33 pm
This is very interessing!
I want to create the same myself but I have a little problem: I also use XTest library but when I move the mouse with XTestFakeMotionEvent, I don’t see the mouse moving but she clic on the right position. How did you do? I’ve searched on your code but I don’t find anything. Could you help me?
Great work!
Thanks
spirou91 — July 26, 2009 @ 12:06 pm
Great program!!! Can you put a link to download the source of your program???
Greetings
Daniel — February 17, 2010 @ 1:39 am
@Daniel — the link in the post (and all other code links on the site) should now be working. Let me know if it works for you.
Jacob — February 17, 2010 @ 3:24 am
Ohhh… yes. Now work. Yesterday the link don’t work… jejeje. I’m not good en c++ (i know c#) and i don’t know how put the librarys. I download and install the OpenCV-2.0.0a-win32.
The errors are in the #include “cv.h”, #include “highgui.h”, #include , #include .
Thanks for all… Greetings.
Daniel — February 17, 2010 @ 7:46 pm
hi.. thanks a lot for the code.. nice program!!!
Richa — March 21, 2010 @ 11:31 am
I have some question , where I can get XTest library. I have some know about opencv ,I think I can make some common interest
if I want to run your source , if I use
- opencv1.0
- visual c++ 6.0
- windowXp sp3
it will work ?
thank for any answer
why I don’t run before ask you ?
because my computer can’t use now.
sixno — April 18, 2010 @ 6:30 pm
XTest is for computers with X Windows only, but I’m sure there’s some equivalent library for MS Windows
Jacob — April 26, 2010 @ 10:48 pm
Hi, I have the same problem than rock amadeus, I have installed Xtest, but I’m not sure if you add an extra option for compile. I’m using ubuntu 9.04
Thanks!
Zarlink — May 17, 2010 @ 4:24 am
I can build with
g++ redglove.cpp -lcv -lhighgui -lXtst.Jacob — May 17, 2010 @ 8:41 pm
Hi, do you have a same project in Vb.net ?plz. i don`t know c++, i have start with c# but is difference. thks!!!
Diego — May 29, 2010 @ 7:41 pm
Hi, what is the license for this code? Is it the same as OpenCV (BSD)? Thanks for sharing this!
Mike Chelen — June 30, 2010 @ 2:21 am
Public domain (no license). Enjoy!
Jacob — June 30, 2010 @ 4:55 am
hi i am gettting
“missing type specifier – int assumed. Note: C++ does not support default-int”
error after compiling your code
please help ?from where to get this X11 liberary for windows?
thanks in advance
atulrai — September 1, 2010 @ 5:33 am
I’m not sure you’ll be able to get this working on Windows — you can get X itself in the form of Xming, but I have no idea if you can actually build against all of the required libraries.
Jacob — September 1, 2010 @ 5:51 am
hi is the code is runnable on visual studio 2005 on windows platform? if yes then how to use liberary x.lib and x11 etc.??
siet — September 1, 2010 @ 7:35 am
so for windows what i have to do??
and which platform you have used??
atulrai — September 1, 2010 @ 7:36 am
I did all of the development for this project in Linux. I’m afraid I can’t be of any help in developing for Windows. I will be very surprised if you get my code to run, but let me know if you do. Good luck!
Jacob — September 1, 2010 @ 1:54 pm
actually i am new for open cv so i can try only but if you will help i will be really very thnkful to you!!!!
atulrai — September 3, 2010 @ 1:31 pm
hey i had tried but not able to find the solution can you please help me for the code for windows please??
thank you
atulrai — September 5, 2010 @ 7:06 pm