delete
Gesture control has never worked that well. Human gestures are naturally very ambiguous. Computers on the other hand are very explicit. That combination of human ambiguous with computer explicitness is why we have gone to great lengths to invent devices like mice, keyboards, joysticks, touch pads and so on.
Heck, even modern VR setups, which would be the perfect place for gesture control, are using explicit controllers.
^ I knowā¦
I am not heading for a ārevolutionary keyboard replacementā after all, the fact that Microsoft is well Microsoft - while Iām just one person should tell you that my goal is at best small app alternative to keyboard and not the ānew revolutionā of human-computer interaction.
Start by examining the ways you will bring gestures into the computer. Unless youāre planning on providing custom hardware devices you will have to rely on whatever hardware the user has available. If youāre planning on using the camera you may want to investigate the quality of the average camera people are buying.
Additionally check out your primary competition. Neuro Technology has an application that allows you to use gestures to control your mouse pointer. You mentioned your software would be affordable but I want to point out that theirs is free.
http://www.neurotechnology.com/npointer.html
Finally determine your target audience and why they will want your software over the competition or traditional input devices as having an idea and developing it are only a part of the process for creating a new product. You need to market it too.
1 Like
Iām not sure if youāre asking how to do this or claim that you know how to do this.
Microsoft put a lot of money into kinect, there are also ather attempts to make gesture recognition, and to my knowledge theyāre all somewhat unreliable.
I think it is unlikely that youāll able to beat everybody who give the idea a try. IF you CAN beat them all, you should make a prototype and post a youtube video of it in action.
If you were asking how to do it, you should start looking into stereo vision, disparity maps, blob detection, opencv and the like.
I think you would get a decent chance of at least being able to make SOME gesture recognition software, if you used Zed Camera or something similar, but that camera cost over $500. Without any special hardwareā¦ I have doubts that youāll be able to make this happen.
2 Likes
^ Thanks to both answers.
Some clarifications.
Does neurotechnology work on games or just pc app control? Because with games itās different thing. When I made my app to control desktop apps, I soon discovered itās worthless on games, because:
- Games with directx āhijackā the camera;
- The key pressing doesnāt work on directx;
- directx key pressing doesnāt work when the app isnāt on focus.
I had to fix all these three issues separately.
Besides neurotechnology is āmouse point substituteā, but is it keyboard substitute? Surely you canāt control any game with mouse only?
Iāve done it, currently ātweakingā, testing. Iāve tested it on 2 pcs only and now improving the turning left/right on the racing games.
^ Sorry, do you mean Arx Fatalis has gesture control integrated? Or do you mean I should make sure my app works with it? I am going to make more tests, thatās for sureā¦
Arx Fatalis had a gesture controlled magic system where you would cast spells by drawing magic runes with your mouse. I gesture = 1 rune, 3 runes made up 1 spell as far as I remember. You could āpreloadā the spells to 3 slots that you could then release at the press of a button each. I take that as a sign that a straight up 100% gesture based magic system was impractical for a number of reasons. Also you might be able to get some insight on what people think about the concept of gesture based controlls by reading reviews of the game.
I have to admit though, I only skimmed the whole thread really quickly and didnāt get what you were trying to do. I thought you are making a game.
To be honest I think itās a bad idea to try to slap on gesture based control systems on complex games that were built for gamepads and/or keyboard and mouse. Gesture based controlls imho will always be a gimmick, and in that case you better build your game around the gimmick, to work as well with it as possible.
1 Like
^ OK, got itā¦
I donāt know - I am just trying to make something that can be used on many gamesā¦Kinect for xbox 360 seems to have just one NFS for example, I am hoping to make it work on most of them + Test Drive and what not.
The focus is still on PC, I m not sure if Iāll ever try to make even mac version as Iām clueless about mac development.
btw:
just tested it on some old need for speedā¦works ok yet a bit strange an one sort of negative - doesnāt catch the arrows so 2 options here:
a) the game has enabled by default WASD;
b) the game should have the option to change the keys from the game menu, to WASD.
Otherwise no idea why directinput doesnāt have the arrow keys 0_0ā¦it should be the most important keys in games, yet they have āhā key but not thisā¦go figure.
I still cant fully understand what are you trying to do vs what has not been possible before. FYI Leap motion is like $30 and gestures can be bind to normal keyboard actions via different softwareās and it works both 1st person or in front. Gestures in desktop gaming also seem like a dead idea because of the inaccuracy and lazy people in general. VR is the future for any type gesture/motion control gaming but on that front there are already a lot of stuff going on.
2 Likes
^ Leap Motion isnāt like $30 - just checked it:
http://store-us.leapmotion.com/products/leap-motion-controller
$79.99 + $7.2 shipping: $87.2. Apparently something called āgamewaveā costs $3.99 to make it control any game, thus making it: Over three times what you mentioned.
So, youāre living in the UK? Even ābetterā: https://www.amazon.co.uk/Leap-Motion-Controller-Interacts-your/dp/B00C66Z9ZC
$123 USDā¦according to googleā¦and even without shpping.
Sureā¦if it was $3 vs 9 - then you can argue anyone who is paying 3 will also pay 9 but certainly not in this case. Especially as Iām not planning to make mine $30 eitherā¦
Again:
Itās hardware, not software: hardware should be avoided unless really necessary (eq.: TV screen, fridge, car. etc), but the time needed to get the product, prone to fractures, theft = not a good idea.
And finally: no such thing as ādead ideaā,MS made the news with kinect and that was itā¦speaking of which even MS are history now.
You can get the device from other places too and with free shipping. Reddit - Dive into anything
But on topic, what are your gesture detection is based on then if you plan to even semi compete or match existing solutions? I have Leap Motion and Rift+Touch plus Vive randomly available for dev testing. Leap motion is heavily gesture based but naturally lacks haptic feedback and not reliable for fast paced gaming.
You should have a prototype showcase video.
The āgestureā (hand detection) part is my own: that is, my own algorithm, itās on C# (mostly everything is, unless we count C++ directinput thing wrappingā¦). Right now itās NET 3.5, hoping to make it NET 2 (not sure if Iāve used LINQ anywhere on the code or anything that wonāt work on net 2.
Initially I was planning to use this:
http://www.codeproject.com/Articles/16447/Neural-Networks-on-C
Probably one of the best non-commercial, ācommercially allowedā libs ever created. The only problem: itās absurdly slow for real time detection, let alone racing game playing. Hoping to detect your face/hand movement in a fraction of a second while making a sudden āsteer gestureā is a lost cause with these neural nets - I even tried several approaches but all are too slow, even with all possible improvements possible.
Iām still using it thoughā¦BUT: only on the āpc controlā part, that is: face detection, moving the mouse pointer by moving your head and then clicking by pointing something with your hands. Yes, itās still slow but again, when itās desktop software, itās no big deal. The issue comes when you need to say: shoot enemy in FPS immediately, or when you need to turn left/right in racing game especially.
@
Yes, webcam.
Thatās the ideaā¦using as little hardware as possible.
Either integrated or attached separately via USB should be OK.
Again: SHOULD, canāt be 100% sure it will work on most pcs/laptops: so far itās tested on 3 only :).
About your joke/comment: well all i can say is that you can use both the keyboard and the software at the same time.
Example: say while playing Call of Duty:
you shoot by pointing your right hand in āgun gestureāā¦but you need also to jumpā¦which if youāre sitting can be done by pressing āspaceā on the keyboard with the other hand. Similarly: you may shoot with mouse but reload with gesture, for example.
edit: I made a video + screenshots and even launched crowdunding on it but donāt want to spamā¦if someone is curious by chance PM -:
Problem is, hardly anyone uses Kinect on Xbox any more, even I kind of stopped apart from using voice commands to do stuff like record YouTube footage (surprisingly the thing understands my accent!).
@RichAllen2023
Thanks for the comment, but you do NOT need Kinect to use it - only webcam. Thatās the ideaā¦
Iām not sure why so many are having trouble understanding what youāre doing. Itās webcam-based gesture-detection with a Windows desktop mode, an FPS game mode, and a racing game mode.
There are some pretty solid comments in the thread about the reasons gesture control seems to be something of a gimmick, so that meets the ācriticismā portion of your request (although Iād add the problem that gesture-based controls are simply tedious to use ā despite all the gushing when Minority Report came out, Iāve always thought it looked like some sort of ridiculous tai-chi-jazzercize workout regimen).
Iām not sure what you expected to get by asking for āideasā though.
1 Like