Jun 14, 2015 at 2:50 AM
Indie game enthusiast
"What is a man!? A miserable pile of secrets! But enough talk, have at you!"
Join Date: Apr 18, 2006
Location: Forever wandering the tower...!
Posts: 1787
See some interesting product or technology that's captured your interest? Talk about it here!
Human interface devices, voice recognition, eye-tracking, face-detection, and EEG mind control!
Topic inside!
Human interface devices, voice recognition, eye-tracking, face-detection, and EEG mind control!
Topic inside!
Some background before I begin:
Last month I ordered a Kone XTD max customization laser gaming mouse (and a Corsair Gaming MM600 Double-Sided Aluminum Core Gaming Mouse Mat), since I use the desktop so often. Was hoping to move up from a basic throw-a-way mouse and ancient plastic rectangle moon picture mousepad to a life of luxury :]
The good news is that, it is an amazing combination of equipment. The mouse glides like ice on the smoother side of the pad, and the actual mouse material *feels* something like mouse leather. And those crazy lights that are all the rage can be customized to a soothing violet blu-ray pulse. I *really* liked the thing.
Except, the bad news. It costs and arm and a leg... or actually just my arm. My palm / arm started bothering me just after a few days after I started using it. Not sure if it was just the tip of the iceberg, since I've injured the arm in the past, but the timing seems like switching to a new large mouse from the small HP two button just wasn't the right move for me. (I used to have an intellimouse explorer ages ago that was comparable in size... but...) Well, choosing mice seems more complicated than, what's the best one. There's claw, palm, and finger-tip grip styles for instance. And I switch between palm and claw. Apparently it's really best to try out different mice in person, and better yet demo mice that you might think fit you well, to see how comfortable it is in the long term. We can buy shoes online, and hope for the best... and the same principle goes for the mouse. One-size does not (comfortably) fit all.
Anyway, fortunately I was able to return it within 30 days (Amazon!), and get a refund minus shipping cost.
And so, today's tech discussion really begins here. Human input devices. I've since looked at alternative and ergonomic mice, pedals, voice recognition, eye-tracking, face-tracking and even actual EEG brain tracking devices! While a normal mouse is currently great for most users (minus the fact that they come in all shapes and sizes), when you use a PC as much as I do you might want to find ways to improve your experience, your workflow, and protect your body from repetitive-use; or just for fun!
Where I started: Voice recognition (real examples):
I.e. Enter room. Collapse on bed, eyes closed. Say, "Start Winamp". "Shuffle". "Volume Down". "Next". Grin as music plays. Or, dictate an essay as you sit outside through the phone to your PC*.
I.e. Say, "Start Cave Story". Cave Story Forums pop-up. "Down". "Down". "Find Satellite Lounge and Go". Browser searches for the first entry, and opens that link.
I have access to Dragon Naturally Speaking and a good microphone with a general compression filter on it, so I can be anywhere in the room and have the PC hear me and respond. I actually remember using Dragon for fun years and years ago! They've come a long ways... And thanks to the internet community it suddenly is a lot more powerful than ever before. People have created Natlink, Vocola, Unimacro, Dragonfly; software to connect Dragon to Python and different libraries, and anything you can imagine and say, you can assign actions or scripts.
Learning about voice recognition brought me to an aside topic... Text-to-speech.
Check out this link, they've come a long ways from Microsoft Sam.
http://www.acapela-group.com/ (Try typing in some stuff there and testing out a few voices! They say whatever you type... keep it clean... )
After I looked into voice recognition, I was interested in having the PC actually respond back! For instance:
I.e. I'm currently focused on something. I say, "What time is it?". Computer says : "The time is six o' four PM."
I.e. I load up a digital textbook I have to read... I say, "Start reading". And bam, I can keep myself focused with a good voice helping me along when I need it.
And then just yesterday I looked at my first Chatbot script. Customizable bots you can script to respond to input (or so it seems.) So, add voice recognition into the mix, and maybe I can setup a simple computer monitor with personality :] The bot I found uses Rivescript. Just starting to play with it. If I got really into it, I could add an avatar and hook it up so it will actually go through the motions of talking. I've seen that... Add in OpenCL for face detection and/or recognition with a webcam... I could have the computer blank the screen when I'm not there!
One issue though with TTS, voice recognition tries to listen to the TTS. So, I can make it talk to itself with different voices lol. But it becomes quite troubling when it starts to injects all the dialogue it says into what you are doing. Might have to make the audio recording switch to zero input while there is TTS activity as a simple fix...
Oh, and since I mentioned face detection:
http://kinesicmouse.xcessity.at/
With that software, if you have a X-Box One Kinect One, you can bind actions to facial gestures.
https://www.youtube.com/watch?v=IjhrR4YQtRE
Hands-free gameplay in a FPS! I love it xD ...Unfortunately the price I don't love: $400 (not including kinect and windows cable). But, the target audience are those who NEED it currently... Not, someone like me who would just like to add an extra sensory input.
...I can see online games coming where all your facial actions are read and displayed in-game! That'll be quite epic. And already seems to be implementable already... If not for the hardware price. Actually, I haven't been keeping up with the latest consoles... wonder if they've already done something like that, in a non-gimmicy way.
I was actually considering getting a kinect to try this, since it apparently also serves as a 3D scanner!!! But, sadly apparently it only does well with a macro level. Don't expect any detail of smaller objects.
Oh and I believe that video above uses eye-tracking to aim**. Which leads me to Tobii EyeX and the SteelSeries Eye Sentry!
Back to Dragon and voice recognition software. Using voice commands to navigate works decently. You can even "Mouse Grid" or specify coordinates to place the mouse on the screen or move it by voice. But, it is horribly inefficient, even if you have a location perfect memory for coordinates. However, with eye-tracking, you can simply look on the screen and have the mouse warp to the position you look. Now, I like the sound of that.
Currently the SteelSentry consumer eye tracking device is being described as a kind of gaming statistics device, and an immersion enhancing device (poorly natively supported currently). I.e. one supported game allows you to look around with just your eyes. And as for statistics... Well, originally this technology is BIG BUCKS (1k - 30k) for research purposes. Like, where do the eyes go when you look at this advertisement. How long do your eyes look at this, or that. Well, this $100-$200 device is being used to let gamers analyze their gameplay or show streams where you see what the player is monitoring exactly. Pretty nifty. But, thanks to the EyeX SDK, you can do things like this:
http://iris.xcessity.at/
Same guy as that kinect mouse project. Still pretty expensive... But, it allows you to define vision hotspots. When your vision looks at a spot on the screen, you can have an action occur. Think about it... Maybe have 6 minimized or shrunken windows; three on the left and three on the right, top to bottom. When you look at one of them for a pre-determined amount of time, it could shrink your current main window and switch it out with that one for your main. I don't know the best way to do it, but just imagine the customization :] Or for example, playing a game where you have a mini-map on the corner. When you focus on that map, have it increase in size, until you look away.
Oh, and I liked a random youtuber comment: Imagine NPCs reacting to where you are staring. Fun times ahead. lol
...This has gotten way longer than I meant, but let me skip to one more interesting device, ignoring the ergonomic stuff for now.
EEG or electroencephalography input devices. Like the Emotiv EPOC or what I have my eye on... the Insight. Basically, it reads certain brain activity, allowing you to map brain actions as yet another input device.
Think forward... you move forward. Think up... you move up... Are you bored? Are you interested? It Knows. Now, imagine adding THAT functionality to a RPG. The AI can tell you are bored or interested and comments on it. lol. Magic performed not with a key press, but with the mind. Love the sound of that. YES.
https://emotiv.com/insight.php
https://www.youtube.com/watch?v=yJBB5rxs-8E
Interview and demo
I'd say this tech is still in consumer infancy, but because it's becoming affordable... Well, good things are hopefully coming to our future. ...No Sword Art Online side effects though please ;D
Last month I ordered a Kone XTD max customization laser gaming mouse (and a Corsair Gaming MM600 Double-Sided Aluminum Core Gaming Mouse Mat), since I use the desktop so often. Was hoping to move up from a basic throw-a-way mouse and ancient plastic rectangle moon picture mousepad to a life of luxury :]
The good news is that, it is an amazing combination of equipment. The mouse glides like ice on the smoother side of the pad, and the actual mouse material *feels* something like mouse leather. And those crazy lights that are all the rage can be customized to a soothing violet blu-ray pulse. I *really* liked the thing.
Except, the bad news. It costs and arm and a leg... or actually just my arm. My palm / arm started bothering me just after a few days after I started using it. Not sure if it was just the tip of the iceberg, since I've injured the arm in the past, but the timing seems like switching to a new large mouse from the small HP two button just wasn't the right move for me. (I used to have an intellimouse explorer ages ago that was comparable in size... but...) Well, choosing mice seems more complicated than, what's the best one. There's claw, palm, and finger-tip grip styles for instance. And I switch between palm and claw. Apparently it's really best to try out different mice in person, and better yet demo mice that you might think fit you well, to see how comfortable it is in the long term. We can buy shoes online, and hope for the best... and the same principle goes for the mouse. One-size does not (comfortably) fit all.
Anyway, fortunately I was able to return it within 30 days (Amazon!), and get a refund minus shipping cost.
And so, today's tech discussion really begins here. Human input devices. I've since looked at alternative and ergonomic mice, pedals, voice recognition, eye-tracking, face-tracking and even actual EEG brain tracking devices! While a normal mouse is currently great for most users (minus the fact that they come in all shapes and sizes), when you use a PC as much as I do you might want to find ways to improve your experience, your workflow, and protect your body from repetitive-use; or just for fun!
Where I started: Voice recognition (real examples):
I.e. Enter room. Collapse on bed, eyes closed. Say, "Start Winamp". "Shuffle". "Volume Down". "Next". Grin as music plays. Or, dictate an essay as you sit outside through the phone to your PC*.
I.e. Say, "Start Cave Story". Cave Story Forums pop-up. "Down". "Down". "Find Satellite Lounge and Go". Browser searches for the first entry, and opens that link.
I have access to Dragon Naturally Speaking and a good microphone with a general compression filter on it, so I can be anywhere in the room and have the PC hear me and respond. I actually remember using Dragon for fun years and years ago! They've come a long ways... And thanks to the internet community it suddenly is a lot more powerful than ever before. People have created Natlink, Vocola, Unimacro, Dragonfly; software to connect Dragon to Python and different libraries, and anything you can imagine and say, you can assign actions or scripts.
Learning about voice recognition brought me to an aside topic... Text-to-speech.
Check out this link, they've come a long ways from Microsoft Sam.
http://www.acapela-group.com/ (Try typing in some stuff there and testing out a few voices! They say whatever you type... keep it clean... )
After I looked into voice recognition, I was interested in having the PC actually respond back! For instance:
I.e. I'm currently focused on something. I say, "What time is it?". Computer says : "The time is six o' four PM."
I.e. I load up a digital textbook I have to read... I say, "Start reading". And bam, I can keep myself focused with a good voice helping me along when I need it.
And then just yesterday I looked at my first Chatbot script. Customizable bots you can script to respond to input (or so it seems.) So, add voice recognition into the mix, and maybe I can setup a simple computer monitor with personality :] The bot I found uses Rivescript. Just starting to play with it. If I got really into it, I could add an avatar and hook it up so it will actually go through the motions of talking. I've seen that... Add in OpenCL for face detection and/or recognition with a webcam... I could have the computer blank the screen when I'm not there!
Was watching an anime, Strike the Blood...
He was a nice bot :]
"Mogwai, what's the answer to ..."
He was a nice bot :]
"Mogwai, what's the answer to ..."
Oh, and since I mentioned face detection:
http://kinesicmouse.xcessity.at/
With that software, if you have a X-Box One Kinect One, you can bind actions to facial gestures.
https://www.youtube.com/watch?v=IjhrR4YQtRE
Hands-free gameplay in a FPS! I love it xD ...Unfortunately the price I don't love: $400 (not including kinect and windows cable). But, the target audience are those who NEED it currently... Not, someone like me who would just like to add an extra sensory input.
...I can see online games coming where all your facial actions are read and displayed in-game! That'll be quite epic. And already seems to be implementable already... If not for the hardware price. Actually, I haven't been keeping up with the latest consoles... wonder if they've already done something like that, in a non-gimmicy way.
I was actually considering getting a kinect to try this, since it apparently also serves as a 3D scanner!!! But, sadly apparently it only does well with a macro level. Don't expect any detail of smaller objects.
Oh and I believe that video above uses eye-tracking to aim**. Which leads me to Tobii EyeX and the SteelSeries Eye Sentry!
Back to Dragon and voice recognition software. Using voice commands to navigate works decently. You can even "Mouse Grid" or specify coordinates to place the mouse on the screen or move it by voice. But, it is horribly inefficient, even if you have a location perfect memory for coordinates. However, with eye-tracking, you can simply look on the screen and have the mouse warp to the position you look. Now, I like the sound of that.
Currently the SteelSentry consumer eye tracking device is being described as a kind of gaming statistics device, and an immersion enhancing device (poorly natively supported currently). I.e. one supported game allows you to look around with just your eyes. And as for statistics... Well, originally this technology is BIG BUCKS (1k - 30k) for research purposes. Like, where do the eyes go when you look at this advertisement. How long do your eyes look at this, or that. Well, this $100-$200 device is being used to let gamers analyze their gameplay or show streams where you see what the player is monitoring exactly. Pretty nifty. But, thanks to the EyeX SDK, you can do things like this:
http://iris.xcessity.at/
Same guy as that kinect mouse project. Still pretty expensive... But, it allows you to define vision hotspots. When your vision looks at a spot on the screen, you can have an action occur. Think about it... Maybe have 6 minimized or shrunken windows; three on the left and three on the right, top to bottom. When you look at one of them for a pre-determined amount of time, it could shrink your current main window and switch it out with that one for your main. I don't know the best way to do it, but just imagine the customization :] Or for example, playing a game where you have a mini-map on the corner. When you focus on that map, have it increase in size, until you look away.
Oh, and I liked a random youtuber comment: Imagine NPCs reacting to where you are staring. Fun times ahead. lol
...This has gotten way longer than I meant, but let me skip to one more interesting device, ignoring the ergonomic stuff for now.
EEG or electroencephalography input devices. Like the Emotiv EPOC or what I have my eye on... the Insight. Basically, it reads certain brain activity, allowing you to map brain actions as yet another input device.
Think forward... you move forward. Think up... you move up... Are you bored? Are you interested? It Knows. Now, imagine adding THAT functionality to a RPG. The AI can tell you are bored or interested and comments on it. lol. Magic performed not with a key press, but with the mind. Love the sound of that. YES.
https://emotiv.com/insight.php
https://www.youtube.com/watch?v=yJBB5rxs-8E
Interview and demo
I'd say this tech is still in consumer infancy, but because it's becoming affordable... Well, good things are hopefully coming to our future. ...No Sword Art Online side effects though please ;D