Thoughts on the Leap Motion

Well my Leap Motion finally arrived today, and my girlfriend and I had a bit of fun with it.

When I ordered the Leap Motion, I thought it looked unbelievable. Possibly a revolutionary new way to interact with our computers. As the waiting stretched out, however, I watched more videos – tech demos and the like – to keep myself occupied while I waited. That’s when I first started to see the cracks. Fingers couldn’t be registered unless close to parallel to the device, slight delay on movements, and “misses” when using the interface.

So by the time it arrived, I had a pretty good idea what to expect. It delivered on that. I know quite a few people are very disappointed: https://forums.leapmotion.com/showthread.php?2703-Unimpressed-is-an-understatement, but I’m not. My hopes just weren’t as high as they originally were.

Don’t get me wrong. The Leap is miles ahead of the Kinect or Move. The originals, certainly, but I’d also wager the new ones coming with the PS4 and XBone will be nearly as useless as the originals. The apps that are designed to use the Leap (basically the little games and things in the Airspace store) all work quite well. It takes a little getting used to.

So – although the hardware of Leap is a smidge less than I’d hoped for, I think with the right software it could do a lot of really interesting things. We might be looking at a over-hype backlash, which would hurt the chances of seeing that software, though. If the Leap Motion can struggle through the initial, rocky, days we might get some talented developers churning out some really useful apps. On the ideas forums, for example, I see an idea for “a boxing game” and gestures for a media controller (i.e. circle forward for fast-forward.)

My ideas basically revolve around cleaning up the interactions. At the moment they seem too wielded to an “invisible touch screen” floating between you and your monitor. It’s pretty annoying and inaccurate to try and break this plane with just the tip of your finger to get the “click” to work. I’d much prefer to see more intuitive ideas implemented. A whole new interface is one common suggestion (i.e. something more like Windows Metro interface) but I think you could, with clever programming, get the Leap working in Windows 7 too. For example a pinching or fist gesture to “grab” a window, so you can move it around. Some ray-casting so you can point at what you want to click, rather than having to “hover” your finger over it. A real “click” gesture, rather than the crude “jab”. Or at least, base the “click” off the acceleration of the finger, not whether it is breaking an invisible plane. And don’t give me any of this malarky about the Leap being unable to recognise a fist. Natively, perhaps. It seems to look for “sticks” and a “thicker stick” for fingers and a hand. But if the fingers curl up and disappear, and you’re left with just your original, shorter, stick — surely the user just made a fist? Besides. It’s just an infrared camera. Unless some of the latency trickery is built directly into the Leap hardware, can’t we have it recognise a fist along with fingers and arm?

My point is – these problems are fixable with clever software. Whether that is Leap Motion improving their Leap drivers, or 3rd party devs making customised recognizing algorithms, I think it’s doable. It will take a few months or maybe even a year or two, to really iron out.

Of course, the main reason to get a Leap, is so you have an interface for the Oculus Rift.

Why the Oculus Rift will succeed

The internet is all a-buzz, all a-tweeter even, over the Oculus Rift(OR) and for good reason, I think.

Look, there’s a lot of speculation about the OR, but there are very few nay-sayers. That’s rare for the internet. Everyone that tries it are quick converts

(and penny arcade)

other than a few niggles, such as low resolution, which the developers promise will be ironed out there’s not much to complain about.

Is the OR the future of gaming? Or even the future of virtual reality — I’m talking virtual offices now (a Google+ post)?

Allow me to speculate:

Virtual Offices

Firstly let’s shoot down the “definitely nots.” No virtual offices. It’s just too different from what we currently do, there’s not enough gains, it will pull people too far out of their comfort zones and it will initially carry a stigma of being “for gaming.” People will also rebel at having to wear the device. For a work day? Not going to happen. People will even rebel because of how they look. Something with augmented reality might catch on – like Google Glass – because glasses have been, can be, and are, fashionable. Google has a ways to go on that front, and the tech, while promising, still has a ways to go to be actually useful.

Will the OR usher in a new revolution in gaming?

Well, it depends on what you mean by revolution. The OR won’t be for all games, or all gamers. There’s just nothing to gain for RTSs, strategy games, MOBAs, or even traditional FPSs. In the case of an FPS, it’ll actually be a detriment. You may find that odd, since a FPS might initially seem like the logical place to use an OR (and indeed the first games to support it are FPSs) but FPSs, in their current state, are strictly designed with a keyboard, mouse, (with controller shoe-horned in) and a monitor in mind. There are many conventions in place to make up for the shortcomings of a monitor to the point where… a FPS is really a very terrible way of simulating anything. It’s just a trope that is popular nowadays, just as a RTS is a terrible way of simulating war strategy.

What will happen, though, is a new raft of VR games will be created. You will need the periphal to play, or the experience will be pretty sub-par without. Something akin to the early days of mouse-driven games. It’s always possible to play with the keyboard, but really it was designed with the mouse in mind. I expect in the early days many games will be converted current-gen games. Games like Skyrim, Mirrors Edge, and TF2 overhauled. There will also be a number of games that can really benefit from VR as-is and just need to be made compatible with the OR: DCS and related modules, Hawken, Star Citizen, Cliffs of Dover, MechWarrior Online, iRacing – basically anything where you sit in a cockpit and ride.

Off the basis of this initial success will be the VR games. Games strictly from the first person, and will run the gamut of RPG, shooter, stealth, and let’s say “other” to cover bases that I might not have foreseen.

One side-effect of these new VR games will be the desire for a new form of input. This could be solved in many ways, but the old WSAD mouse and keyboard (or controller if you’re that way inclined) just won’t cut it. Not only will it be disorienting, uncomfortable, and a little unintuitive, but it’ll also put you at a distinct disadvantage. Those players who forego VR in favour of a traditional set up will simply be out-gunned, unless you turn the OR into a simple wearable monitor… which means you’ll quickly revert to your monitor for convenience’s sake.

The point of a full VR environment will be to take advantage of what it provides, and that means an avatar whose arms, hands, and head (at a minimum) you can control. Already, there are several solutions

(and The Leap Motion)

(or hell, some related solution built into the OR. Imagine a Leap Motion, 1 more year down the track, strapped to an OR) and I don’t know which will ultimately catch on. If I had to guess, I’d guess a multiple-camera solution. Largely because I think some sort of Kinect-like set up will be quite common with the next-gen consoles, but also the solution is all-round more powerful. With it, you could put your real body in the world, or as a skeleton for a 3D avatar, and it is not limited to just speculation based on where your hands are but actually represents your full body. Also, multiple-camera setups with the sorts of algorithms this guy is playing with (and I saw some early prototypes of it about 6-12 months ago) could have many other applications:

(e.g. 1998’s Enemy of the State’s near-future technology.)

Why so sure?

VR has been tried before. It was “just around the corner” from the mid-80s through to the mid-90s, but it just never quite got there. Those headsets were essentially two tiny little displays, strapped to your head. To be fair, the OR is basically the same idea, but it’s amazing what 20 years can do. The prototype screens are 1280×800, and the consumer version is supposed to be full 1080p (≥1920×1080.) Basically, compare your modern smart-phone screen – flat, thin, extremely high fidelity – to the “portable TVs” of the 90s

By cannibalising mobile phone / tablet components the OR is able to solve a myriad of other problems of prior VR attempts – latency, field of view, etc. This is why anyone that has tried it are converts.

But I have to tell you the smartest thing they have done. That’s raise the money with kickstarter – but for the development kit. This is essentially a prototype so devs can see how it works, pull it apart, and build things for it. It doesn’t really matter that the resolution is only 2/3rds the eventual consumer version, or that it only tracks rotation rather than lateral movement. You can make allowances for that when building your game. It means that when the OR officially drops, there’ll be a ton of good content out of the gate. And I know that devs will jump into this feet first, because of the hype.

It doesn’t take much Googling to see that the hype is there. That means we can guarantee that the first few hundred-thousand units will fly off the shelves, but what could really kill the OR from that point, is if it’s a gimmick without much to do. To be honest I think the Leap Motion could suffer from this, and Google Glass almost certainly will. The OR, however, I think will have real content – AAA in the form of Hawken, Star Citizen, and TF2 – and indie (and who knows what form that will take) that will keep users begging for more. With the successful launch, you better believe more content will come. And fast. With more content will come more devices and with more devices will come competition, innovation, and hopefully standardisation. I’m sure many games will shoe-horn the OR in when it’s not really needed… but I fully expect some very exciting, immersive, and quality content as well.

One I can think of is a first-person, VR, co-op Splinter Cell; silently signalling my allies with hand-signals, as we use our silenced pistols to clear rooms. Actually, I am also thinking of a lightsaber dueling game. Oh, and VR co-op zombie survival… how about a new X-Wing vs TIE fighter…