As reported on Wired.
BY RYAN TATE
SpaceTop, a 3-D desktop environment you can reach into, was shown at the TED conference today by Jinha Lee, who developed the system during and after his internship at Microsoft Applied Science. Image: TED
LONG BEACH, California – The history of computer revolutions will show a logical progression from the Mac to the iPad to something like this SpaceTop 3-D desktop, if computer genius Jinha Lee has anything to say about it.
The Massachusetts Institute of Technology grad student earned some notice last year for the ZeroN, a levitating 3-D ball that can record and replay how it is moved around by a user. Now, following an internship at Microsoft Applied Science and some time off from MIT, Lee is unveiling his latest digital 3-D environment, a three-dimensional computer interface that allows a user to “reach inside” a computer screen and grab web pages, documents, and videos like real-world objects. More advanced tasks can be triggered with hand gestures. The system is powered by a transparent LED display and a system of two cameras, one tracking the users’ gestures and the other watching her eyes to assess gaze and adjust the perspective on the projection.
Lee’s new 3-D desktop, which he just showed off at the annual TED conference in Long Beach, California, is still in the early stages. But it lights the way toward the sort of quantum leap that’s all too rare in computer interfaces. It took decades to get from the command-line interface to the graphical user interface and Apple’s Macintosh. It took decades more to get from the Mac to the touch interface of iPhones and iPads. Lee and people like him might just get us to the next revolution sooner.
Others are working along similar lines. Gesture-based control has been incorporated into Microsoft’s Kinect, Samsung’s Smart TV platform, and products from startups like Leap Motion and SoftKinect (not to mention in cinema fantasyland ). Three dimensional display interfaces, meanwhile, have been brewing at the University of Iowa (home to “Leonar3Do” ), in the Kickstarter gaming sensation Oculus Rift.
Lee’s SpaceTop weaves these two threads together, joining 3-D interface with 3-D gesture controls, a smart convergence that will likely become more common. In his talk, Lee said SpaceTop and ZeroN, which he also demonstrated, are part of a broader shift toward interfaces we can grab with our hands. Humans seem to prefer collaborating via physical interfaces; think of a scale model, map, or whiteboard. People also like interacting in multiple modalities; think of reading a book, underlining words and scribbling in the margins in pencil, and taking separate notes on a pad.
Today’s computers allow none of this, flattening all interaction onto a single screen.
“If you somehow allow computers to accept different types of modalities in the same workflow, that will be much more effective,” Lee said in an interview. “Physical activities like how you dance an how you play sports – there will be some sort of digital aid in there.”
At TED, Lee showed SpaceTop and ZeroN alongside a collapsible pen that can be pushed “inside” a computer display; as the pen folds into itself, the monitor shows the end of a pen moving deeper and deeper into the display. He also showed a concept video of a smartphone app that, when paired with augmented reality goggles, would allow the user to “try on” a virtual watch from an online store before ordering the real thing. The common thread between these systems, Lee says, is that they bring the physical world and digital world much closer together, allowing automated physical interaction he refers to as “programming the world.”
“Programming the world will alter even our daily physical activities,” he told the crowd. “With our two hands we’re reaching into the digital world.”
It’s not clear whether this type of user experience will remain stuck in a niche – embraced chiefly by say architects, geneticists and other 3-D designers and researchers – or whether it has the potential to go mainstream. People are used to gently flicking computer mice and grazing keyboards and tablet screens; do they really have the stamina to reach into their computers and flail their arms around?
Lee thinks so. He says he’s not looking to replace lazy interfaces for activities like writing email or consuming video. But 3-D interaction makes sense for certain uses cases – collaboration, design, and potential new activities Lee envisions like trying on virtual clothes.
Much of the success of systems like SpaceTop and ZeroN will ride on the details, like how much space the user must traverse and when a 3-D interface is suggested to the user. That’s why this technology deserves attention from companies that can be smart about refining it, like Microsoft, or even Apple, which popularized the computer tablet after such devices had been languishing in obscurity. For now, 3-D computing seems to be off to a good start in Lee’s careful hands.
“It shouldn’t be in the hands of scientists, it should be in the hands of normal people,” Lee says. “It’s really important to have that eye when we think about what we want to do with this to design a beautiful world. It could be anything when the power of digital escapes the screen but it’s really our responsibility to design this together.”