A new 3-D display lets you reach in and touch virtual objects

Floating displays you can reach into and manipulate, like the ones Tony Stark uses to design his super suits in the Iron Man movies, are one step closer to reality.

A new device renders 3-D graphics that users can grab, drag and rotate. Such interactive visuals — which can be seen without a VR headset — could help create new hands-on educational tools or museum exhibits. They might also be used to make 3-D artwork or video games.

In the past, machines have rendered 3-D objects by sweeping a flat screen up and down and projecting different 2-D slices of the object onto the screen at different heights. When the screen moves extremely quickly, those slices blur together and look like one continuous shape.

“It’s like a real 3-D object. You can see it from all directions,” says human-computer interaction researcher Elodie Bouzbib. But you couldn’t touch the fast-moving screen without hurting yourself or damaging the device.

Bouzbib and colleagues at the Public University of Navarre in Pamplona, Spain, replaced the flat screen with a row of elastic strips like the ones used in the waistbands of stretchy pants. Users could then reach down into the display, fingers slipping through the oscillating strips, to touch virtual objects. Cameras tracking the user’s hand allowed them to pinch, swipe, spin and otherwise manipulate the graphics.

Eighteen volunteers tried the display, controlling virtual objects with both their fingers and a 3-D mouse. People were able to make selections, drag and drop objects and trace paths faster and more accurately when using their fingers. Most also said it felt easier and more natural to handle virtual objects directly, and several expressed their surprise that the device felt nice to the touch.

“It feels really soft, actually. It tickles a bit,” says Bouzbib, whose group will showcase the device April 30 at the CHI Conference on Human Factors in Computing Systems in Yokohama, Japan.

“What really gets me excited about this research is that the display is reasonably sized and interactive,” says Tatsuki Fushimi, an engineer at the University of Tsukuba in Japan who has created 3-D graphics using particles levitated by sound waves. You can touch those displays, he says, but it’s hard to use levitation to make images bigger than about a centimeter across.

The elastic band display is 19 centimeters wide and 8 centimeters deep, or about the size of a square Tupperware container. “Now, people can start to experiment and play around with it,” Fushimi says. That could bring in fresh ideas for how to use 3-D displays. The volunteers in this study suggested everything from building control panels for surgical robots to viewing products in 3-D while online shopping.

“If you want to make it bigger, it’s probably going to face issues,” Fushimi says of the elastic band setup. “Imagine it room-sized. Then you can no longer reach through the display.” That would render it useless for building something like a Star Trek holodeck. But Fushimi could imagine it being scaled up to a desktop-sized device.

Bouzbib is also interested in adding touch, or haptic, feedback to the display. For instance, a focused beam of ultrasound waves could create sensations at a user’s fingertips as they handle virtual objects. Her team is also brainstorming how they might project images onto a layer of gas rather than elastic strips, for even more seamless interactions with midair graphics.