Google is building a life-size 3D video call booth


Google is working on a video call booth that will use 3D images on a 3D display to create a lifelike picture of the people on either side. While still experimental, “Project Starline” builds on years of research and acquisition and could be the core of a more face-to-face video meeting in the near future.

The system was only shown via video by unsuspecting participants who were asked to enter a room with a heavily obscured screen and camera setting. Then the screen lit up with a video feed from a loved one, but in some ways none of them expected:

“I could feel and see it, it was like that 3D experience. It was like she was here. “

“I felt like I could really touch him!”

“It really felt like you and I were in the same room.”

CEO Sundar Pichai stated that this “experience” was made possible with high resolution cameras and custom depth sensors, which are almost certainly related to these Google research projects, to convert videos of people and places into essentially interactive 3D scenes:

The cameras and sensors – probably a dozen or more hidden around the display – capture the person from multiple angles and determine their exact shape to create a live 3D model of them. That model and all of the color and lighting information is then (after a lot of compression and processing) sent to the other person’s setup, which it shows in compelling 3D. It even tracks their heads and bodies to match the image with their perspective. (There’s a bit more about an early version of the technique here.)

But 3D televisions have more or less fallen by the wayside; It turned out that nobody wants to wear special glasses for hours and the quality of glasses-free 3D was generally pretty poor. What is this special 3D image?

Pichai said, “We developed a breakthrough light-field display,” likely with the help of the people and intellectual property that it got from Lytro, the light-field camera company that failed to get its own technology off the ground and dissolved in 2018.

Light field cameras and displays create and display 3D images using a variety of techniques that are very difficult to explain or display in 2D. The startup Looking Glass has created several images that are extremely impressive for personal viewing. They show 3D models and photographic scenes that really look like tiny holograms.

Regardless of whether Google’s approach is similar or different, the effect seems to be just as impressive as the participants indicate. You have tested this internally and are preparing to send units out to partners in different industries (e.g. medicine) where the feeling of someone’s presence makes a big difference.

At this point, Project Starline is still a prototype and probably ridiculously expensive – so don’t expect to have one in your home anytime soon. However, it’s not wild to believe that a consumer version of this light field setup could be available across the board. Google promises to share more later this year.


Katherine Clark