Webex Hologram AR VR

When we introduced Webex Hologram — our real-time, photorealistic holographic collaboration solution—we showed you that hybrid work is no longer limited to flat screens, slide sharing, and video conferencing. Coworkers even thousands of miles apart can now create and innovate together as if they were in the same room. In the last year, we’ve given more than 700 demos and listened to valuable customer feedback. We’ve perfected our latest prototype, which features an array of 12 cameras that can capture your image in three-dimensional form, then transmit it to your colleague wearing an AR/VR headset in another location.   You and your colleague can also share virtual content with one another and collaborate on projects in real time. It’s game-changing, and it’s part of the future of hybrid work.

The technology behind the innovation

Holograms have been on everyone’s minds for decades. But what’s taken us so long to make it a reality as a collaboration tool? Simple: It’s quite hard to do. There are a few approaches to achieving holograms, and our team of determined and inventive engineers chose the path less traveled and most difficult. But in the end, it’s one that yields the most realistic, lifelike result: Light field technology. Light fields aren’t a new concept^[1]. It works like this: In a room are millions upon millions of tiny points of light. Each of those light points reflects a certain color, and that color can change depending on where you, the viewer, are standing. The images above illustrate how light changes depending on the viewpoint. The left image of the orange shows the view of the X straight on, and the light reflection aligns perfectly. The right image shows the same orange, but from a different viewpoint. The reflection has shifted because the viewer has moved to the right. What’s more, the viewer’s right eye is sending a slightly different image to the brain compared to the left eye. We replicate this experience using multiple cameras that capture the colors of the various light points. We combine those images with information from a headset about where your eyes are and how they’re moving. In the end, our capture device synthesizes a 3D image of what you might have seen were you physically in the room. We certainly considered easier methods. Take texture-map polygons, for example. These work by using a depth camera to measure and capture where an object is in space, creating the foundation for a 3D image. It overlays the image with a mesh of polygons, then paints a color on top of each polygon. The first problem with this approach is that colors on the polygons are based on the initial perspective of the camera. Just move your head around, and immediately everything feels off. Plus, not everything is captured well by polygons. What about wisps of hair? Or the smooth angle of a shoulder? These fine lines appear jagged, and immediately the photorealism effect is shattered. And finally, depth cameras sometimes make mistakes and might measure a certain point incorrectly. That will manifest as a huge dark spike popping out of the image. Nobody wants that. Point clouds were another potential pathway to building Webex Hologram. This technology estimates all the little points that make up an object. It computes the color of each point and where it would fall on a 3D image. As long as you have enough points, it looks solid. It still has a hard time representing finer details, and mistakes still are quite noticeable, but it can do more than polygons. Now, back to light fields. There’s something this technology captures that others can’t: reflection, which, as we know from the images of the orange above, is important to achieving photorealism. Human skin also reflects light. If you’re having a conversation with someone, that light is moving across their face as they talk. You can see the light reflecting on skin in the images above. But also notice the glass of water in his hand—how you can see his shirt through the glass, with all the magnifications that come with that. Webex Hologram, thanks to the power of light fields, provides that very real, in-person experience that wouldn’t be possible with texture-mapped polygons or point fields.

The humanity behind the innovation

But why is it so important to have the most photorealistic experience possible? Why didn’t we lean on these simpler, good-enough technologies to make this happen? Because as humans, we value in-person connection. It fosters trust and understanding. It builds relationships. It lets us rely on those unspoken social cues. If the person you were speaking to suddenly had a black spike coming out of their face, that would really get in the way of that. Our beta customers know the benefits of this in-person interaction even when far away. Creativity, ingenuity, and hard work happen across the globe; why should distance hinder meaningful collaboration? Our customers are using Webex Hologram to design, engineer, brainstorm, and teach—all in a way that facilitates deep understanding. During our early conversations about Webex Hologram, one customer expressed interest in using Webex Hologram so its employees spending long stretches abroad could enjoy time with family. That feeling of your loved ones in the room with you is irreplaceable. That’s what Webex Hologram can provide. Interested in experiencing the tool for yourself? Learn how here. ^[1] Levoy, Marc, and Pat Hanrahan. “Light Field Rendering.” Proc. SIGGRAPH ‘96.

Banner image is an artistic rendering by Lasse Roxrud Farstad

Webex Hologram