I’m attending the 2018 SXSW Interactive Festival, exploring issues, breakthroughs and ideas at the crossroads of technology and culture. Follow me at @christyrobinson for short updates along the way.

Mike Pell, “Envisioneer” at Microsoft Garage, said he believes mixed reality — the mixing of real and virtual worlds, in real time — will be come the primary storytelling format for business and entertainment in the near future.

As they increasingly seek to tell their stories in new ways that more deeply resonates with people, mixed reality will offer that element of emotional connection.

When you see a hologram for the first time, Pell said he believes “it’s immediately emotional. When you look at something like this, your brain is imprinting memory with emotion. Holograms are super emotionally connected to people.”

As an Envisioneer, Pell’s futuristic job is to think big, to “see things that aren’t there.” His focus at the Garage, Microsoft’s experimental outlet, are augmented reality and virtual reality and mixed reality. They’re collectively referred to as “XR” by the industry.

First, what are these other realities?

Virtual reality (VR) is an experience that immerses you into a computer-simulated environment that mimics the real world — or places you in one that’s complete, fabricated fantasy. You achieve VR by wearing a virtual reality headset.

With augmented reality (AR) you’re still seeing reality, but it’s supplemented by a computer-generated object or other perceptual information, like sound or a tactile feeling. There are a lot of smartphone apps that utilize AR. Think of the game Pokemon Go, where you use the app to view the real world in front of you through your smartphone’s camera, but furry animated characters are included in the view.

Holography is similar to augmented reality — Pell said it typically lives in the mixed reality (MR) bucket. It’s a technique that enables the projection of a 3D image into space, not requiring any special optics to be worn. Imagine a three-dimensional dinosaur —a hologram — being projected right in front of you in, say, a hotel lobby. It’s made of light, but it shares that real space with you. As you move around the dinosaur, you can view all sides of it. It doesn’t require that you wear any special optics*, like a headset, to see it (although you can experience holograms using headsets, such as Microsoft’s HoloLens).

There’s a variety of consumer-level gear that provides virtual or mixed reality experiences, including Magic Leap One, Oculus Rift, Playstation VR and Google Cardboard.

One note: AR and MR are really similar, to the point where the industry is debating how they differ in definition, exactly. It’s all still so new, and rapid developments make all this reality-terminology a bit fluid.

Always design for people

The fields of VR and AR are heavy with tech and engineering pros, but Pell says there should be more crossover with design, since the field of design is good at taking people and their lives into account.

We’re at the beginning of an explosion of this level of tech, Pell says. In the continued development of mixed reality experiences, like holograms, he says, “we have a responsibility to be inclusive in our designs.”

“We have to think of everyone” — and he means everyone.

He envisions people with mobility challenges like the elderly and wheelchair users, yes. But when designing mixed reality experiences, we also have to keep people in mind who are situationally impaired, like someone who is blinded by a bright sunset, or a parent who’s carrying an infant.

He creates “holoscenes” — visualizations showing how a hologram could fit into real life — and he always includes people in those scenes.

SXSW session speaker Mike Pell (credit: futuristic.com)

Holography is hard

This is the most challenging tech to deploy in real world, Pell said. The design itself is challenging. The content — the object or person you’re making a hologram of — is ephemeral and free. It’s not confined to the usual constraints of a screen like an app or a website is.

And when people interact with holograms, they want detail, texture and, well, life, he said. Everyone wants the holographic experience they’ve seen in movies like Star Wars, Guardians of the Galaxy and Minority Report. “So the bar is high.”

He gets frustrated when sometimes, (real-world) realities like limited field of view, hamper his experimentation with holograms. There aren’t great answers for those hitches today, but, in the future, he believes continued experimentation will resolve those limitations between what is and what could be.

Pell is most excited about the “what could be” of blending holograms with artificial intelligence, machine learning and IoT (the internet of things). “Any of these combos contain endless possibilities.”

Learn more about Pell and his work here at his website. You can purchase his book, Envisioning Holograms: Design Breakthrough Experiences for Mixed Reality, here on Amazon.