O
ver the last decade, video game graphics have progressed at an exponential pace. From indie games utilizing unique graphic styles to big-budget AAA games created with realism in mind, game graphics are helping to define the future of virtual experiences. Creators are able to utilize robust engines such as Unreal Engine, Unity, GameMaker, and others to spawn immersive, realistic virtual experiences with ever-increasing ease and accessibility. As a result, consumers grow more and more engaged in video games, virtual influencers, and other digital experiences.
Despite seismic jumps in the graphics industry, there are still a handful of issues that stand in the way of consumers experiencing convincing virtual human facsimiles. For one, there's a phenomenon whereby a computer-generated figure or humanoid robot bearing a near-identical resemblance to a human being arouses a sense of unease to the person viewing it, called the "uncanny valley."
The uncanny valley hypothesis was originally introduced by Masahiro Mori, a pioneer in the field of robotics. His theory was that the closer an artificial entity was to near-perfect human replication, the more humans would feel unnerved, or cold to it. Though, as with any valley, there is the potential to reach the other side, and a lot has happened in the field of virtual humans since the 1970s.
To overcome the problem of the uncanny valley, virtual humans need to look and move flawlessly. Anything less than perfect would pull a consumer out of the immersion and interrupt the experience. That is, if the consumers' expectation is for a perfectly human experience. We’re not quite past the valley yet, but the day is fast approaching when a virtual human is so lifelike, no-one will know the difference. Until then, the uncanny valley will define most virtual human experiences.
To learn more about virtual humans, their role in the video game industry, and blockers to surpassing the uncanny valley, I reached out to Doug Roble, the Senior Director of Software R&D at Digital Domain.
We met at Virtual Beings Summit last November, exploring the growing impact of next-gen avatars on social networks, commerce, and the arts.
Doug is part of a world-class team that develops software and conducts research to advance the artistry and technology for feature films and a variety of new media. Doug has won two Academy Sci-Tech Awards: one for "Track," a groundbreaking computer vision system developed in 1993 and still in use today; and for Digital Domain’s fluid simulation system which set the standard for large-scale water effects when it was released in 2001.
Doug is also an active member of the Visual Effects branch of the Academy and is the chair of the Academy’s Sci/Tech Awards Committee.
Doug, it's great to speak with you today. What do you think of virtual influencers?
I’m fascinated by them. I’ve made a career in visual effects and most of it has been spent in pursuit of work that is indistinguishable from reality. If we can achieve that, we can help realize the director’s vision and we’re less likely to pull the audience out of the suspension of disbelief.
With virtual influencers, however, it’s interesting to watch the public embrace “people” that are deliberately fake. In my limited understanding of advertising, I thought that the general goal was to show the audience a person they could relate to, using a product that suggested that the viewer could be better, or at least be more like the person they were looking at, if only they had that product.
It certainly seems that way with the whole (real) influencer culture that’s sprung up. Effective influencers seem to be people that are charming or pretty or funny (or all of the above) and they are showing off things that appeal to them—and by extension, the viewer. The hope is that the viewer thinks, “I like or am impressed by this individual. They are using/wearing/traveling to/experiencing this <thing>. I sure would like to be/hang out with that person. Maybe if I had/wore/traveled to that <thing>, I would be slightly cooler/better.”
With virtual influencers, nearly all of them are obviously and deliberately digital creations. How do they sell a “lifestyle?” Perhaps it’s because they are fake that the viewer is more able to project themselves on them. Or maybe it’s because they are often funny, outrageous or novel and the lifestyle promotion is replaced with simply getting the viewer to look.
What role could virtual influencers and digital humans play in the video game industry?
We’re already seeing digital characters that are obviously modeled after real celebrities in video games. They look and sound just like the celebrity and I find them quite compelling – as long as they don’t look super-odd. As the celebrity and notoriety of virtual influencers increases, it seems like it would be even easier to incorporate people that are already virtual into video games. Instead of trying to recreate real people digitally (which is very hard!), game creators just have to match a character who is already digital. Easy!
The exciting thing is how it’s becoming easier to create a lifelike digital character that looks like a user. We’re already seeing games where the player’s face can be used for one of the characters. We’re taking the technology we’ve developed over the years for film and adapting it so that it’s easier to create photorealistic characters that work in any environment. These characters are built to look like, sound like and behave like the person they are modeled after. If you have a quirky way of pronouncing the word “pancake,” your character in the game will have that same behavior. Being able to recognize yourself in a video game could be extremely compelling.
What’s getting in the way of creating photorealistic humans?
It’s all about the little details: how light reflects off skin, how skin wrinkles when you smile, how individual hairs cast complex shadows, or how your face changes throughout a single sentence. All these subtle details are always present in real life, and we are trained to expect they always be. Even if you’re watching a friend on a low-resolution video conference, you can tell if they’re missing some unique quality that makes them appear like themself.
Capturing and reproducing this detail is the hardest part of “high quality, live human capture,” and it’s why we’ve turned to machine learning and the most advanced graphics cards. Machine learning lets us accelerate the processes that can create, move and render the detail. The computational power in the GPUs lets us execute the staggering number of computations needed to create a photorealistic human. All in the space of 33 milliseconds.
The progress we’ve made over the last two years has been astounding. Just think of where we’ll be in another two years!
Thanks for speaking with me, Doug. Let's see what the future holds.
