Here is a film that is extremely ambitious. It is a time-travel odyssey spanning a wide swath of time shown through visual cues including dinosaurs, early Native Americans, Colonists, imagery from the 19th and 20th centuries, and more. The main “character” in the film is not a person, but a place, with a focus on multiple generations of a certain family occupying that space. Telling this story is Director Robert Zemeckis, who again turns to complex digital techniques—in particular, aging/de-aging—to bring the story to life on screen.
For decades, Director Robert Zemeckis has been turning filmmaking on its head by applying state-of-the-art technologies to create innovative productions. This includes the use of performance capture to bring CG characters to life in The Polar Express (2004), Monster House (2006), Beowulf (2007), A Christmas Carol(2009), and Welcome to Marwen (2018). In a joint venture with Walt Disney Studios in 2007, Zemeckis opened ImageMovers Digital, which was committed to using performance capture. It was here that Zemeckis would produce and direct 3D animated films using the technology.
While the studio’s projects were recognized for their technical achievements, not all received rave reviews—a disappointment considering their production costs. And so, in 2011, the studio closed, with its last release being Mars Needs Moms, a big box-office disappointment. However, the production company ImageMovers (minus the Digital) would continue on.
Also known for many VFX-infused live-action hits like Back to the Future, Forrest Gump, and Who Framed Roger Rabbit, Zemeckis continued his push to bring something new to filmmaking. His most recent endeavor is the film Here, released in November 2024. The live-action feature, co-written with Eric Roth (Forrest Gump, Dune, Killers of the Flower Moon, The Curious Case of Benjamin Button), once again broke the typical filmmaking mold.
Here is told in what can be described as a disjointed narrative, with the plot lines and events unfolding in a non-parallel, non-linear way—no easy task when it comes to films. The story evolves around “place,” as opposed to a particular character.
The story moves through generations of families (one multi-generational one in particular), with the consistency being the one location. (To help audiences remember the name of the film, or perhaps the underlying concept, the word “here” is iterated and reiterated in the dialog throughout the film.)
The camera takes audiences inside a house built in the early 19th century and remains stationary in a fixed spot in the living room of the home once it is constructed, with the large living room window always present in the background. The imagery shown outside the window reflects the time period of the scene. Unreal Engine was used to drive that background imagery, which was projected onto an LED screen. Because the background imagery would be shown at 4K, it was upscaled using the Magnific AI-powered upscaler, transformer, and generator tool to sharpen the imagery and increase the level of detail.
Flashbacks take the audience back in time to that particular locale through the years, decades, centuries, and even millennia, although the plot doesn’t always unfold linearly. The film opens multiple windowpanes on the screen to quickly time jump, showing various events unfolding at the location during various time periods until, finally, slowing down to settle on a family living on the plot of land, starting in the mid-1940s and taking us through their lives to present day. The main characters include father Al (Paul Bettany), mother Rose (Kelly Reilly), their son Richard (Tom Hanks), and his wife, Margaret (Robin Wright), from the time Al and Rose purchase the house in 1945, to 2024, when the elderly divorced couple Richard and Margaret visit the house as it is being sold by the most recent owners.
To tell this complicated, time-traveling story required a complex digital aging/de-aging process, among other visual effects and CGI techniques.
Metaphysic’s new aging/de-aging technology
The digital transformations of the characters were performed by Metaphysic, an AI-powered production studio, using its Metaphysic Live tool. VFX wunderkind Kevin Baillie was visual effects supervisor on the film and worked with Metaphysic to digitally augment performances, tapping the company’s proprietary generative artificial intelligence technology. Metaphysic said it spent significant time developing the architectures, neural performance toolset, visual data science recipes, and artistic fine-tuning to create the high-resolution outputs that are true to the likeness and performance of the actors.
After seeing Metaphysic’s previous work, Baillie invited the studio to screen-test for the project in 2022, re-creating a young Tom Hanks, reminiscent of his appearance in Big, while maintaining the emotional integrity of his contemporary performance.
In a podcast from fxguide, Baillie explained that traditional de-aging methods using CG or makeup were not viable because of the number of scenes requiring de-aging and the range of the aging, as well as the required emotiveness of the performances in this emotional story. The budget for this amount of VFX alone (53 minutes of face replacement) would have been prohibited, Baillie had stated in the interview.
In all, the character played by Hanks is shown at five different ages, Wright is shown at four, and Bettany and Riley each at two. Achieving this through traditional computer graphics techniques, involving 3D modeling, rendering, and facial capture, would have been impossible given the scale and quality required for Here, Metaphysic pointed out.
As Zemeckis noted during a video interview on IMDb, they were at the right place at the right time in making the movie, as there was a large library of film archive of Hanks and Wright at different ages (including face movements, skin textures, lighting variations, and camera angles) from films they all did together, including Forrest Gump. “We can process it and put it on like makeup,” he said.
How it works
Metaphysic’s proprietary process involves building neural network models to help convey the lead characters in the film at different ages, resulting in the need for training multiple models for each actor at their various ages.
For Here, the models were trained on footage and images of the actors, such as Hanks, at different ages, followed by refinement by a team of digital character artists led by Metaphysic’s VFX supervisor, Jo Plaete. The artists continued refining the model until it was deemed ready for production. From there, an actor or performer could drive the model.
The AI facial transformations occur instantly in-camera as the scene is being shot, eliminating the extensive post work that had been a hallmark of prior de-aging effects in films. As a result, Plaete and his team were able to run the workflow in real time during the shoot, providing a visual reference of what the performers would look like, which became an integral part of the filmmaking process. This allowed Zemeckis to view both the raw camera feed and the digitally augmented feed with the actors’ younger faces while on set, with only about a six-frame delay, and provide direction accordingly.
Metaphysic also set up a camera and monitor system during the shoot that allowed the actors to rehearse while seeing themselves as their younger selves in real time. According to Metaphysic, the “youth mirror” system only had a two-frame delay and provided the actors with feedback that helped them fine-tune their performances to better match their younger selves.
Baillie was in constant communication with the team to review and refine the performance transfer, and Metaphysic’s artists were on deck to correct eyelines or tweak the amplitude of emotions portrayed by each character.
While the neural network models used for the real-time outputs generated photoreal results, Metaphysic artists needed to further enhance them to ensure they held up to cinematic 4K standards. (At the end of the movie, prosthetics were combined with the digital makeup for a more realistic result of the elderly couple that included skin translucency and fine wrinkles.)
An innovative epic, Here pushes the boundaries of filmmaking from a storytelling, creative, and technical perspective.
“Working with forward-thinking filmmakers like Bob Zemeckis and Kevin Baillie was a privilege. Their faith in this novel technology pushed us to new heights and allowed us to deliver on their ambitious vision,” said Plaete
The power of AI
Indeed, de-aging is new-ish but not brand-new, gaining quite the attention in 2019 as several lead actors in The Irishman, Gemini Man, and Captain Marvel received digital face-lifts to play themselves at much younger ages. This work was performed atIndustrial Light Magic, Weta Digital, and Lola, respectively, using various methods with some AI assist involved. But that occurred in the earlier stages of AI and ML technology and prior to the founding of Metaphysic in 2021 and the rise of more sophisticated AI and ML tools.
“You couldn’t have made this movie three years ago,” Zemeckis said in a New York Times interview.
Once again, however, Zemeckis’ technical ingenuity and penchant for novelty may be ahead of its time. Or, possibly, his technical concepts are not yet in sync with visual delivery and/or storytelling. Either way, the film received mixed reviews. On Rotten Tomatoes, as of yesterday, it had an average rating of 6.5 out of 10, an increase from 5.2 a month ago; on Metacritic, it received a score of 93 out of 100. However, the reviewer on Roger Ebert.com pans the film as being hollow and lacking depth and emotion—and he is not alone in that sentiment.
And with a reported budget of $45 million to $50 million, Here has only grossed about $15.7 million worldwide at the box office to date. A costly experiment to say the least, but one that may help usher in the next stage of filmmaking and visual effects.
LIKE WHAT YOU’RE READING? INTRODUCE US TO YOUR FRIENDS AND COLLEAGUES.