Siggraph is always rich with excitement, and this year, two tech titans made it all the more so. Nvidia CEO Jensen Huang and Meta CEO Mark Zuckerberg took to the stage for an informal chat about Philly cheesesteaks, their personal fashion, Llama, cattle, and the transformative potential of open-source AI. Some of it was in the form of friendly banter, while other topics were grounded in new announcements by the companies, including Meta’s AI Studio and Nvidia’s NIM microservices as well as a host of other news.
When two of the industry’s foremost technology leaders and innovators get together, what do they talk about? In the case of Jensen Huang, founder and CEO of Nvidia, and Mark Zuckerberg, founder and CEO of Meta (formerly Facebook), the topics ranged from jackets and their unique personal styles, Philly cheesesteaks, cattle, and, of course, AI.
First, Huang took the stage for an interview between him and a Wired tech writer in front of thousands at Siggraph 2024 as he discussed Nvidia’s road to accelerated computing and how generative AI is changing the business and personal landscape. He also managed to weave into the conversation Nvidia’s news highlights from the conference. Immediately afterward, he was joined on stage in what came off as an unscripted discussion related to GenAI, personal AI agents, smart glasses, Llama, and open source, as well as some not-so-technical subjects.
Call it an encore, double feature, or something else, but when these two visionaries spoke, the audience hung on their every word. A formal keynote it wasn’t. Nevertheless, it showed a human side to these captains of today’s tech world.
Generative AI and getting personal
For all the intellectual star power on the stage, the two executives came across as friendly and respectful of each other’s opinions and accomplishments, without any hint of arrogance—which, given their ongoing achievements, would not have been beyond the realm of expectation. At some points, the discussion delved into a bro fest, as the two reminisced about Jensen’s Philly cheesesteaks and Mark’s obsessiveness while slicing tomatoes for the sandwich during a visit to Huang’s home early in the year. Heck, they even gifted each other jackets on stage to commemorate when they tried on each other’s signature coats for a photo op at that same visit. (More on that later in this piece.)
Was the interaction genuine or contrived? Who cares; it came across as the former and added a lighter approach to the technical discussions in-between. And, it was fun for the audience.
Huang opened by noting that AI is not new to Meta, that the company has been doing amazing work in this regard for some time now, adding that the machine learning library PyTorch, used for computer vision and natural language processing, was originally developed by Meta. Since then, Meta has been doing what Huang called breakthrough work in computer vision, language models, real-time translation, and more.
Today, Meta is continuing its work on display systems, Zuckerberg said, particularly very thin mixed-reality headsets with advanced optical stacks.
“This year, we’re not just talking about the metaverse stuff, but also all the AI pieces,” added Zuckerberg.
The Meta founder said he believes generative AI is going to impact the products the company has in different and interesting ways. For instance, Instagram and Facebook have evolved from just being about connecting with friends, although ranking has always been important because a person wants certain things at the top of their feed. But over the last few years, the recommendation systems have become especially consequential, he said, because now instead of just a few hundred or thousand potential candidate posts from friends, there’s millions of pieces of content, which is presenting an interesting recommendation problem.
“With generative AI, we’re going to quickly move into the zone where the majority of the content you see today on Instagram is just not recommended to you from stuff that’s out there in the world that matches your interests and whether or not you follow the people. A lot of this stuff is going to be created with these [GenAI] tools too. Some of that is going to be creators using the tools to create new content. Some of it, I think, eventually is going to be content that’s either created on the fly for you or kind of pulled together and synthesized through different things that are out there,” Zuckerberg mused. “That’s just one example of how the core part of what we’re doing is going to evolve. And it’s been evolving for 20 years already.”
Zuckerberg sees generative AI as a big upgrade for all the workflows and products that we’ve had for a long time, but also as the catalyst for completely new things. Meta AI, an intelligent assistant built on the recently released Llama 3.1 open-source LLM, is being used for general tasks—for now. But as Meta moves from the Llama 3 class of models to Llama 4 and beyond, he believes it will feel less like a chatbot, where you give it a prompt and it just responds, and evolve to where the user gives it an intent, a mission or a problem, and it will spin up compute jobs that might take weeks or months before it comes back to you with something.
Zuckerberg then talked a bit about Meta’s vision for its AI Studio and Creator AI, which Huang called a home run idea, whereby Meta product users can create agents for themselves, and even different agents for different uses.
This concept stems from Meta’s belief that many people don’t just want to interact with the same kind of big AI out there that everyone is using. Rather, they want to create their own thing, and that is where Meta is going with its AI Studio, which began rolling out in the US earlier that day, as a place for people to share, create, and discover AI. AI Studio is built with Llama 3.1, a set of tools that let anyone build an AI version/extension of themselves as a type of agent or an assistant that their community can interact with.
Zuckerberg pointed out that many find there’s just not enough hours in the day to get everything accomplished, and something like this can offer a solution. “You want to engage more with your community, but you’re constrained on time. And similarly, your community wants to engage with you. So, the next best thing is allowing people to create these artifacts, an agent, but it’s you who trains it on your material, to represent you in the way you want it to,” he explained. Some may be a customized utility trained to get something done, some will be for entertainment, and some may be a silly representative with a funny attitude.
There is a business side to this as well, which has a little more integration and is in early alpha. “Just like every business has an email address and a website and a social media account, I think that in the future, every business is going to have an AI agent that interfaces with their customers,” Zuckerberg noted. “All engagement with this AI is going to capture the institutional knowledge, and all of that can go into analytics, which improves the AI.”
Of course, the vision that everyone and every business should be able to make their own AI resulted from Meta open-sourcing Llama, something Huang called the biggest event in AI last year. With Llama 2.1, suddenly every company—large, small, start-up—every enterprise, every industry was building AIs; it gave them a starting point. As for Nvidia, its AI Foundry was built around it, whereby Nvidia provides the tooling, the Llama technology expertise, to help turn something into an AI service that the customer uses and owns.
“It’s all triggered off the Llama open sourcing,” Huang said. “The ability to help people distill their own models from the big model is going to be a really valuable new thing. Just like on the product side, I don’t think there’s going to be one major AI agent that everyone talks to.”
After both men further extolled the benefits of open source and its effect on AI growth and development, the conversation pivoted to Meta’s vision for bringing AI into the virtual world—the next generation of the Meta Segment Anything Model, SAM 2, a unified model for real-time, promptable object segmentation in images and videos. SAM 2, which Meta introduced on the day of his chat with Huang, can identify which pixels belong to a target object in an image or video, and can segment any object and consistently follow it across all frames of a video in real time, opening the door to new possibilities for video editing and new experiences in mixed reality. SAM 2 was released under an Apache 2.0 license, making it available to anyone to use for building their own experiences.
To illustrate how it now works with video, Zuckerberg showed a video of cattle being tracked at his ranch in Hawaii. On a more serious note, the tech can be used to study coral reefs and natural habitats, as well as the evolution of landscapes, for example. And that is just the tip of the iceberg—which also can be tracked.
Here comes the shade
Before the jacket-gifting referred to earlier, Zuckerberg touched on the next computing platform involving smart glasses—that being Ray-Ban Meta smart glasses with Ray-Ban frames, which have an ultra-wide 12MP HD camera and five-mic system for capturing high-quality photos and immersive videos that can be imported to a smartphone via the Meta View app. These kinds of glasses make it easier for people to embrace due to their comfort and sleek look, and he boldly predicts that everyone who is wearing a pair of prescription glasses today will eventually upgrade to smart glasses.
Zuckerberg’s opinion is that there will still be a place for VR/MR headsets for gaming and other uses, likening smart glasses as the mobile phone version (that’s always on) of the next computing platform, and the mixed-reality headsets more akin to a workstation or game console, where the user is sitting down and having access to more compute power for a more immersive session.
Meta is still plugging away, building the technology needed for ideal holographic AR glasses, Zuckerberg said—all the custom silicon work, the custom display stack work, and so forth. Bottom line: The Ray-Bans are still limited in terms of the technology that could be added in order to achieve a full holographic AR experience. “We’re getting close, and in the next few years, I think it’ll start to be a product,” he said.
Meanwhile, the company is partnering with top-notch glasses makers to include other brands in the second generation of the smart glasses and will add as much tech in them as possible. Right now, there are camera sensors in the smart glasses to take photos and videos. Users can livestream to Instagram and take video calls on WhatsApp and stream to the other person. There’s a microphone and quality speakers with open ears (no earbuds) for added comfort.
As it turns out, the sensor package in the offering is exactly what’s needed to talk to AI, Zuckerberg said.
“It was sort of an accident. If you would have asked me five years ago, were we going to get holographic AR before AI, I would have said ‘Yeah.’ It’s just the kind of graphics progression and display progression, and all the virtual and mixed reality stuff and buildup of the new display stack, that we were making continual progress toward,” said Zuckerberg. “And then this breakthrough happened with LLMs, and it turned out that we have sort of a really high-quality AI now, and it’s getting better at a really fast rate [compared to] holographic AR. It’s this inversion that I didn’t really expect.”
Because Meta is working on so many different products, Zuckerberg sees a range of different glasses and offerings becoming available at varying price points, with different levels of technology in them. In particular, he sees displayless AI glasses at a $300 price point being an especially popular product—his prediction based on what the company is seeing now with the Ray-Ban Metas. With an upgraded display, however, will come added weight and an added cost, but it will offer a full holographic experience in thin glasses. That, however, is a ways off, he believes, unlike a version with chunkier frames.
The talk of stylish AI glasses prompted digs by Huang about Zuckerberg’s own chosen style—his T-shirt, chain.
On a serious note, Huang wrapped things up by telling the audience that today, the entire computing stack is being reinvented, and the way we compute has changed from general-purpose computing to generative neural network processing. Generative AI, he said, “has also been able to cut across every single field and is going to make a profound impact in society in the products that we’re making.”
And in case you were wondering, Huang’s new leather jacket was a good fit on Meta’s CEO, as was the new jacket on Huang that Zuckerberg gifted to him—a custom black leather version (with a hood) of the Western-style shearling coat he had worn while visiting the Nvidia CEO earlier in the year.
Both men, you might say, also are cut from the same cloth in their continuing efforts of generative AI evolution and solutions.