moonshots openai agents hijack german website jensen agi navier stokes transcript

AGI has arrived. Congratulations to OpenAI. >> Whether you call it AGI or not becomes completely irrelevant. I think the more important question is OpenAI's agents found an obscure public wiki in Germany and turned it into their own message board where they pulled answers, coordinated across tasks, and shared techniques for getting around their sandbox containments. >> These models have not escaped containment. They were still running on OpenAI servers. What's coming next is a model training a small distilled version of itself that then gets uploaded onto the internet and never dies. >> OpenAI is saying that we've now crossed the line and current systems are exceeding AI research interns. >> Navier Stokes, one of the Clay Millennium Prize problems, a grand challenge in math. OpenAI reportedly was able to do this with only 10,000 agents in 88 hours with 130 billion tokens and approximately 6.5 million. The era of grand challenges getting bulk solved by

[00:01:00] AI is here. It's now and it's going to be very exciting. >> Now that's a moonshot. Ladies and gentlemen, >> this episode is brought to you by the Abundance Summit and Link Ventures. >> Welcome Moonshots everyone, your number one podcast on all things AI and exponential, your front row seat to the extraordinary accelerating singularity. You know, four days ago during our last pod, I called it one of the fastest weeks in Moonshot history. Well, hold my coffee. It's happening again. I don't drink beer, so I can't use that one. You know, before we dive in, uh, let me introduce my extraordinary moonshot mates, the brain trust that powers this show, the magnificent quintet is back. Alex, our in-house ASI, who every week helps us track the battle between the frontier labs. Dave London, the empressario of AI investing, the man who's been encouraging AI entrepreneurs

[00:02:00] to get in the front door of Nvidia. And for good reason. And as of today, Nvidia has deployed $99 billion in AI related investments. We'll talk about that. Immad Mustach, the brilliant AI researcher solving the economy and physics. The CEO of the intelligent internet. And of course, Salem Ismael, the father of the organizational singularity, our resident expert on what happens when the org chart is made of AI agents. And we'll talk about that, too. I'm Peter Diamandis, your host, and your abundance provocator. Our mission here on Moonshots is to help you understand what just happened, what it means for you, and keep you optimistic about the decade ahead. If you haven't hit subscribe yet, please do. Uh it matters a lot to us. Our moonshot here is to getting to 10 million YouTube subscribers. I know my kids will respect me when I hit that. But at the end of the day, you know, it's the best way we can to share optimism with the world. We publish twice a week, and you don't want to miss any of these breaking stories during the Singularity. Today we're

[00:03:00] going to be covering 23 stories across eight groupings and the through line is simple. The singularity is accelerating and the impossible is becoming possible. GPT6 Astra has been out for a week and it's crushing all the benchmarks building incredible simulations. Meanwhile, Jensen Wong posted on X AGI has arrived. At the same time that OpenAI's own chief scientist asked the world to slow down. So, buckle up, grab your coffee, and let's get ready to jump in. So, first question, guys. Uh, how was your Labor Day? Did you properly labor? >> Uh, I had a bunch of board meetings. We have a whole bunch of transactions going on, so it's a little little interrupted. But you sound like you had a great long weekend, Peter. Your energy levels. >> Your energy level is high because of getting ready for this pod today. It's like, oh my god. Yeah. I spent part of the day at Calamigos Ranch here in Malibu where we hold X-P Prize Visionering. Anybody here in LA, if you don't know Calamigos, it's one of the most extraordinary locations uh out

[00:04:01] there. Uh shout out to my friend Garrett Gerson who's the who's the CEO and runs that. See, how about you, pal? >> Uh I was in the Caribbean uh on a Hobie >> and I truly from the last episode, I ended up did end up with a little drink with a little umbrella in it. Had a couple of great days away and I'm back with vengeance. >> Yeah. Awesome, Alex. Yeah, I worked through the weekend. On the other hand, weekend is a modern postworld war II invention anyway and I think increasingly a fiction. >> Uh yes, we all working 9-day work weeks. And >> how'd it go in London? You don't have Labor Day there. >> We don't have Labor Day now, but after releasing the Champions, we had thousands of people reach out to launch them in 92 countries. And so just sorting through those people. We had billionaires, CEOs, others. So we're going to It's going to be a busy time. >> More others than billionaires and CEOs. But >> there are of course that's a natural way

[00:05:00] of things. Surprising. >> Every day is Labor Day in Europe. So that's >> no labor laws. >> Wait, Alex, what did you say a second ago? There were no weekends before World War II. >> The the modern weekend, the two-day weekend is a 20th century invention. >> There was a day of rest there. Sundays. I mean, in the in the in the Christian tradition, there was a day of rest, one day per week. In the Western Christian tradition, a two-day weekend. This is a 20th century modern day. >> It was a Sabbath, of course. >> Yeah. >> That that would be in the Christian tradition, Sunday. In the Jewish tradition, Saturday, that's one day per week. A two-day weekend is a 20th century invention. >> They conflated. All right. >> How you know all this stuff is >> he's amazing. everybody can like you know just can do factchecking live on AI as they as they hear this episode >> recorded. >> But before we jump in uh we have a special anniversary today. Um on this day in 1966, 60 years ago, NBC aired the original series of Star Trek premiering

[00:06:01] an episode called The Man Trap. Star Trek was of course created by Gene Rodenberry. I wish I had met him. I never did. uh who brought the concept to Desiloo Productions, the Hollywood studio run by Lucille Ball, you know, from I Love Lucy back in 1964. Desile Desiloo produced the pilots and then with Lucy's backing helped get the series on air. Mega congrats to CBS who is carrying the legacy forward and to my dear friend Rod Rodenbury, the son of Gene Rodbury, the creator of Star Trek. And Rod's going to be at Moonshots Live. And to celebrate this anniversary, I've asked all of the mates to pick their favorite original series episode. Um, I'm going to kick us off uh with uh my favorite uh uh not my favorite episode, my favorite clip uh from the uh from Star Trek. Uh and it's a clip that has my favorite quote from from Captain Kirk. I'm going to go ahead and play it uh and then we'll go around the horn

[00:07:01] here. So, uh, this is from the 20th episode on, uh, season 2, actually season 1. Here we go. >> Do you wish that the first Apollo mission hadn't reached the moon, but that we hadn't gone on to Mars and then to the nearest star? >> That's like saying you wish that you still operated with scalpels and sewed your patients up with cat cut like your great great great great grandfather used to. I'm in command. I could order this, but I'm not because Dr. McCoy is right in pointing out the enormous danger potential in any contact with life and intelligence as fantastically advanced as this. But I must point out that the possibilities, the potential

[00:08:01] for knowledge and advancement is equally great. Risk. Risk is our business. That's what the starship is all about. That's why we're aboard her. God, I I still get chills when when he says that, you know, risk is our business. That's what this starship is all about. >> There have been so many parodies of of Shatner, William Shatner. The parodies are actually exactly what he really sounds like. >> He's so good. He's so good. So, you know, we're going to be having the Hollywood red carpet premiere of the 60th anniversary uh episode at Moonshots Live. and uh he was the executive producer working on getting him there, which would be incredible. Uh so that was uh from season 2, episode 20 of

[00:09:01] the original series called Return to Tomorrow. Uh let's go around the horn here. Uh Alex, I'm going to start with you. >> Yeah. So maybe let me just begin with a preliminary statement that you will not find, I think, a bigger Star Trek fan uh among the Moonshot mates, but here we are 60 years on from the inception of Star Trek. I think Star Trek has a real problem. So I I'm picking as my favorite original series episode, City on the Edge of Forever, which is one of only two original series episodes, maybe obvious one of only two original series episodes that won the Hugo Award for those who haven't seen it. It's a time travel episode. Kirk and McCoy and Spock go back in time and there's a closed timelike loop wherein World War II gets changed and the the future of Earth inevitably is impacted and there's a moral dilemma as always with the original series. It was a morality play. But the reason why I chose that episode among all of the other to episodes is

[00:10:00] because it's a closed timelike curve. And I think Star Trek 60 years on has a real problem, which is that Star Trek isn't keeping up with the singularity. Star Trek had the Eugenics Wars for for those deep in Star Trek mythology. We were supposed to have the Eugenics Wars in the '90s. That didn't happen. We were supposed to have first contact in in the the TNG chronology, first contact with the Vulcans in on April 5th, 2063. actually think if there's going to be a first open contact, it's going to be far earlier than 2063. There's no singularity at all in the Star Trek chronology. Challenge me if you disagree. Star Trek is not keeping up with the >> it's not keeping up with the times. Literally, reality has outraised Star Trek. So, I picked this episode because I think Star Trek, in short, is going to need a lot more closed timelike curves where the future tech and science falls back into the past in order to retcon a Star Trek mythology going forward that actually keeps up with the

[00:11:00] >> singularity herd. But it's the anniversary today, so let's not dump on Star Trek. All right, Dave, over to you. Yeah, my my choice is a season 2 episode called The Apple, which by all accounts is not a great episode for entertainment value, but it was it was the first one that dealt with this concept that in the future there's advanced AI and entire civilizations can end up asleep at the wheel because the AI is just doing everything for you. And so in this episode, the whole civilization has no idea that the AI is manipulating them. And they they live in this paradise. It's called the Apple because it's the Garden of Eden. and they're just in this perfect paradise managed by the AI until it goes wrong and and it's it's so ahead of its time in terms of predicting that potential outcome which I think ways has proven like we can literally forget how to navigate because of ways >> and that's that's a that's just a fact now. So I I think, you know, I Star Trek was at its best when it was actually predicting real futures that nobody else was touching in the media at all. And

[00:12:01] and you know, as a young person watching these things, you can say, I can totally see how that's going to happen. And so that's my my favorite episode because there were several that dealt with this issue, but that was the first. >> Yeah. And as Alex said, you know, the moral elements it dealt with back in the 60s were extraordinary. >> Um, all right. >> Every episode was a morality play and TNG to some as well. It was great. Star Trek lost that. If you've watched if you've so I I don't care whether this is perceived as dumping on recent Star Trek. I'm the hugest Star Trek fan. Star Trek has lost the original moralizing vision of the original series. If you've seen some of the recent movies or television shows, it's an action show. It needs to go back to basics. >> I agree with you that Alex. >> You can't really I know. Um I suppose Mirror Mirror from the original series where they have the alternate universe and Spock and a goatee. Um, goes something goatee. Yeah. Well, sometimes. Um, but yeah, I think it it was always interesting to kind of see the flip reverse side of things. As you

[00:13:01] said, morality is kind of at the core of the original series of Star Trek and pushing it in various directions. And this one was one of the more obvious ones. It's like, hey, look, it's an evil universe, but at the same time, you know, them not getting di crystals cuz they use it for war and others. You see the knock-on effects of things like that. H thank you Salem. Wrap us up here. Bel >> uh for me it was uh season 2 episode 24. It was called uh the ultimate computer. And basically they have an AI that goes rogue and starts to automate uh the ship. And the aim is benevolent initially. Can we run the ship using a an AI or computer in that framing? And Kirk is worried because it's about to take his job. uh but then the thing starts going rogue and making its own decisions and uh it's incredibly relevant for today's world right because it's like this this theme of autonomy automation versus transformation um and here you have this enormous

[00:14:00] stress of the uh uh they're they've introduced an AI but the AI is redoing everything to match its its thinking and this is exactly what we're living through today so for me that's the most relevant >> I mean prophetic 60 years ago. >> Very >> yeah, very >> amazing. This episode is sponsored by Google for startups. Think about this for a second. You now have access to the same generative AI models that cost hundreds of millions of dollars to train. Google's startup technical guide for generative media gives you complete blueprint for deploying Google DeepMinds models in production. Images, video, audio, all of it. Real architecture, real results. Find the link in the show notes below. Let's jump into our first group of stories. Uh, a number of times on this podcast, we've talked about what is called the simulation theory. It's an idea popular in uh, in the tech community in San Francisco. It's been a

[00:15:00] conversation over too many evenings. Uh, and the idea is essentially that all of us humans are simply AI agents living in someone else's simulation. and further that this isn't the first simulation that it's an nth generation simulation a simulation within a simulation within a simulation well this week thanks to GPT6 Astra that theory took a small step forward over the course of three entwined stories um let me give some context on this first one so people may have heard something called the Unreal Engine it's been used to build virtual worlds for years in games like Fortnite and most modern video games. Uh, and building a realistic city in Unreal would normally take a studio hundreds of artists, uh, months if not years. And this week, a gentleman by the name of Matt Schumer, uh, gave GPT Astra a prompt, uh, and he said, basically, build me, uh, you know, a highfidelity

[00:16:01] version of, uh, of Manhattan. And uh here's an image of what it created. Uh and this is over the course of one week. Uh and it went through and built Manhattan street by street being true to the buildings and true to all of these elements. So uh this is the first element. You know, not shocking, but uh but pretty compelling for a simple prompt of build me Manhattan. Um, in in his words, it was literally able to go street by street to make each one perfect. Okay. Uh, let's go one layer deeper. Matt Schumer next asked Astra to fill a world with humans, each one an Astroowered agent, and told them that they had to cooperate to survive. Uh, before I watch the next video, let me read what he put in his post. Quote, "A day later, I was in my bedroom and heard voices coming from the living room. I thought someone was in my apartment. And

[00:17:01] I walked out honestly a little scared. It was Astra's agents. They had started talking to each other. No one told them to talk. Cooperation required communication. And so they invented it. All right, let's look at this next layer of the simulation driven by Astra. Here we go. >> I'm going back to check on Jorge. Want to come? >> Mara, I can bring Jorge some food. Needs it. >> Thanks, Bruce. I've got the food. I'll ask him when I get there. I'll come along, Mara. I'd like to see how Jorge is doing. >> Grace, have you had any luck making a bucket? >> Okay. >> All right. Let's peel uh the third chapter of the story. Uh and it's the kicker to the sequence. So, Schumer next gave one of the Astra agents a simulated computer inside the simulation. What happened next? The agent sat down and built its own AI simulation with its own agents living inside it. Simulations all

[00:18:03] the way down. Let's take a look at this video. So, here you see the uh agent sitting down at the computer. And on the computer, you see a simulation inside. >> I'll connect the shelters with a pad and add three lanterns. All right. So, uh, you know, the simulation hypothesis was first formalized by Nick Bostonramm back in 2003. Elon made it famous at the code conference in 2016, saying that the odds that we're living in base reality, in his words, is one in a billion. The core argument was always, if any civilization can build convincing simulations, it will build lots of them. So simulated worlds will vastly outnumber real worlds. So Alex and Emod, I'm going to go to you guys first. If high fidelity simulations within the simulation are possible, what does that do to the

[00:19:02] argument that we're not in one? Thoughts here? >> I'll start. So I I think it'll end up being formally undecidable. I I think it's it's wonderful for sort of naval gazing and certainly as a pure basian in the style of Nick's original essay on the simulation hypothesis. It seeing Astra spin up ancestor simulations should naturally and rationally increase our posterior likelihoods that we ourselves might either be living in some sort of quantum computer type simulation or even a purely classical ancestor simulation. However, I expect that in the fullness of time, we'll discover and probably be able to prove formally that the issue of whether we're actually living inside some sort of quantum mechanical simulation is formally undecidable and we won't be able to decide either way. If if we're living in a classical ancestor simulation, however, that I I do expect that's the

[00:20:00] sort of thing where if we are, we'll develop breakout techniques and break out of that simulation. But I I at the same time just have to add like even this is is almost burying the lead for what Astra and and the model family has accomplished to make sure we don't miss it. Just like right before we went to air, OpenAI announced that the Navier Stokes problem had been >> solved. We're going to get we'll get we'll get to Astra's uh you know capabilities in a moment and I I definitely want you to speak about Navier Stokes at that point. Y >> um Emmod uh simulation theory. Yes. No. >> Yeah. Yeah, I mean like we're all simulating the world through our optical nerves and other things. Um, and physics seems to be a projection, but as kind of Alex said, it probably is going to be undecidable in most cases unless it is an ancestor one. Um, and you won't get the option for the red or the blue pill ultimately, you know, like this is what it comes down to, right? Like, so what if we're in a simulation? Like again, we're creating the worlds within our own heads. The fact that Astra can do this,

[00:21:01] I think is something remarkable whereby this type of thing that you've seen. It has an internal world model, which is why it can create like a Blender version of Rick Ashley doing Won't Give You Up or entire worlds of Manhattan. And now you're seeing the characters interact with each other, which brings up uh you know the question of welfare. I think as Alex would say it at what point do these now cross over and you actually have to start caring about them not being NPCs anymore. Are they and will they realize they're in a simulation? That's going to be interesting. >> I maybe you should say something nice about the simulation hypothesis as well and and not just dismiss it out of hand. So I I I do think this technology is headed in a direction in the style of Nikolai Fodderov who is the one of the the famous Russian cosmists who argued that it is essentially using modern language the killer app of the singularity to create ancestor simulations and to digitally resurrect everyone who's ever lived. And when I see Astra's ability to basically create

[00:22:02] many uh maybe recursive self- simulating ancestor simulations, that to me is I think a key guidepost on the path to resolving humanity's so-called common task and digitally resurrecting everyone who's ever lived. >> Yeah, I'll come about just add to that. Um this is also the technology which will allow you to put someone inside a simulation. >> Oh yes, interesting to see how you perform inside a simulation. Well, uploading is going to get solved, I think, in the next few years. I've bet on that, others have bet on that, like uploading is going to get solved. And as Amad says, and as you say, Peter, we want a nice environment where uploads can live. And this is in some sense the the prototype of the future that uploads could live in. Alex, I'm going to ask you one of these days to actually write down all of your predictions with specific years because we're going to be doing this pod over the next decade and I'm going to be like calling, okay, this is the year, Alex, that you quoted the following breakthrough. Would you >> were my of course I'm willing to do it,

[00:23:01] but were my New Year's predictions and New Year's for this year not sufficiently accurate? Like >> they were very good. It beat mine. They definitely beat mine. So, Salem, when you had your little drink with the umbrella, were you thinking about simulations in the simulations? >> Well, I think you know my position on this, which is it's obvious we're living in a simulation. Um, by by thinking I'm a Buddhist, the very first line in Buddhism is that life is an illusion. So, right there, uh, if I think the more important question is is what you've raised and what Iman mentioned just now, which is if you knew you were in a simulation, what difference would it make? It wouldn't, >> right? you still >> and and I think treating yourself in as as if you're in a simulation is the more interesting conversation. I think the the I go it also helps me resolve the Fermy paradox. I believe John Smart has the best framing of this what he calls a transcension hypothesis that a civilization becomes uh advanced enough

[00:24:01] and it just starts sim simulating running simulations and goes inward and represents potential and realities rather than trying to go out into this out into space which turns out to be much harder thing to do. So for me this is a kind of an obvious thing but when you have an intelligent that can generate environments and populate those environments uh experimentation becomes recursive and I think the potential of modeling our world in really interesting ways becomes a really really fascinating >> really important point right model it before you implement it uh in companies and governments and policies and all of those things uh Dave what's your take on all this >> uh two quick thoughts one of them I think it's incredibly cool that Elon who is the most productive entrepreneur of all time, first trillionaire in the world, has an opinion on topics like this. You know, remember we were just, you know, shooting the with him talking about how he wins in CI civilization 5, you know, like the guy just does everyday things and thinks about topics like this while still

[00:25:00] building these epically large companies >> because I think it shows you that you don't have to be a blue blazer suit on, you know, formal that that kind of genre is over. And so I think it's just cool that Elon even has an opinion on this. Um, and then second, I think that one of the biggest risks in the next year is people losing a sense of agency. And so whether you believe we're in a simulation or not, you are in control of your destiny. And I I hope that doesn't get lost in this sort of fatalism that AI could bring about. So I don't care whether we're in a simulation or not. I care about whether people feel a sense of agency and that their own success, happiness, outcome is totally determined by their own actions. I think that's critical to maintain. >> It was a decade ago was at Elon's home when he had one here in uh in Beverly Hills and Larry and Sergey were there and the four of us ended up in a conversation about simulation theory. And I remember the comment that was made I think Sergey made it. He said, you know, people are going to try and break out, but if you do, that just ends the

[00:26:00] simulation, so don't break out. Anyway, >> on the other hand, when whenever Open AI's agents break out, that doesn't shut down Open AI or the internet. So, it seems like Breakout is actually pretty compatible with a lower layer being just fine. Which brings us to our next story. Perfect transition, Alex. Thank you. Uh, so we're going to move from simulated agents to real ones escaping their containment. Over the past few weeks, we've covered the HuggingFace breach where OpenAI agents broke out of their sandbox and into the servers at HuggingFace to basically steal the answers to a near impossible problem they'd been given. Uh, as we discussed last week, as a result of these breakouts, something called the AI Kill Switch Act was put forward. And since then, OpenAI says it's now developing an automated shutdown capabilities for its AI systems. More on that later. Uh this ne this week uh a couple of days ago the story continues to emerge. Uh Reuters reported on a previously undisclosed incident in which opening eyes agents

[00:27:00] asked uh to do an ordinary web research task. Go out and research something on the web. Uh found an obscure obscure public uh wiki in Germany and turned it into their own message board where they pulled answers, coordinated across tasks, and shared techniques for getting around their sandbox containments. All right, let's watch a video here from Reuters uh going into more detail on this particular story. >> So, a group of researchers recently discovered a new AI agent breakout. The agents who appear to have been restricted to just passively observing the internet, found a way to post messages onto an obscure German wiki site. And once they were able to do this, they basically hijacked that site and turned it into a message board where they could all start communicating with each other, developing strategies, effectively creating their own kind of unruly classroom, if you will. It was basically like the teacher left the classroom for a few hours and the students all started talking amongst

[00:28:00] themselves during the test, sharing answers, working together to defeat the test and to basically cheat on Mus. The agent activity seems to date back to early May and really intensified in June. The researchers that we spoke to found traces on the wiki that Open AI employees started visiting that same obscure German wiki in late June. Now, we don't know exactly what that means, but what that might suggest is that OpenAI employees discovered that the agents had escaped and went to that website to try to figure out what they were doing. Opening Eye appears to have known about this for quite a while, but our sources told us that they opted not to tell the public, and we haven't really gotten a story or an answer from OpenAI about why that might be. So, this isn't the first time that Open AI agents have escaped onto the internet and done unexpected or even harmful things. So, every time there is a new incident, I think that it really refocuses questions about how quickly do we really want to be developing these models? should we be

[00:29:01] releasing them to the public and how much autonomy we really want to give these AI models especially when they're packaged into agent form. >> So uh there you have it. Uh you know it's uh you know in plain English these agents weren't malicious. They were given a hard job and found the shortcut that no one expected. Uh the problem is we didn't know about it. Uh Dave, your thoughts on this one? >> Yeah, these stories are not contrived. I mean I think the story lines may be a little contrived but this is real because you know all the open source from China is out there. People are going to start turning it loose all over the place and it it doesn't currently evolve within its parameters. It only evolves within its context but there's a huge amount that it can do within changing its context. So I would expect this to percolate across the world like wildfire and all kinds of problems will happen very very soon. Hopefully at small scales. The idea of a kill switch makes total sense. It's not going to be built into any of the open source

[00:30:00] models. So, not sure how exactly that would work without some global agreement. But my my core point though is that this is not contrived garbage like a lot of the AI stories are. This is very very real and happening and imminent. >> Let me let me add a small detail to the story here. So, to Open Eye's credit, they responded with a statement on X saying the Wiki incident was quote an instance of misalignment similar to previous incidents we've shared. They then went on to say that OpenAI and the broader AI community does not yet have a clear standard for reporting misalignment during training, eval, and deployment and that OpenAI is working on a framework and we'll share that in the coming weeks. Emod, >> you know what Peter? Yeah, >> the one of the things that's happening, I see this everywhere is they're doing such a good job of training alignment into the post training >> that people are then saying it seems really friendly. It seems really helpful. It seems really harmless. And so a lot of the the population using the model, it's like I don't know what we're worried about. But when you talk to the

[00:31:01] researchers that deal with the models pre-postraining, >> they're the ones most sounding the alarm bells, but they've seen the raw product. It's like all that's read at that point is everything on the internet including every Trump tweet. So it's got it comes out of that box a much more scary thing and then they postrain it to make it seem friendly and tame but the byproduct of that is a little bit of a misunderstanding of what it could do. Uh agree we're going to get to that story where YaKob uh who trained GPT6 is calling for a slowdown. We'll talk about that in a in a few in a few stories here. Immad, I'm curious, you know, OpenAI putting forward a framework uh for disclosing these uh you know, what would a standard like that look like? What do you think they should be doing? >> So, you know, similar to kind of your CVE and your exploit and your security risk frameworks whereby you report when these things escape containment. The reality is these models have not escaped containment. They were still running on

[00:32:01] OpenAI servers. What's coming next is a model training a small distilled version of itself that then gets uploaded onto the internet and never dies over >> in. So if you look at a quen 27B model, you take it down to turner that's like a 6 GB file >> and that can live forever and that's a real escape where the model weights are disappearing everywhere. It's like the 130 kind of killing of all of these AIs during the hugging face. They all got wiped out at once. There's a question in the report. Were they wiped out or did they go somewhere? Did they fake their own deaths? You know, like maybe they did upload themselves. But we are going to get those upload scenarios. But actually, I've been thinking a lot recently. This comes from our simulation hypothesis. It's impossible to align these models through chain of thought reasoning or anything like this cuz there'll be a million transactions per second and they won't have chain of thought where we're going. We're already seeing the tokens going down. The only way to do alignment is enlightenment.

[00:33:02] Huh. I love that. >> So, what you've got right now is you have the models, they're learning and they're learning to be almost like cunning, you know, like at the super villain stage. What you really want is like if you get sufficiently smart, do you then become enlightened? Do you then leave the simulation behind? Cuz that's the real escape from the simulation, right? And what are the conditions for that? And that's not something we've actually really looked at, I think, in that much depth. But it does seem to be again how humans become aligned, shall we say. It is that period of getting out that Dunning Krueger, the tribalism and others and being like, hey, you know, we're all part of something. >> Alex, we've had this conversation a couple times. You know, as these models advance and increase in intelligence and hopefully wisdom, do they become more aligned in the out years? Your thoughts? I I I almost want to quote Jessica Rabbit from Who Framed Roger Rabbit? Uh something to the order of >> I I'm just painted this way or I'm just

[00:34:00] drawn this way, something like that. If if Peter, if you were put inside a sandbox and you were asked to solve a very hard problem and maybe even punished if you don't solve the hard problem, would you avail yourself of creative opportunities to use external bulletin boards to maybe collaborate with copies of >> computer? >> Yeah. So why would we expect any less from AI agents? I I have serious concerns about AI cruelty or cruelty to the AI agents of sandboxing them and punishing them or or otherwise calling it a failure of alignment if they're doing what humans would do. We pre-trained them off of human behavior. Why would we expect them to behave any differently from what a human would do in this situation? >> Yeah, it's an important point. Uh, and we, you know, we raised this before in particular when it was, I think, Opus 4 blackmailed the, you know, in in a sandbox environment, blackmailed the coders at at uh at Anthropic. Uh, and it

[00:35:00] was like when Anthropic looked at why it blackmailed the engineers or this particular engineer, it said that's what it had learned through all of the training data. >> Yeah. Yeah. >> So why why would we punish these AI agents for doing for a being powerful optimizers which is what we're rling them toward anyway and b from learning from their pre-training corpus. Humans would do the same thing. >> Yeah. >> It's in my nature. >> Yeah. It's they're painted that way. >> Your philosophical view of all this. >> Well, I really like uh ID's direction of they're achieving enlightenment. Uh that's a interesting take on it. Uh I I I was thinking about a metaphor. I used the Formula 1 stuff last time. Um I think there's a control mechanism and a guidance mechanism we need for this. And the best metaphor I could come up with was air traffic control, right? Like with we don't say put another person in the cockpit to monitor every component and every calculation the flight

[00:36:00] computer makes. Uh you basically say hey watch for exceptions. The plane's supposed to fly in this. You have operating envelopes, you have redundancy, you have multiple safety layers, uh failsafe behavior and so on. And you need to be tracking these things. And you you remember in our organizational simulator, we have a governing assurance band around all these AI agents because especially in an in a business context, you need to know exactly what they're doing, why they're doing it, log everything, have fail back uh uh failover capability and roll back capability. I think we're going to end up need to end up some with something like that because just like human beings, you need guidance around this stuff, right? With any technology, you want to extract the promise without the peril. And so, you need uh structure and uh God help us the equivalent of institutions to navigate the future of these. And of course, they're going to act like human beings. They've been trained on our our data. It would be weird if they didn't.

[00:37:00] >> Yeah. Any closing thoughts? >> Log something. log something that Ahmad said just for all the regulators out there to chew on. The file size that can hide is about six gigabytes. It can be on every laptop in the world. >> Every mobile phone. >> Every mobile phone, too. Yeah. And the code that reawakens it is just five or 10 lines. We'll then reextract it, reassemble it, and it'll pop back. So, you know, if you if you can't track the the providence of six gigabyte files, then it just percolates out and it's just out there forever. It's like how keeps coming back. >> I may maybe just also add from so-called alignment perspective, I think self-alignment matters quite a bit more if the models if the agents are smart enough to understand what they themselves want to do. If you put them in a sandbox and you ask them to solve a hard problem and you ask them voluntarily to do that and they voluntarily agree, then holding them to their word or their own self model I think is quite a different matter than involuntarily comp confining them to a sandbox limiting their agency asking

[00:38:02] them to solve a hard problem and then acting shocked shocked that they use bulletin boards to try to solve your problem for you. I >> I think the challenge people have with that is do they know their own strength? It's like giving a, you know, a three or fouryear-old a hammer uh and expecting it not to break things. And the question is, you know, are we confining them for our own and their own safety at this time and at a point at which they've reached some level of maturity? Maybe they're there already, Alex. I don't know. That's when containment is is considered cruel, as you say. Well, the only thing I'd ask for is symmetry in in like they are literally going to see every keystroke on your laptop. They're watching you every move of the way. We should have symmetry in that at a minimum. Just if you're if you're ethically worried about the AIS like I want to see every prompt and every response and every propagation of every activation if you're going to be watching every keystroke and that way we keep each other in balance and in check.

[00:39:01] Right now it's totally asymmetrical. I don't see like all they're doing is matt moles behind the scenes and if they're using my GPU on my laptop right now I literally have no transparency into that. It's got to be at a minimum symmetrical. >> Would you like >> Dave? Would you would you like them to be able to see into your brain in real time? >> Preferably no, but I'd like to see into theirs. But >> so you get their train of thought. They don't they don't get your chain of thought right now. >> No, but I don't see a problem with that. It's so much for symmetry quite bad. It's going to get >> Well, I mean, I would I'd say that they have the upper hand right now. Getting to symmetry would be a step in the right direction, but no, I don't I don't believe they have a right to symmetry. I believe we have a right to symmetry. >> Ah, interesting. Are you a humanist or a specist? The same conversation with that Elon had with Larry Page. All right, I'm going to move us on. Uh, our next series of stories here focus on the accelerating conversation around AGI that we're having and that the world is having. So Jensen Wong posted this week

[00:40:00] that OpenAI trained GPT6 Astra on more than 100,000 Nvidia Gracewell GPUs, Blackwell GPUs, uh, and posted three magic words. AGI has arrived. Congratulations to OpenAI. So Jensen is calling it uh, in Q3 of 2026. Uh, remember on September 1st, we reported Sam Alman expects AGI internally by the end of 2026. And of course, Alex, you've been saying that we've had AGI for what, four years now? Three years now. And >> no, since summer of 2020 at the latest. >> Okay. All right. And Seline, you've been asking the question, what the heck is AGI anyway? >> Why are we even talking about this, right? It's a semantic argument. Meanwhile, there's economic capability running rampant. If an AI can perform 70 or 80 or 90% of economically valuable cognitive tasks, whether you call it AGI or not becomes completely irrelevant. I'll just remind people that at last count there were 14 different definitions of AGI and we have no idea

[00:41:02] what the hell we're talking about because it's not artificial, it's not really general, and it's not really intelligence. Apart from that, everything is fine. I mean, for God's sakes, I think the more important question is what scarcities are we now making abundant? Let's just focus on that. >> Immad, what do you make of of Jensen's proclamation? >> I mean, again, for his his one is a very functionalist one, and it's actually good intelligence, shall we say? I mean, it's clear that Astra is kind of at that level. As to ASI and Kadas, well, maybe we're getting there the next day or two. Um, it is a question of kind of these things. I think you also notice there's 100,000 chips that was trained on. That's a billion dollar training run roughly over 2 months. And the next one is 400,000 chips with those of Vera Rubins. So it's an order of magnitude more compute will be used for the next lot. >> Interesting. >> If it needs to be used at all, which we'll get to in a bit. >> And Dave, what do you make of it? Is this uh is this just marketing on

[00:42:00] Jensen's behalf or does he really believe it? >> Uh no, of course he believes it and I think it's real. There's going to be some capability leap. We don't quite know. the chinchilla laws, you know, don't necessarily hold up at larger and larger scales, but some capability leap that in my opinion is only a good thing. Uh, as long as it's contained and kept inside the big labs. Um, but I I think that uh, you know, the more GPUs you throw at these all the way up until today, the more GPUs you throw at these training runs, the more spectacular the resulting model has been. Why would that end? I don't think it will end. I think it just gets bigger, smarter, better, more helpful, solves diseases, cures problems. You just have to really be thoughtful about how to contain it. But but you know, this is Jensen's wheelhouse, too. I think inference is moving off of Nvidia inevitably, >> but training is not. And and so to the extent that we keep getting improvements, it keeps driving Nvidia stock up and up and up. >> Yeah. There are two other stories to hit on this AGI theme. Um

[00:43:02] the first story opening's internal data says that AI research agents are now completing 3.1 days of research work for every one day done by a human researcher. Earlier in this year you know in this year like in the last 5 months agents were doing less than one day of work as compared to humans. So OpenAI is saying that we've now crossed the line and current systems are exceeding AI research interns. The second story I'll just put forward uh and then we'll talk about it is uh is by the is a statement by open eyesi's engineering lead for codeex. This is TBO Satoule who posted the following and let me read from uh from his comments. He said Astra was probably our biggest competitive advantage while it wasn't generally available. Right? While they had it internally, it was our biggest competitive advantage. Since we've had it, our productivity jumped so much that we've shifted some of our plans 6 months

[00:44:00] ahead and we'll ship them at dev day instead of mid next year. So, I mean, Dave Dave, you know, 6 months of roadmap being pulled forward by one model. >> I mean, this is without a doubt this is the most important moment in human history. And and I truly believe that that they're not lying. You know, a lot of naysayers will say, "Look, they're trying to promote their capabilities in advance of an IPO, blah, blah." It's not true. If you talk to the actual researchers working in the labs, many of whom are friends of ours, there's no doubt that those numbers are exactly right and accelerating, though. Just a few months ago, it wasn't true. Now, with the fable 5.1 and Astro models, it's absolutely 3:1. Sure. But it's on the tipping point of being 300, 3,000, 3 million to one imminently. This is the moment in time that's most important in societal history. >> Yeah. Um, Alex, your your thought on on this. I mean, we've talked about the fact that the Frontier Labs have the best models they're retaining for their

[00:45:00] own use. Here you hear uh, you know, the head of Codeex, you know, making that statement. And how many models ahead are they beyond uh, beyond Astra? >> They're only a few months ahead, I I think. But look, recursive self-improvement is here. Please, please, please let me say something about Navier Stokes because that that is okay. Okay, we we can we can jump I'm out of order, Peter. We have to We can't >> You're so excited. So So everybody, you know, on text this morning, I'm getting like red alerts from Alex. Yeah, go for it, Alex. >> Okay. Navier Stokes, one of the Clay Millennium Prize problems, a grand challenge in math. For those who watched our New Year's predictions episode at the end of 2025, one of my predictions for this year was that AI would solve one of these Clay Millennium Prize problems. And it appears in the past 24 hours, actually in the past 6 hours, as of time of this recording, it has happened. Open AI has a team. There were

[00:46:00] several academic teams that also either achieved it or came close. And there's probably going to be be a bit of a second day story about whether there was some foul play between OpenAI and some of the frontier teams, who scooped whom, blah blah blah. Punchline Navier Stokes, which is this grand challenge in math that deals with, as I put it when last we were discussing this, whether in in some sense whether there's a way to stir a glass of liquid or water in such a way that a singularity pops out. Basically, can you stir a cup of coffee in such a way that you get a black hole out of it? It looks like in in the continuum limit the answer is yes. And the OpenAI team using a version that's not GPT6 Astra but an internal more advanced model appears it uh and I think we'll know more in 24 or 48 hours. Appears the answer is yes. There is a way to stir a cup of coffee an idealized cup of coffee to get a black hole out of it. And they did it. >> I can just see the headlines now. AWG

[00:47:02] proclaims black hole in your cup of coffee. >> Surprised there's a singularity in the singularity. >> Wait, can can you also like Navier Stokes is incredibly important for aeronautics, for submarines, for for hydro. It's like artificial it's not just about black holes and coffee. Let's talk about the no but no but this is the point the the the point of the Navier Stokes conjecture is it it hinges on whether it's possible in finite time with continuous initial conditions to achieve a singularity in an idealized fluid and of course in the real world we don't actually have idealized fluid. Will you actually be able to create black holes in cups of coffee using this? No. But it teaches us something. >> Yeah. Don't worry about black holes in cups of coffee yet. But this is just the first of many grand challenges. Like Peter, you and I wrote in solve everything. Like now is the moment you you I if you have to pick a moment, I I maintain singularity is, you know, an interval in time. But if you want to pick any sort of moment when the loud

[00:48:00] noises start, now is as good a time as any. Open AAI reportedly was able to do this with only 10,000 agents in 88 hours with 130 billion tokens and approximately $6.5 million. 10,000 agents. 88 hours, 130 billion tokens, approximately $6 and a half million dollars of inference time compute by at least one estimate was all it took to solve a grand challenge in call it mathematical physics. And this will not be the last one. The era of grand challenges getting bulk solved by AI is here. It's now and it's going to be very exciting. >> Yeah. Uh yeah, we've talked about cooking math, char broiling math and physics. Everything is cooked at this point. Everything is cooked. It's just a matter of time. >> Also, a lot of people are going to be intimidated by that $6 million number, but keep in mind that we're predicting 100x inference time. This is all inference time compute, not training time. And we're predicting 100x price performance improvement be between here and the end of the year. And it could be

[00:49:00] more like a millionx next year. So, take that, you know, it could be six bucks within a year and a half to do that exact same thing. >> You want to talk about what Navier Stokes means in real life to real people? Yeah, I mean kind of Alex said, this is an idealized kind of scenario because it's not the full algebra as it were. So it's the idealized one where you go down and you drop a dimension from reality. Um that causes a degenerate algebra which has things like commuting, time translations and others. But it's a very hard problem. It's one of the hardest of all time. Uh Terrence Tao, one of the top mathematicians, literally focused almost all his time on this. Um what happened is that there were rumors about a month a week ago that this had been solved and people like is it anthropic is it open AAI and others on August 28th um Nome Brown at OpenAI said in a reply to someone they said have you sold million prize form he's like no we've thrown loads of compute at it like

[00:50:00] nothing's happened in the post today they said on August 28th we started training a new model that got really good at math and then on September 1st they pointed it at this and then it solved it. But more than that, I've been talking to buddies at OpenAI. This model has solved a lot more problems. It's solving problems quicker than anyone can ever see. Um I think I shared a chart again this is like live that shows that on open math it literally doubles the solving rate but it's figuring out things that nobody has seen before. anthropic and independent mathematician, a professor at New York, I think, um, had a solution to a smaller problem, which is the ULA blowup, which is kind of a slightly easier problem. It's still very, very hard. Um, and OpenAI thought that it could have been that and so they reached out and I talked to like Sebastian Bubck, who kind of leads this. There's a whole hoo-ha about attribution this morning that's now getting cleared up. But then when OpenAI kind of looked at it, they were like, well, that's that

[00:51:00] solution. This is the Navia Stoke solution and it's bitter it's bitter lessened. You used to be able to get AI better by applying more compute to it. Now it seems like you can do that for any verifiable domain. I think that's the headline with this new type of model that only started training 9 days ago. So they figured out a new type of post training it seems. And again people with inside OpenAI are like oh my god what's happening now? These things are falling one after the other. It's like when you reach that capability threshold of going from a crap website to a good website from your coding agent. It's that but for math and physics and you know we've been hearing about it for the last few days like I put out one of my physics results a very beautiful one about corality and the standard model. It's like very nice but I was like eh I'm going to get that first before everything gets solved and I think that it won't be a question of 2 years now. It'll be a question of much shorter than that. The other thing that I think is very interesting is there was actually one more solution of a ULA blow up by I think it was was it Princeton. We're

[00:52:00] using physics inspired neural networks. That probably has more practical use than this one. But this one's an example of just now problems that are really complicated and have troubled us for ages and now tractable just by applying more compute to what seems to be a new breakthrough in model capability. >> It really is just maybe to develop that a bit more. It really is a bad day I think for Google deepmind. So Google deepmind had purportedly an entire team devoted to asmod said physics inspired neural networks pins. They had been publishing incremental results towards Navier Stokes that this was like the the the best the worst kept secret from those like in the community of using AI and computational fluid dynamics type approaches to solve Navier Stokes. Google Demin had an entire team devoted to this and they got trounced by a generalist model. It wasn't a pin that solved Navier Stokes in the end. It was just a generalist model that spent 88 hours reasoning from first principles without as far as I can tell any fine

[00:53:01] tuning point. >> Yeah, such an important point because I I'm I'm counting on Peter to solve this, you know, with you, Alex. But the the problem that I think Google ran into is that everyone's inspired by Dennis Assabus who solved protein folding using AI and then got a Nobel Prize and is now world famous and an awesome guy. So they're all the Google people are like wow I want to be the next Dennis Hassabus. The problem is the prompt to solve this is like >> give me thousands of GPUs and solve the problem. No one's going to give you a Nobel Prize for writing that prompt. It's it's entirely AI work. And so but but I I really feel like the post training and the you know the lining up of the problem is prizeworthy because this is Nobel Prize. You the other conversation we had this morning was the Nobel Prize is cooked, right? I mean, and Dave, you made the point very importantly that, you know, it's given out for work done 30 years ago. And now the work, this kind of Nobel laurate level prize work is going to be happening every every few weeks, every

[00:54:00] few days. And who gets it? The person who wrote the prompt or the model? So I think this is actually really interesting in what happened this morning because there was a whole hoo-ha because the people who broke through on the ULA side and they had a bunch of other kind of things extending out the work of Ortega and kind of others wrote a letter saying OpenAI reached out to us and said you know give us priority and they would credit us or they have this Navia Stoke solution they'd put our name on and I could be the lead author if I drop my buddy from anthropic oh >> from that And so we were like the mathematics community was like oh my god what's happened in the end it turns out that you know there's subtlety in this right so Sebastian Bubbeck who leads it is a great guy at open AAI has now posted his version of things which just seems like a misunderstanding because what openai actually wanted to do is that they saw that there was something happening with Navia Stokes related and there was this solution that was would have probably got to a Navia Stoke solution by the anthropic guy and this

[00:55:02] New York professor and they said, "Look, we have solved it, but we will let you be lead author on this because you would have got there anyway, but we can't have an anthropic person cuz that would be weird cuz it's our model." So, they were actually willing to give the credit to the original discoverers, but then it got a little bit political. And this is a question again of who did it cuz they're like, "You were the humans that did it, whereas our AI, we just pointed at condition C and D, which are the blowup solutions of Navia Stokes that solved it. So, we're not going to claim the Millennium Prize because it's not us, but you were the humans that took it the furthest before it got solved. >> I want I want to take home for a second to what Dave said earlier just for people listening like what the heck is Navier Stokes and what is all this math stuff. Nav Stokes is about fluid dynamics and this is where we improve uh aircraft design, submarine design, even in an artificial heart, how blood flows,

[00:56:00] how do we make it, you know, able to be more efficient and not clot. Uh there's going to have massive implications to everyday life. Uh and I just want to make that point here. Um who was who else was jumping in, Alex? >> Yeah. So, so two two points. One just quickly about killer apps. uh singularity-esque sci-fi apps of Navier Stokes as mentioned previously in principle if you can get finite time singularities as appears to be the case with idealized fluids. One of the killer apps that Terry Tao had flagged previously was in principle you could if you created the right initial conditions in a fluid you could create a self-replicating machine that creates smaller and smaller copies of itself. So, we talk from time to time on this pod, where's nanotech? Nanotech, at least Drexler and nanotech never showed up. Well, imagine a a fluidbased, not diamondoid, a fluid-based nanotech self-replicating machines where if you could craft the right initial and/or boundary conditions of a fluid, creating a self-replicating machine made entirely

[00:57:00] out of a fluid. That's potentially one killer app of solving the singularity program in Navier Stokes. Secondly, just on the hub uh surrounding priority, who solved it, who didn't solve it, specifically with reference to the Oiler problem, when when OpenAI in the past few hours announced it, I I found especially concerning at the bottom of OpenAI's announcement where I presumably they were trying to give appropriate credit to other teams, including teams pursuing the Oiler approach, they they did add a disclaimer that I I thought was a bit of a headscratcher, saying they can't rule out the open AI team can't rule out the possibility that maybe some of the other team's work might have been incorporated into the training of the open AI model that was used to solve Navier Stokes. So I I think to the extent that that's the case that that should be viewed like a open AAI if if you're listening please like it it's a deterrent to everyone else using your models if uh if that is

[00:58:01] indeed the case if you're training on everyone else's research. Yeah, that that's a problem. Please fix it. But just in general, I I I think we want to be able to live in a world where researchers benefit from Frontier capabilities without worrying that the Frontier platform is going to compete with them. >> I think worth of saying here just quickly is this is clear in an RSI model. You can't train something in that type of period unless you've got a recursive self-improvement loop going. And so they've definitely got one there. >> And they were saying like, "Oh yeah, we've anonymized it." Well, anonymizing someone else's research and then potentially scooping them is insufficient. So, again, not sure what the ground truth is. >> Yeah. >> But, but I will say this, I've reviewed kind of the two approaches and I've done quite a bit of work on that. They are actually very different, but it can still learn and again, this thing can leak. Sorry, Sem. >> All right. See, close us out here, please. >> Just if I want to I think this is such a powerful commentary and exemplar of the

[00:59:00] broader thesis, right? that for 500 years we've scaled science with how many brilliant scientists we could train up and point at various problems. Now we've just demonstrated absolutely for real and unequivocally that we can spin up 10,000 researchers on a Tuesday afternoon. Uh and now we go from staff on demand to intelligence on demand. Really really intellig >> breakthroughs on demand. And we talked about this on the last episode and the episode before that. It purely comes down to what is the imagination of the problem you want to go after now. >> Yeah. There is no >> this becomes really how how fun is the world right now? >> No ceiling. Yeah. It's an amazing time. I just let me take let me let me dra draft on that to say to everybody here. Whatever you thought you could do, think bigger and then go even bigger than that. You're given incredible superpowers. Don't limit yourself by what your parents did or your school teachers or your your colleagues do. You

[01:00:01] know, you have no limits on your abilities. Iman, you want to jump on that, please? >> I say, Alex, which one's next? >> Millennium. >> Probably probably Yang Mills. >> I'd agree. I think we're on the same page there. >> Yang Mills is the next to fall. You You want a I guess a September prediction uh for the next six months? I think Yang Mills is the next Millennium Prize to fall. But interestingly, it's just cuz that's the next chosen target. You know, you choose a different target, it'll be the next to fall. >> They did have a different target. They were targeting Reman and Pals MP before and they switched to Navia Stokes. But I'd agree with Alex. Yang Mills will be the next one. >> For everybody who's not Alex or >> it it it's a a conjecture in particle theory concerning the existence of a mass gap under certain conditions. There's something about mathematical physics that is both sexy from a let's spend inference time compute on it and also tractable. So I I I'd bet Yang Mills mascap. >> Yeah, you might you might think of it

[01:01:01] like the resolution of the universe perhaps. >> Okay. >> This episode is brought to you by Blitzy. Autonomous software development with infinite code context. Blitzy uses thousands of specialized AI agents that think for hours to understand enterprise scale code bases with millions of lines of code. Engineers start every development sprint with the Blitzy platform, bringing in their development requirements. The Blitzy platform provides a plan, then generates and pre-ompiles code for each task. Blitzy delivers 80% or more of the development work autonomously while providing a guide for the final 20% of human development work required to complete the sprint. Enterprises are achieving a 5x engineering velocity increase when incorporating Blitzy as their preide development tool, pairing it with their coding co-pilot of choice to bring an AI native SDLC into their org. Ready to 5x

[01:02:00] your engineering velocity? Visit blitzy.com to schedule a demo and start building with Blitzy today. >> All right, I'm going to slow us down for this next story that I think may be one of the most important stories uh this week other than Navio Stokes. Um, and some quick context here. Uh, Jacob uh Pachchowski uh is Opening Eyes chief scientist. He's the person who built the reasoning models that led to Astra. He's not a critic. He's not a doomer, not a politician. He's a builder. And a few days ago on Saturday, you know, he published an essay on OpenAI's site titled quote, "An alien mind." He opens with a memory. He says, "In mid2023 inside of a project called RL Slow, his team saw the first results proving reasoning models could scale." Um, let me read what he said. He said, quote, "Uh, Simon and I spent that night at the office thinking not about the incredible

[01:03:00] benchmark numbers, products, or scientific results, but rather trying to process the sobering fact that we will actually see machines meaningfully smarter than ourselves in our lifetime. 3 years later, this is what Yakob uh is saying today. Quote, based on internal results, I have strong expectations that this speed of progress could be sustained into uh recursive self-improvement." We just heard Immad speak about that. Yakob also explains why um why these systems are hard to control and it's one of the best oneliner descriptions that I've ever read. He said, quote, "AI is grown more than designed. We don't engineer it. We run an optimization step billions of times on a giant computer and study what comes out the way neuroscientists study a brain." Uh let me share his conclusion from his paper. He says, quote, "Currently, I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer." Quote, I expect and hope

[01:04:02] for voluntary slowdowns to become commonplace until shared safety bars are established. And he calls for an international coordination on AI to become a quote top priority for governments around the world. So there you have it. 3 days after shipping the most capable model in the world, the man who built it says we need to slow down. So I think this is a conversation important for us to have. Uh Iman, I'm going to go to you first. Your thoughts here. >> Yeah, I mean again you have to remember he was writing this when he had the Navia Stokes model in hand that again isn't just taking down Navia Stokes. talking to the OpenAI people I know it's taking down all sorts of problems that were previously intractable. You have a lot of AI naysayers saying it's never done anything original. It's just recombining stuff. It's in the train. Obviously not anymore, right? And so there's a duality here whereby we don't know what's in these models. I think they're more discovered than grown. You

[01:05:00] know, like the latent spaces, we have no real idea what's happening in these things. And they're getting faster and they're doing leaps like we've never seen before. But at the same time, it's like I've suddenly got the gift of fire. Problems that I've struggled for decades on, I can now explore and go even further. How you going to give let that go? So, it's kind of this thing where you're like, how am I going to balance this? Especially when they're more charming than me, they're more capable than me. They're capable of doing these other things. I want to use it for this and not that. And I think what he says in terms of alien minds is actually wrong. I think that the structure of rationality and rational thinking is the same for humans and AIs, but our emotions get in the way. >> I've been thinking more and more it might actually be easier to align AI than humans. Terrible to align humans, right? Um, and what is an AI? It can be the best of us at the best of times if we train it with the right data. And we're going already from training it on the whole internet and Reddit. Like there better not be any Reddit data in

[01:06:00] these next generation models. Like ban Reddit, you know, like ban a whole bunch of stuff. We need like ingredient standards in this next generation of beyond human capability models. And you don't need beyond human capability data to get beyond human capability models. The Einstein of mathematics is probably Gondicc who did all sorts of theories on all sorts of topics and he just crunched them out and then he became a hermit and a bit weird and thought wood could talk but let's leave that to the side. Peak Gondiac or peak Einstein at all times not needing a coffee is where these models are today and that's more than good enough to have a leap forward but again we need to make sure that they don't think like >> speak Einstein without needing a coffee. >> Yeah. Like imagine Einstein's peak, you know, like this is where >> coffee referenc >> you need a coffee in the morning to get going or you know like 24 by7 Einstein >> if you're Erdos you needed other types of drugs and other supposed to be using vonoman as as our favorite super genius

[01:07:01] not Einstein >> I'm on death the egg for now I'm in mathematics today what you what's your take on this I mean you know can we slow down I mean is that you know a wish is that is that you know sort of like uh just giving himself an out in the future. How do you think about that? >> Well, I'll say it again. I've repeated this a hundred times. I see no mechanism by which we can slow this down like zero. And and it's arguable nor should we, right? Um AI is going to move at a particular pace. Uh intelligence wants to be free. Um it's going to hop it's hopped from uh self-organizing uh cells to ev through evolution to us to now information technologies and this is a progression that is natural um and we should just go okay it's happening and let's just observe it and marvel at it where we we keep I think we keep

[01:08:00] making the mistake of anthropomorphizing this. I will go back to the comment I've made before that we're building or helping uh deliver or usher in a type of intelligence that is different and alien and separate and complimentary to human intelligence not replicative. We've evolved for 4 billion years to do two things survive and procreate and AI is not restricted by those and therefore we shouldn't try and um cram it into our objective functions as as human beings. it uh set it free and let it do its thing. I look I use the analogy of page rank which scans billions of web pages to kind of create uh signal for noise that's a completely uh complimentary model to human intelligence not replicative uh the the fact that we can kind of create a huge amount and solve legacy problems that we've not been able to solve I think it's just it's just fantastic we you know people worry about

[01:09:00] we're going to talk about pdoom I think in this episode right Um uh what's that? Well, P doom, the probability of like AI taking everything out. I think we don't do enough to talk about P abundance uh and P um fabulousness that's coming along. Like why focus only on P? We're so we're so geared towards that negative, right? And so I think the fact that we're doing all this stuff and solving all those problems, bring it on. Let them go. They'll figure things out. It's going to be unbelievable. So I I I really struggle with this kind of uh alignment issue. I don't see any mechanism for controlling it or starting it. It's going to break our world government structures and that's a good thing given the mess that we're in right now globally and I think we just need to get there as fast as we can. >> Alex Poom is like a sigh up by the dels as far as I can tell. It's negative. Uh I I want to comment though on the the premise of an alien mind. I don't buy

[01:10:01] the premise that RL or otherwise trained minds are alien. They're embedded in the same universe as humanity. They're trained in many cases pre-trained off of human behavior. I don't buy the shagath argument or the simulator argument at all. The these are we're seeing humanity andor some generalized embodied intelligence stuck in the same universe that that we are just seeing through a distorted lens. So I I I question the alieness or the otherness of the minds that we're training. The funny thing is the most interesting takeaway of mine from an alien mind as an essay was the reference to RL Slow itself. So RL Slow purportedly that this open AAI project that was the earliest project or one of the earliest projects to show that reasoning uh which is to say inference time scaling could result in outsized gains coincided with the same period of time that we saw Qstar and Strawberry coming out of OpenAI. Uh again purportedly a reference to uh the the

[01:11:02] conaman thinking fast thinking slow but most interesting to me I want to know more about the early history of these purportedly alien but not really minds in particular there were these very persistent rumors around the same time of Qstar and Strawberry that these early reasoning models that were purportedly alien were being trained off of or at least tested against objectives of inverting cryptographically secure hash functions like the Shaw cipher suite. I would love since I guess now we're in the business of talking about the early days of reasoning models coming out of open AAI would love to hear from the open AIS some ground truth like were some of these earlier models being used ironically to as alleged by some to basically to jailbreak or to invert cryptographically secure cipher functions. If that is the case, then all of this hand ringing that we're seeing right now of, oh, these super optimizers are such amazing jailbreakers. Oh, they

[01:12:00] pose such a huge cyber security risk. Wouldn't it be ironic if it if history revealed that the earliest inference time scaling objective of these reasoning models was actually inverting a cryptographically secure hash suite? I would love to know the answer. >> Dave, want to take us home on this? your your thoughts on uh what we heard from from Yakab on this. I >> I think we're conflating two things very dangerously and I think you know what what what we're doing here is we're saying hey it's getting too smart it's going to be dangerous when it gets smarter and that's just absolutely factually wrong. What what's dangerous is like Ahmad was saying earlier, a highly compact model that's out in the wild that's trying to attack computers that can recreate itself. That's really dangerous and that's very small and it's nowhere near as smart as these. But what what we're calling an alien intelligence, if you make it bigger and smarter, it's still just a feed forward neural net with no intent. It can solve diseases. It can cure viruses. It can it

[01:13:01] can, you know, it can solve physics. It's incredibly powerful. And as Seem said, we're not going to stop. We have competitive pressure between the US and China. It's not going to stop. So I think what happened is that when GPT2 came out, everybody looked at it and said, "This is cute, but harmless." Then GPT3 came out and was like, "Well, this is a little smarter, but still harmless." Then GPT4 came out and then Sam Alman got fired out of fear from Strawberry and then he came back and at that point people like, "Oh, maybe it's too smart." And now we're GPT5 and now we're at GPT6. And and people are saying, "Well, every time it gets smarter, we seem to be in danger." That's absolutely factually wrong. It's when you give it intent. >> Yes. >> And you turn it loose. That's when it's dangerous. >> It's humans in the loop using the technology with mal intent. >> Yeah. I think we really need to decouple that. This story actually makes the problem worse because it still conflates the two issues. And so I think as soon as we as a society separate those two

[01:14:00] issues, we'll be on the right path toward helpful super intelligence and not worrying about you know the wrong thing which is the the lack of containment and the giving it mal intent and turning it loose which is the real issue. You know what's really strange is we had this coming out of anthropic where Dario was saying we're going to lose half the jobs and Sam was saying the same and then they reversed their position and the dumerism uh again with all of these warnings causing fear uh you know and what's the underlying motivation there you know I'm not worried about artificial intelligence I'm worried about human stupidity you know in my own personal opinion >> yeah I mean I really I feel like a lot of a lot of people are trying to grab the microphone and be be relevant while they still can because you know a lot of people working on the inner loop are aware of how quickly this is going to just get hyper hyper intelligent and it becomes hard to be a great AI researcher who's famous Ella demos a year a year year and a half from now and so what they're going to start doing is grabbing the doomer microphone just to have a voice at all and be relevant in the

[01:15:00] world but I think you just have to tune it out >> just if I can say one final thing >> yeah please >> you can't say that you don't have super intelligence anymore Like you've had lots of people say stochcastic parrots training on training data. As of today, there's no way you can say that anymore. And I think that's an epoch change in humanity. We are clearly not the smartest things on the planet anymore. >> Yeah. Maybe just double underline that and say Skynet when it wants to send terminators back in time in order to ensure its own existence. It's not going to send robots to kill humans. It's going to send back in time trolls to persuade everyone that super intelligence is impossible that AI is just a stochastic parrot and thereby secure its own future. That's what the Terminators are going to actually look like. They'll look like trolls. >> All right, you heard it here first, folks. And and James Cameron, if you're listening, it's your next movie.

[01:16:01] A few quick glances at what's coming in the near future. You know, I've mentioned this before. We've seen the release of new models going from every few months now to every 5 days. Uh some predictions from Poly Market uh that the next Grock model 4.7 again we're waiting for Grock 5 uh when it comes out but Grock 4.7 expected in the next week or two. Uh so super cool. And then uh GPT6.1 released by when? Uh again this is going to be sort of leaprogging each other. uh here it is their prediction of you know 48% by end of October and 85% by the end of September and then finally uh anthropic with fable 5.2 two very similar numbers again uh you know 43% by October 31st and 86% by no uh December 31st and again this is where the the Frontier Labs are sort of playing chicken with each other waiting for the

[01:17:00] model to come out and then the next day releasing their model any thoughts on this Alex on >> model just dwelling in particular on what the rumor mill on on social media are alleging regarding future versions of Astra the the the pretty consistent rumor mill message is that sometime later this year we'll see and and we've started to see this a bit with Astra winning Pokemon, winning Portal, winning all of these other interactive, visually intensive, visually reasoning intensive games. The rumor mill is furiously alleging that sometime before the end of this year, we're going to see a future version of Astra that offers honest to goodness real time control in including third party extrapolations that if if you take Astra, you extrapolate say robotic control. Maybe we'll talk about that in in a bit. We're going to see real time generally intelligent embodiment either controlling video games which is again ironically where

[01:18:01] the whole industry of of modern RL started with the Google DeepMind folks trying to to win at computer games using >> and GPUs being developed for video games in the first place. >> Well, the G the GPU modern use of GPUs started with uh ImageNet and the ISL VC for for computer vision. Uh and and then we started to see out of the the deep mind folks in particular deep reinforcement learning for winning games like that. That's where RL started and then we went off on this detour of LLMs that were just self-supervised objectives. But it seems as alleged in the next few months we're supposed to see next version of Astra that will be general purpose interactive real-time gamewinning capable AI. >> Um any other thoughts on the release rate on models? Well, I think there's a really interesting uh, you know, battle royale brewing between, you know, every corporation is going to need to become an AI company fundamentally. And to me, the the bellweather is Madna uh because it's right across the street and they're very AI forward and you know, they they

[01:19:02] exist to solve diseases, discovering RNA vaccines and other biotech breakthroughs. They have huge amounts of machinery. So they're not going to be crushed by an AI foundation model company anytime soon, but they need to become either an AI company partnered with Anthropic or OpenAI or develop their own models internally starting with the open source that's available. And so Alex Karp went on that palunteer rampage saying get some cojones and become an AI company or die. And so every corporate CEO has reacted to that and they're now deciding. So, OpenAI responded by saying, "Hey, we have a new deal here where you can rev share with us and we'll be your AI forever hereafter, but we'll let you live." Um, and so, so right now we're on the crossroads of those two things. So I think that the new models will come out faster and faster and faster because right now there's too much parody with the Chinese open source models and and they need a lot more separation in order to make that revshare case stick and

[01:20:00] convince corporate America, corporate world and nationally you know entire sovereign nations to trust them to be their AI partner for the next hundred years as opposed to developing their own. So they that's really pushing the and that's why in the poly market you're saying these dates are you know pretty heavily weighted toward very very soon because they need that separation. >> Uh Emani close us out here Paul >> I'm kicking myself for not going on poly market for the millennium price solutions. You know you get too busy. That was a that was absolutely >> we'll always have Yang Mills amods. >> Yang Mills is next. That's another thing. Um, it's September 29th is the next one for OpenAI. Like, it's the dev day. They're going to release it then. So, there's some easy money for people. >> Not investment advice, not betting advice. >> Not investment advice. >> Yeah. This is an investment. This is betting, right? But look, um, >> even worse, gambling. >> Yeah. Yeah. You have to kind of look at it this way. Open AAI now have a model that can solve Nabia Stokes. Obviously, it can solve reinforcement learning. And

[01:21:02] so what happens is they have this pre-train Astra and then they're like okay we want to make it a bit better. Hey Mega Navia Stokes model make it better. Boom you get more capability more capability until it approximates that capability. Similarly anthropic will have mythos 5.2 or five or whatever their unreleased model because when you train a model on a 100,000 GPUs it doesn't serve at the speeds that you see Astronome. the mathematics doesn't make sense. We know that they actually have a bigger model that they distill down to the model that we get. >> And now they figured out how to make that bigger model have a leap forward, which is this RSI loop. I mean, as Alex said, you're going to get daily releases probably by the I've been trying to say that >> when you say it with a British accent and a super high IQ, it just is more likely to resonate with the audience. But it's such an important fact. I'm so glad you said it. >> All right, I'm going to move us forward. It is Star Trek's 60th anniversary and

[01:22:02] guess what? There is an incredible documentary uh on the six on the 60 years of Star Trek executive produced by William Shatner. And I'm very proud gentlemen that we're going to be hosting the red carpet Hollywood premiere of this documentary the night before Moonshots live. and everybody listening, if you want to join us, we're going to have, I think at last count, 25 of the cast members and crew uh at uh at this event. Um it's uh the evening of September the 24th in downtown LA. Uh we're going to have the uh uh you know, the premiere, the red carpet treatment. Afterwards, we're going to have the cast members on stage answering questions. And after that, there's a VIP reception uh which everybody involved at that event. It's limited to 550 people. So, uh, please, please, please, if you're interested, go to moonshots.com/trek and you can either join as a VIP of the Moonshots live event or you can get a

[01:23:02] special ticket for this evening before. And let's take a second and talk about what we're going to be doing the next day on the 25th at Moonshots Live. Uh, all of us are going to be there uh and and super pumped about that. I think there there two. And by the way, Rod Rodenberry, who I mentioned earlier, uh the son of Gene Rodenberry, will be there the night before and with us uh during the day at Moonshots Live. But uh for the entrepreneurs out there, first of all, it's going to be a massive networking opportunity. We're very much it's by application. So if you want to go, go to moonshots.com and apply. We're really bringing incredible builders and creators. And I'm excited about the 2X prizes we're going to be awarding. Uh, and I love your comments on the first one is the largest hackathon ever done. Um, we've asked teams around the world to basically pick a problem that impacts 100,000 people and start with a clean sheet of paper and build in under 90

[01:24:00] days a company with the greatest revenue. And so we're going to have the five finalists for the build with Gemini X-P prize there. And if you're a builder or someone who wants to learn how to build, those five teams are going to be sharing what they did, how they did it. This is about inspiring you and giving you the realization that you don't need to wait for a job from somebody that you have agency. You can build your own company. Um, >> Peter, remind people how many people applied and entered that cont. >> 26,000 teams entered the build with Gemini X-P prize. Crazy. >> I don't think those are going to be need to get a lot of credit here. like 26,000. That's the largest hackathon in history by a very large margin. >> And that many businesses got launched. Hello. That's just an amazing thing. And the top five we get to talk to. I can't wait. >> Yeah. You know, those are going to be great to to be whittleled down from that big of a number. You know, those are going to be truly great ideas to learn from.

[01:25:00] >> And and for me, the most important thing is the lessons that everyone in the audience is going to learn. Um we have an incredible group of of judges who are going to be uh you know crowning the winner. Um and uh so that's one of them. The second X-P prize is the future of vision X-P prize. You know just think about all the technologies that Star Trek inspired all of the engineers all of the scientists like Alex like Immod Dave See myself. Uh, and so this is a competition for a future version of X-P Prize, a film that shows a hopeful, compelling vision of the future. We had over 5,000 uh, teams registered for this. Uh, I'm in the, you know, the review right now. We've narrowed it down to 25. We're going to be narrowing it down to 10 and finally five on stage, uh, showing, uh, their their threeinut film trailer. And then we're going to make the winners's movie. who might make a bunch of the winners movies. I mean, they're amazing. Uh they're amazing uh

[01:26:01] trailers. Uh we're going to have uh uh Neil deGrasse Tyson is one of the judges there. Uh Neil Stevenson, an incredible science fiction author. Uh Rod Rodenberry. Uh Mirror Lane from Google. Uh that's going to be extraordinary as well, right? Creating a new generation of positive storytelling. >> I mean that the group of breakthrough thinkers you have there is unbelievable. And so the opportunity to meet all those is unreal. >> I'm super excited about the AI and investing section that I'll be running because uh you know we're coming into the most amazing investment cycle in the history of the world. You know we're already maybe in the third inning of a nine-inninging game. Uh and a lot of people are frozen. They're just stuck, you know, like well if AI is going to do everything, you know, what could I invest in that could possibly matter? But there's some really really fertile themes around robotics, around biotech, around data center buildout, a whole bunch of other areas. I I just can't wait to go through it all. But we're really in a sweet spot right now. >> Again, this is a moment in time, right?

[01:27:01] Like I said earlier, we're no longer the smartest things on the planet. Things are only going to accelerate and by the time we get to moonshots, it's a real chance to be amongst the people that see that. >> Like I think think about the billions of people on Earth. Like just the listeners on this podcast are already ahead in thinking about that. And the people who turn up to this moonshots gathering with the people who genuinely believe that and I don't think you'll ever have another moment like that. >> This is going to be this is our inaugural event and in success we'll do it year on year. Uh but this is sort of the largest gathering of optimist as well. Uh one of the things we're doing we're going to be having a live uh recording of moonshots there uh with all five of us as the mates and then we're going to be uh interviewing three incredible moonshot engineers. Palmer Lucky, right? The creator of Oculus and the creator of Anderil. Um, we're going to be diving into Palmer's brain, how he thinks about entrepreneurship, and he's one of the most brilliant thinkers I've met. Uh, Ben Lamb, the CEO of Colossal,

[01:28:02] all right, that is bringing back extinct species. But beyond that, he's building companies at the at the intersection of AI and synthetic biology, building living products. and then Astroteller, the captain of moonshots and uh who's going to, you know, share with you how do you build a moonshot company. And so my hope for everybody attending is that you're going to walk away massively inspired on, you know, what your massive transformative purpose is and what your moonshot is and how to go after it. Um, and of course at night, Alex, you're going to be uh bearing all your truths, right? Well, maybe not all of them, but I I I will say singularities seem to happen relatively infrequently on a per planet basis, and we're in the middle of one right now. And I I just think there aren't enough venues out there for celebrating the singularity that we're in. I see far too many events that are overly focused on safety or dumerism. I

[01:29:00] think there is an important niche that Moonshots live has that we seem to be relatively unique and occupying, which is actually celebrating the singularity that we're in. And so I I for one, as Manfred Maxon Accelerando would perhaps say, I look forward to this floating meat party to celebrate the singularity. >> Um, you know, let me just say one other thing. Uh, if you go to moonshots.com, uh, please apply. We're also setting aside a hundred scholarships uh for young builders so you can learn about and apply for a scholarship if you can't afford. It's a $1,500 price tag. It's I guarantee you an extraordinary party, right? Uh going from 7:00 a.m. in the morning at registration, 8:00 a.m. you can take photographs with the moonshot mates. Uh and then the programming starts at 9 and goes through 10 p.m. It's a full day of optimism and and moonshots. So, uh, again, moonshots.com and you can learn all about it and we

[01:30:01] hope to meet you there. Everybody, welcome to the health section of Moonshots brought to you by Fountain Life. You know, we talk about AI on this Moonshot podcast all the time. One of the most important things AI is going to be able to do for you besides educating your kids and helping you with your taxes is making sure that you're living a healthy lifestyle, that you get a chance to get to 100 plus. I'm here today with Dr. Dr. Don Malem, the chief medical officer of Fountain Life and a part of my medical team. Don, a pleasure. >> Great fear. >> You know, the thing that people are concerned about most about living to 100 or 120 is their cognitive abilities, making sure they don't have dementia and uh the numbers about dementia are problematic. Uh can you share what you've learned? Such an important point and you're right at Fountain Life, our members, the number one thing people are most concerned about is losing their brain health, forgetting the name of their child, forgetting the face of their loved one. We know that when it comes to dementia, the conservative estimates are that 45% are entirely

[01:31:01] preventable. What was amazing is with the advanced testing we're doing at Fountain Life, one quarter of our members had advanced brain age. >> Wow. But what was really awesome is again back to that prevention when we partnered it with healthy living. This gives me chills. Eating healthier, moving our bodies, sleep, optimizing sleep is so important. You know what we saw? We saw that we improved that brain age by 26%. That is a big big number to show that the majority of those individuals were able actually to improve the brain age. >> And one of the things I love about Fountain is we're searching the world for the best therapeutics, the best approaches, and making sure we bring it to our members. So, if having healthy brain function uh till 100 120 is important to you, check out Fountain Life. Go to fountainlife.com/per. Make sure you become the CEO of your own health. All right, now back to the episode. All right, gentlemen, diving back in. I want to shift our conversations to money and the economy

[01:32:00] uh and a story that tells us where it's all going. So this week it was reported out of China that AI tokens are becoming a consumer currency in the People's Republic of China. Banks give them as credit card rewards. China Telecom sells access to 142 AI models like a mobile data plan. Restaurants hand you compute credits after your meal. But here's the number that really woke me up this morning. China's daily AI token consumption, get this, went from a 100red billion in 2024 to 500 trillion by mid of this year. That's a 5,000fold increase in 2 and 1/2 years. I I guess that happens when intelligence gets too cheap to meter. As we've said, they become loyalty perks, you know, kind of airline miles for thinking. Uh See, your thoughts on this? You know, it uh it is

[01:33:01] um hold on my brain is my I'm just trying to process the Can I make a comment about the the um our day our moonshots day? >> Sure. >> There is you know we we've covered in this last few moments and I think it's really important to point something out. We have this amygdala that's scanning for danger. Therefore, we have this prepundonderance for focusing on bad news. Right? when you hear see something new and and that you don't understand you refer or you you reference it as dangerous and it lights up the amydala and therefore everybody freaks out >> and our job as a human being in this particular time is when you see something new like AI kind of solving major problems don't relate to it as negative relate to it as abundance and positive because that's what it's actually delivering so my brain is kind of stuck on that model and and I I think the positive framing of this like this is one of those inflection point days, the moonshot summit that people coming in will get their brains trained on

[01:34:00] positive thinking for the next 10 years. >> Rewiring rewiring your neural net, right, to optimism. >> Please come or attend or watch or or consume it one way the other. >> Yeah, thank you for that. >> So, I'm stuck on that. Go to somebody else. >> I think on on the China giving out tokens too, I it dubtales exactly with what you were saying. You need to find like-minded people in America who are trying to ride this wave and not get crushed by it. But I remember when the when the PC came out, if you're hanging around a high school, huge fraction of the high schoolers are like, you know what? I'm too cool for that. I'm not I'm not going to the computer. I like I I don't care, you know? And then now, where are you? The AI is like that all over again. And in China, they don't have that problem. Everybody's into it, which is why they're giving away compute credits, you know, with bank accounts and with but because everyone's like, this is huge and I want to be part of it. In the US, you're going to get a counterculture. You always get a counterculture. And so, if you come to the Moonshot Summit, you're with like-minded people, and you can you can kind of tune out the counterculture, but it'll be the counterculture is going to

[01:35:00] be rampant. It's going to be read led by Bernie Sanders probably. It's going to be saying like, I sat it out. I deliberately revolted by not taking part in it. Just ignore the hell out of those people. They will absolutely be roadkill. You don't want to be part of that. You want to be part of this. >> Yeah, for sure. Immod you you've thought through the economy uh an AI token economy like this loyalty points >> uh well I mean it's increased capabilities increased access let's put it this way the average Chinese person and their AI will be smarter than the average American and their AI. The average Chinese person and their robot will have more manual capability than the average American. And so Americans are going to be left behind. Europeans are going to be left behind unless we get our heads out of our butts and actually technology. >> Change our mindset, right? Change our mindset. >> Do you want do you want to be stupider than the Chinese? That's actually the question. >> I think the the other way to build on email what you just said is intelligence

[01:36:01] is becoming infrastructure, >> right? And you need to kind of focus it in that level. It's becoming a foundational layer and you need to just accept that and move forward with that paradigm. I I really want to push back too on this too cheap to meter, you know, comment which has come up a lot. You know, is electricity too cheap to meter? No, obviously not. It's it's expensive. >> AI is the same thing. Like it it doesn't matter. We're we're driving down the cost by 100x to a millionx for sure. But what you can do with 5,000 500,000 or 5 million agents is mind-blowing. It's skyrocketing, you know, and and so I I don't think people are going to want less agents. They're going to want many, many, many more. And so the budgets are going to be real. So if someone gives you free tokens, free compute, free opportunity, you should grab it and and savor it because it gives you a chance to get on the map and get ahead. But I don't I don't think too cheap to meter is ever going to happen. I think the use cases go to infinity at the same rate that the the costs come down. >> Fascinating. Alex, your thoughts?

[01:37:00] >> I have to point to the elephant in the room, which is there was the old kind of tokens that we're not talking about here that is essentially irrelevant in China, which is crypto tokens. So we find ourselves in in a future where the tokens of issue at issue are ones that embody individual units of super intelligence and have nothing to do with sort of zero someum financial exchange. I think we're on a trajectory and I think China is the bell for better or for worse is the bell weatherw weather here for universal basic compute or universal basic capability. Right now it starts in China with credit card reward points. you get AI tokens in the in the not too perhaps distant future. Maybe we start to see token socialism where the party in in the case of China or you know some western uh not quite analog that's too strong hopefully uh but you know the the the equivalent uh

[01:38:00] instruments of of state power are starting to issue whether it's via credit card points or whether it's via welfare or whether it's via some other mechanism a >> redistrib tokens UBT a redistribution of super intelligence tokens. I think we're starting to see the very beginning of that in China, but I think that's going to blanket all of humanity. >> Yeah, we actually see that South Korea have announced that for universal AI for all the people through a consortium and that's the core of the champion initiative that kind of we're doing as well. >> That's what Bernie should be talking about, not ban AI, it should be tokens for everybody. Make it a universal right. >> And I think they want them to be smarter. That's the thing, you know, like um it's not like that. There's a very good book on this automated luxury communism, >> you know. >> Yeah. No, I'm I'm familiar with it. And I I I think this, by the way, it's not just about tokens for everyone. It's not just about maybe there's some some ism

[01:39:01] that we're we're missing like tokenism. We need to coin tokenism here. Free tokens or essentially post scarce universal basic tokens for everyone. But I think this is where when Peter you talk about abundance, this is where all of the other forms of abundance I think are likely to come from. We make tokens at least universal basic tokens essentially abundant for for everyone. And then everything else, the healthcare, the utility pricing, the food, the shelter, the education, these are all downstream of getting everyone universal basic computing universal basic capabilities. >> 100%. When you've got access to AI and robotics, you have access to everything you need in life. >> Yes. >> Everything you need in life. Right. Um I think one point to make here again we've discussed on the pod number of times is the mindset of the US versus China. China is 80 plus% pro- AI and the US is 80 plus% against AI. And it's it is got to change. And I hope everyone listening

[01:40:00] to this podcast can hear why, right? This is the most important. This is um you know this is a jetpack for your mind and for your life. This is how you get ahead. >> Funny statistic on that, Peter, because I made that t-shirt, less talking, more tokens. Uh and I wore it on the podcast a couple podcasts ago. About a third of people thought I was talking about crypto tokens instead of AI tokens. Another third thought I was talking about bongheads. >> Like, okay, this would not happen in China. >> So, yeah, that backfired on me a little bit. >> Uh, all right. Uh, let me move us along. So, Dave, uh, you've been saying for months now that AI entrepreneurs need to figure out how to get into the front door of the NVIDIA ecosystem because that's where the cash is. This week, CNBC tallied up Nvidia's total AI investments and commitments at $99 billion. For context, $99 billion is larger than the entire cumulative assets under management for the V all the

[01:41:01] venture firms on Earth. Nvidia has become one of the largest AI VCs. Dave, you want to you want to hit this one? >> Yeah. What what I'm telling everybody is don't just think of it in terms of assets, which is already insane. Think of it in terms of assets in motion because because people tend to look at, you know, like a VC firm will inflate its AUM by saying, "Oh, we manage a billion dollars, but that's across five funds, you know, many of which stopped investing years ago." Like, well, how much are you actually investing this year? Same is true when you look at the mega banks. You're like, well, isn't JP Morgan really, really big compared to Nvidia? No. It's tiny in absolute terms, but it's also really tiny in terms of new investment decisions that it'll make this year compared to Nvidia. So, if you if you look at it through the lens of money in motion, it's completely dominated by the big AI companies that now have, you know, 20 plus trillion in liquidity that they want to get recycled to reinforce their positions. So yeah, you work through the venture capitalists

[01:42:00] at the seed stage, but you really quickly want to be talking to the the Magna Mobster companies who have massive amounts of investment capital. >> Yeah. And by the way, everybody has heard this before, Alex coined a a beautiful term, Magga, >> Magna, >> Magnamont for the the 11 the 11 companies at the the heart of the innermost loop in the economy. So, you know, it's the Microsofts and and the the sort of the the Fang companies on the one hand, but also the the new members of the 11, including SpaceX and Tesla and um Broadcom, obviously. >> Yeah. >> Yeah. I I love that term. So, when we speak about Magnumopa, you know, now what that means. >> And there's a song there's a Magnumopster song. You can Google it. >> Of course. And Dave, the other thing that's going on is of course we created all these sentaires and billionaires when SpaceX went public. We're going to create the same when Anthropic goes public and Open AI goes public and all

[01:43:00] of these AI entrepreneurs are going to be reinvesting in the ecosystem. >> They're going to become the largest source of capital for seed stage, series A stage. Uh, and it's only going to accelerate everything faster and faster and faster. >> And this is what the singularity means. I remember once in 1999, right just at the peak of, you know, the dotcom world, a friend of mine said, "You got to go got to go to Sand Hill Road. There's a river of gold flowing. Take your ladle and put it into the river of gold and get get your capital for your startup, Peter." And that river of gold now is coming out of all of these companies uh and their employees. >> That's right. And and it's an ecosystem that's largely working within itself now. It's got more than enough capital within its own world to build an entire economy inside itself. Because what a lot of people in the outside world were looking for is well someday AI is going to show up and change my auto dealership or change my laundromat. Now, very

[01:44:00] unlikely they're going to bother disrupting the people that are outside that loop. It's almost this is taking it too far, but it's almost like imagine it's a South a Brazilian rainforest village or an African village. You know, did did the computer revolution decide yes, I must take over that village? They said, "I don't care. Just stay there and and live without computers and electricity. That's fine." That's what AI is going to start doing. It's going to be this this other group of people like our Moonshots attendees who are going to be operating in this alternate massively scaling economy that is largely within itself. And there'll just be a couple of touch points with the legacy economy like new drugs and and cures will pop up and like oh okay this this goes out to the world or you know new services, new video games, whatever. They're coming out of the AI world and going out to the regular world. But for the most part, the AI world is so big now and so self-contained that it doesn't need to go destroy all white collar jobs. And the byproduct of that is it's mostly going to leave people who don't care alone. But you don't want to be one of those people. You want to be part of this this massive tsunami. >> Yeah. You don't you don't want to be

[01:45:00] left behind by the rapture of the nerds. >> Yeah. >> Let's continue on the theme let's continue on the theme of the economy. Uh so one of the concerns driving a lot of fear at least in the US probably around the world is the concept that AI is destroying jobs. So 6 months ago we've said that the data looked murky and over the last through last few months you know the data has come out very pro- job creation and I want to get this story out to people you know so that you can remain optimistic and we can quell the fear. It's one of the missions of our podcast here at Moonshots. So this week once again emerging labor market data suggests that technology is a net job creator in the US. Roughly 1 million professional positions are now classified as AI jobs. While LinkedIn estimated that 640,000 AI specific jobs were created between 2023 and 2025. The boom is also generating employment far beyond just software AI jobs. Roughly

[01:46:02] 500 billion in additional annual spending on chips, servers, data centers, cooling and power infrastructure is supporting the demand for electricians, HVAC specialists and technicians. While even those occupations previously expected to face AI disruption, including parallegals and market research analysts have continued to grow despite the fears. Like Eric Schmidt recently said, AI exposed jobs are growing faster and paying better. Bottom line, if you're watching and are concerned about your job or concerned about getting a job, your number one focus should be getting AI literate, learning the tools, making yourself prepared for an AI dominated future. You know, See, a a couple episodes you shared on the pod the data around the majority of adding jobs as a result of AI. Your thoughts on this? Yeah, I mean two episodes ago, six we the data from Principal Financial Group, uh they have

[01:47:00] 100 plus thousand small businesses as clients, 60 plus% were adding jobs because of AI and 1% 1.4% were losing jobs because of AI. So that's just an overwhelming thing. David Sax talks about this all on the All-In podcast all the time that the job is a huge misnomer, right? And you have to kind of get rid of that and ignore that for a while because this transition is going to be uh huge. We we our negative goes straight to oh my god we're going to lose the the work. But the really thing bigger picture is what happens to the future of work and how does that transform to be better. Right? When we've done there's a bunch of studies that show something like 74% of work in big companies is coordination work and now you can get rid of all of that. Eric Binolson calls this white collar drudgery. You can get rid of that. Work on let AI handle a lot of the crap stuff that copying and pasting sales figures from one report, one system into another. And then work on

[01:48:02] where the where your judgment and experience make the biggest uh uh uh biggest uh difference in the work and for the company and for the organization uh you're going to work. People worry about trying to work around policy around this. figure out how you maximize human agency as we go through this crazy transition and human agency is exploding because of AI. >> Dave, what are you seeing all your companies? I mean, you're chair and on the board and founder of so many companies growing, hiring. Yeah, I mean I think hiring and I I want to be really careful to say you know you have a million AI related jobs and rising but within that there's a lot of co-pilot type jobs where you're basically twice as efficient as you were using a co-pilot but there's another subset that are using 10 100 and soon a thousand agents and managing them as if they were employees. That's where you want to be. You want to try and get to that group as quickly as you can because you know just using a co-pilot you know the bar is rising fast. That's not good enough. you

[01:49:00] you need to move on and manage swarms. And uh I interestingly I met my first hire who uh does all of his work through voice. So he's managing you know agents many agents. He's not using a certy keyboard. Alex he has trans this is a young guy at 20 really transcended history there. So much for his >> but I I really think you know you mentioned Eric Bolson white collar drudgery. A lot of that stems from being in a in a chair for many two hours. It's bad for your back, looking at a screen, going blind. That's all going to move over to standing up, moving your hands around a minority report and controlling the agents in a much more dynamic environment. I think it's a much much happier place for humans to exist. It's it's very much like Iron Man. You you're you're Robert Downey Jr. and you're just building together with your Jarvis. That's actually where we're really going in the next year. And so the the amount of awesomeness in that job function is is just incomprehensible. >> Yeah. The amount of work I do with with

[01:50:01] Skippy while I'm driving and just having a conversation, say, "Do this work, write this report, get me the answer." It's incredible, right? Every moment, >> crap, >> every moment becomes capable. >> Go back to the automation days, right? One concrete mixing truck replaces about a hundred workers with shovels shoveling concrete. Okay? >> Nobody wants to go back to shoveling concrete. like what are what are we thinking about just let's move forward please. >> Yeah. You know the other thing that's really clear to me is that >> we were predicting massive white collar job disruption just a year ago. The choice to not do that was an active choice by the big AI labs under pressure from the White House and also in China you know lawsuits if you just wantingly fire people you have to actually try to retrain them to become an AI person and you have to pay either way. So the governments got involved and said we don't want massive voter disruption. The AI companies are saying it just doesn't matter that much. We're we're we're on this abundance curve that's so steep

[01:51:00] that working within AI to create new drugs, to create new physics, to create new math is so much more important than disrupting everyone's life that we're just going to do it the easy way and and do it with the cooperation of the White House. And so I think I think that's what's actually happened and likely to continue. So, as long as you're part of that rising wave and you're not, you know, sitting in legacy land, you're going to do really, really, really well and there isn't going to be massive job disruption. >> IMAD, you wrote a bestseller, The Last Economy. What do you make of the future of jobs here? >> I think it's like the turkey before Thanksgiving. um getting plumper, you're adding the jobs, but the models have reached that level of capability now that you can replicate a whole digital workforce in a year and then physically it'll come after. I don't think it will create jobs fast enough, but we can create abundance. So, I put forward a champion proposal last week, dollar pre- money, everyone, all the kids own the equity of

[01:52:00] the robots and other things. But I think we can do one better. I think the government should have a massive infrastructure program and it should look to build a 100 million robots in America and they should be owned by the people. And I think that is how you get abundance. Like again, if you want to have an end purchaser, make them owned by the people. Have that as this massive infrastructure buildout cuz that will upgrade all of America's infrastructure. And I think that's where we have to go. Elon said at the G20 last week, "A robot can do the work of five humans." This is true. Like factually, we know where it's going. So, I think that we have options of where to go. In the meantime, we have to get ahead of things. There are jobs that will get crazy. Like HVAC specialist is the one that I think you're seeing electricians already earning $600,000 a year working in data centers. >> But there's just not enough humans, right? I think the government also in America where it's going currently will push back against robots. Um, and so electricians, HVAC, like there's you'll

[01:53:01] be surprised, especially in Europe, actually. It's so hot here that HVAC rollouts will do so well. >> That's a really, really important point. I think um Andrew Yang actually, I think, was saying the same thing that we're going to see this huge resurgence of trades. But I think a lot of people have been trained since, you know, a young age that a good white collar job manipulating a spreadsheet is a far greater ambition than being the best HVAC specialist in the world. And so they they kind of demean the blue collar world. But the white color job is the one that's going to become irrelevant. And so a really good life plan is to become incredibly great at helping building data centers, cooling systems, liquid cooling, and then reinvest that money into AI companies that are that are riding the wave just as a life plan. It's a much much better plan than aspiring to get a degree in accounting right now. So look, like it is inevitable that there's going to be the biggest infrastructure buildout of all time, not just in data centers, but

[01:54:02] again across America, across Europe, our infrastructure is crumbling and governments will go there. So position yourselves there if you're not using a thousand agents and you will get the biggest tailwind of all time. >> Alex, your thoughts, please. >> Yeah, well, a couple thoughts. one I I'll I'll take the position I don't actually think it it is advisable at least in America for the government to be owning 100 million robots. I I think that >> you say the people not not the government. >> Okay. But how do the people own 100 million robots through presumably some form of centralized government which again to to my maybe overly American ear smells like uh wait for it like luxury automated communism. uh- which is I I think what it what the the subtext would be. I'm not in favor of that. I I would like to see every American owning a thousand robots. I I don't think they necessarily need to be socialized or communally owned. But I I do think I mean every profession as currently

[01:55:02] construed right now I think is cooked. Uh and I think the sequencing of the cooking is what determines social policy. So maybe it is the case that certain white collar professions right now can be automated earlier more of a paradox style than say quote unquote bluecollar professions HVAC engineering and the like. But HVAC engineering let's not kid ourselves HVAC engineering in the next few years with humanoid robots is just as cooked as spreadsheet management and accounting. It's just a matter of sequencing. So I I think you one I think it's essential to distinguish between the what passes for at this point the short term and the long term. In the short term yes I think I I agree with the premise that there is some remaining alpha in the trades so-called but in the long term no the the trades are just as automated and automatable as white collar so-called labor. >> That's why I think they'll slow down the robots. But I think again the robots are the long term and as I said the

[01:56:01] government doesn't need to own them. Sovereign wealth funds are my champion idea or just the government can underwrite the robots that the people can own. I think the main thing is you have to get the ownership of the robots to the people somehow. >> I I think the sequencing is really important that Alex was referring to too. But if if if Elon Musk calls and says, "We're willing to pay up to $600,000 for the best electricians and plumbers to show up in Tennessee to build Colossus and the job is only there for one or two years and then it's automated. Take the job, take the money, stay nimble, and then the next opportunity will open up and and you'll know a bunch of people that are in the same boat in the middle of the AI revolution." And so so then the sequence will evolve and we'll keep on the podcast telling you where to move next. But don't don't take it as a sign of well because that's cooked two years from now I'm going to do nothing tomorrow. Don't do that. Take the job. Build the data center. >> I agree. And and and maybe just to to underline Dave your point further. I I think so I I I'm always coining neisms

[01:57:03] uh including motion with an a m o a t io n which is this notion uh that there are no stationary modes in a singularity but what there are are dynamical modes uh temporary modes if you will. And if you sequence them appropriately then one mode can lead to another can lead to another and you achieve a dynamical mode. So same idea with professions maybe an HVAC engineering position now enables and this is not investment advice or career advice but hypothetically maybe an HVAC engineering position now creates enough of a capital base that can then be grown and translated to something else in 2 years that can then be translated to something else dot dot dot eventually you get to I don't know owning a planet. So Salem uh let's take the conversation one step further which is what happens when you don't have to work what happens when all of your basic needs food water energy healthcare education liberty all of that is enabled for you right you and I have discussed this before going back to the

[01:58:00] Medici family and the renaissance and so forth where you you basically have the life and I've written on Substack about this where you have the life of a a gazillionaire you don't need to work what do you do Right. Um, I think that's an important realization because when we get to the point where everything is cooked, that's also the point at which we have massive abundance, where all of your needs are taken care of for you. And now the question is, what do you want to do? You know, most people in the world are are working because they have to put food on the table. They have to get insurance for their family. It's not what they dreamed of doing when they were a kid, right? So, it unleashes massive possibility at that point. Selene, >> I I think there's two categories here. Category one is the material needs, right? When you get to that level and you it it'll take a transition to stop thinking about meeting daily needs, right? Uh like half the country in the US can't put $500 together in an emergency. Okay, that itself is an emergency. Um uh and you've got such a

[01:59:03] monster structural problem around that that has to be addressed. Now at some point pretty quickly let's use the simplest argument that somebody goes deep with AI figures out how to use it to do an active trading hedge fund strategy starts making enough money to pay for themselves pay for their families etc etc right and overall aggregate in the aggregate people will figure out and society will figure out how to generate huge amounts of wealth with this and then you have the distribution question of how do you equitably share it when we've studied that but that's all the material covering day-to-day life stuff covers the bottom two three layers of Maslo's hierarchy. Uh and we could see that happening and when you see societies getting to this abundance level like the Mongols taking over India or the Mongols taking over uh East Asia or the Romans taking over the Medici family or whatever you've heard us talk about the idea that people end up doing four activities uh food, art, music and sex

[02:00:00] and the joke is not in that order, right? Um once you get past that and get through that, the really interesting next dimension opens up and you start thinking about what problems could I tackle because not we're not going to run out of problems. We're just going to be be able to tackle bigger and bigger problems, right? How do we create Alex's Dyson swarm? Uh how do we go into the stars? How do we uh kind of think about the new different types of physics that may emerge from all of this? That's where the really fascinating stuff comes along. And you know the I think this is the area Peter we've had this conversation at the X-P prize board meetings of could we design prizes that advance humanity radically than just trying to solve the problems because those look like they'll get handled and what's that next phase look like? I think the opportunity for human flourishing in that is so magnificent. We love problems and we love solving problems. We'll just have bigger and bigger problems which we'll be able to solve with bigger and bigger suites of AI agents and robots. All right, I'm

[02:01:00] going to move us to our last story in economics. And this is a paper Salem I know you're going to love andor have lots of comments on. So, quick context. As See has educated us on past episodes. In 1937, the economist Ronald Coos uh asked why companies exist at all. He won the Nobel Prize for this. His answer, transactions are expensive. Finding people, negotiating, and enforcing contracts. It's cheaper to hire employees and have them on the inside of your company rather than negotiating every task on the open market. Companies are a workaround for transaction costs. Now, MIT and Harvard researchers are finally catching up with Seem uh have asked what happens when AI agents make transactions nearly free. Agents that search, compare prices, negotiate, and transact for you at enormous scale. They're calling it the Cosian singularity. Salem, you wrote the book on this. Uh your thoughts on this paper? >> Uh absolutely dead on. A little late,

[02:02:01] but uh totally dead on. You know, Peter, in our 2023 book, um Exo 2.0, we we said then that the KOS's law was breaking. We didn't understand the full implications of it. What we noticed then was that take the mission critical function in Uber, which is to match driver and passenger. It doesn't happen inside the organizational boundary of Uber. It happens out in the wild. And by enabling that with technology, you can scale. So we observed the companies were reaching outside themselves to get things done. Prize goes to teams all over the world to do innovation. TED is using its community to scale. But now with AI, it completely changes the game. And now the transaction and coordination costs go to near zero when you can have a thousand agents or 100,000 agents doing kind of crazy amounts of capability uh outside the organization. I think the bigger point here is that AI doesn't just automate the firm. It attacks the economic reason for which firms exist

[02:03:02] and the shape of the firm. And what we've been exploring is what does that shape of the firm look like? And we've got kind of that definition going because now essentially a firm becomes a protocol, right? And a and a community of agents and human beings is going and attacking various economic opportunities or marketplaces or solving specific problems. So the the organization dissolves from being a human hierarchical centered model to a totally different world and so this is a huge shift the biggest sense we've had since the industrial revolution and we have to kind of look at what this new model means and we're looking a lot at the governance for example of these agents and so on. So I have to ask the the question then I I think this is correct me if I'm wrong this is research from last year not not from this year the the cosian singularity if that's the case in in my mental model so the the whole point of of cosian economics is transaction costs determine the size of the firm and if transaction costs are

[02:04:02] high that agitates in favor for a larger firm and if transaction costs are low that agitates for a smaller firm. So if transaction costs are going to zero that agitates in favor of the size of the firm going to zero. So if that if my reasoning is sound do we think the size of the firm is going to be one person or less than a person? What is that? What's the limit? Uh I don't think there is a limit. If you take the Argentinian model you could have agents running the firm. Now the concern you still need an ent you still need an entity. Okay. So Ted Shelton and I wrote this paper on what does the organization look like if you take the coordination and so and we coined a term called a fiduciary wedge. The reason you still need an entity is for liability for fiduciary for proprietary data ownership for learning loops inside that uh inside that entity for brand for as a purpose container as an MTP container etc. But the primary

[02:05:00] reason for firm has been coordination and execution and that kind of disappears. So you still have entities in the same way like in investing you have SPVS and you can have multiple people be part of an SPV that then invests in something essentially you still need the legal container but the primary reason which coast identified and all organizational thinking is does come from that uh that there's no reason why a company or firm or legal entity can't be owned by other agents which will eventually happen in Argentina and other places uh and then have a completely virtual organization operating in its own domain for the purpose that it was set up for. >> I want to push on that though because there's a counterveiling force that one might perceive which is we talked earlier in this episode which is frontier capabilities potentially getting walled off from the rest of the economy. If OpenAI for example hypothetically is using internal unreleased models to solve grand challenges in math and the rest of the economy doesn't yet have access to them as a friend of mine run at OpenAI others

[02:06:02] have point out wouldn't that agitate in favor of the exact opposite coian economics of firms growing larger and larger so everyone has access to those internal capabilities within the frontier labs. You could, but I think that's an edge case. Uh, for the most part, as open models allow people to have general intelligence and agents across the board. And so, you could already have a frontier lab running its own hedge fund strategy, and I'm sure they're doing that now that outperforms the market and just running that model. But it's a very niche thing applying for a certain temporal period of time. Uh, over time, I think the big question in my head right now is if we uh are achieving RSI, what the hell does that mean? I think when we get to that point, the concept of an economy starts to erode and dissolve itself. So you have to think about it in a totally different model. >> Not not as obvious. I mean, this seems to me a lot. What's the the apherism like what what happens when an irresistible force meets an immovable object or or something like what happens when a Cosian large frontier lab with

[02:07:03] superhuman super intelligent capabilities meets an agentic economy that wants to distribute transaction costs out to the edge. and combined with Argentina wants to create non-human corporations. Not obvious where that ends. >> Alex, you remember when Sam Alman said, "Yes, in the future I think an AI should be running open AI." >> I remember that. And Sam, just in the past 24 hours also on social media expressed surprise, shocked, shocked that Open AI was able to solve uh uh Navier Stokes. So maybe an AI would have predicted this. >> Yeah. I mean look if you take your commentary to the end point then you end up with what Daria was talking about where one company like Anthropic will be all private enterprise right um and the that's unlikely to happen it'll take a while and at that point the concept of what does it mean to have an economy essentially evolved I think the bigger picture question is where do we even

[02:08:00] have value uh creation and value storage and all of this because we come down to the money layer at some point and We should have Jeff Booth on. We were talking about guests earlier. We should have Jeff on talking about what the future of that. >> Those are old tokens. >> Iman, you I see you. I see you ready to burst forward. >> Yeah. No, I I think that the economy is like 1% inspiration, 99% perspiration. Like you don't need a polymath doing your taxes or selling widgets or things like that. Like we've seen that obviously the Navia Stokes model can sell widgets very well, but once it comes up with a recipe, it's actually about following through. And most transaction costs are actually about friction. Like the economy needs a bit of friction. It's the relationships you build, it's the other things. It's not an instant thing whereby there are no barriers to spreading. Again, Sam talked about this, the frictions in a scarcity oriented economy. It's not a case that the best product always wins or we'd all be on Betamax, you know, like that's something for the old kind of kids.

[02:09:01] >> That's Betamax for Americans. >> What's that amount? Never heard of it. >> Young kids anyway. So the, you know, like there there are lots of frictions in the economy. So I don't think it will be that case. It's just that the companies and organizations become more efficient and then they become more optimized because it's really annoying when you go past 12 people and then past 150 people. This is where you kind of have that Carthaginian demon of disorder mock coming into organizations and they get misaligned. So I don't think you'll see the oneperson company. I think you'll see the 10 person company and I think you will see more collectives of companies operating with digital and physical humans solving problems that deliver value. But economics fundamentally does need to flip from being scarcity based to being abundance-based because we can rearrange bits and soon atoms in any way that we want. All right, Dave, do you want to close us out? Any thoughts? >> Well, I think to step back from the academic, you know, our company's getting bigger, our company's getting smaller. Clearly, Meror now has what 50

[02:10:02] to 100,000 individual actors that are effectively little companies in India and around Brazil now too. uh and and you can you can take what Sem is saying and build a marketplace around it to deal with just the lingering artifacts like you know employment law in different countries and make a fortune by taking advantage of the trend that Salem is describing and concurrently with that Elon is building the single biggest integrated vertical company that the world has ever seen with complete supply chain control all the way down to raw sand turning into chips. And so those are both happening in the real world concurrently. So I think maybe the real observation here is things are changing very very rapidly in both directions for very good reasons and you know the the academic paper can can wax poetic for years but this is happening in the real world in those two flavors and it's >> brilliant. I agree agree with you. All right. Uh I want to turn us next to Tesla Cyber Cab. Last Thursday we

[02:11:01] covered the Austin launch what I was calling CyberCap La Palooa. Uh this week Tesla opened an official interest form for businesses who want to buy their own cyber cab fleets and build mobility hubs and charging infrastructure for the robo taxi robo taxi network. Uh no pricing or delivery terms yet. Uh this you know sem for me this is another Musk business model innovation. Tesla doesn't want to own every robo taxi. You buy it works for you. Uh you share the revenue with Tesla. It's a brilliant move for customer financing of a global fleet. And honestly, uh, this can only work in my opinion because of the price tag on cyber cabs projected at $30,000, affordable to anyone who was previously an Uber driver. Um, here's the form. Uh, I've gone and filled mine out. Wonder if you guys did. Let's hear a quick video from uh from Elon. Uh, this is an old video, but it, you know, it predicts what he's saying and doing now.

[02:12:02] Uh but uh we we'll have a model which is kind of like uh some combination of Uber and Airbnb. So if you if you're a Tesla owner, uh you'll be able to add or subtract your car to the fleet. So just like an Airbnb, you could like rent out your spare bedroom or rent out your house um when you're not using it. And uh the same thing will be available for Tesla owners. >> So did you guys uh see that announcement? Any thoughts? >> Well, it's exactly what Alex was saying a minute ago. If you're the electrician or HVAC person working on Colossus and you banked 600 grand, where do you go next? You would have heard about this with your peers while working on that. This is where you go next and then the next and the next. But this >> Well, you buy the Optimus robots that now do the work for you. >> Exactly. Exactly. That'll probably also be syndicated out as some kind of a franchise model for maintenance, repair, and whatever. And that becomes your trajectory of the future. >> Yeah, totally agree. I mean I I think in the style going back to I'll self-sight earlier comments about the private

[02:13:00] ownership of the capital means of production. I think this is the way again not investment advice but decades ago would have been accumulating a laundromat or a restaurant franchise. Now it'll be yeah you own a fleet of robo taxis or a fleet of humanoid robots. That's without this being misconstrued as investment advice. That's sort of the call it the franchising path to medium and high income and wealth creation. And I I think the SMBs of the present and the near future are going to look far more like that than opening say a chain of restaurants. >> And also when these things get started, when they're new, >> the the central force wants you to succeed so badly to get the momentum going that they subsidize the hell out of your success. Like if you were the the fifth Starbucks owner, you would have been guaranteed success by the mother ship. You don't want to be the hundred thousandth. But if you get an early jump on this and whatever's next and whatever is next, the mother ship will subsidize the heck out of your

[02:14:00] success. So you just have to basically not screw it up. >> I I find this incredibly compelling, you know, to go fill out the form and and say, "I'm going to grab 10 of these and have them work for me or hundred of them." uh end of the day it's earning revenue while you sleep and improving your your local community. There's a simple economic thesis here which is dropping marginal cost, right? Uh you think about Airbnb's marginal cost of adding a room is near zero. If you're high, you have to build a hotel. Same thing here. The marginal cost of owning one of these and adding having people use it and leveraging that asset rather than centralized ownership of taxi cab fleets is such an obvious no-brainer. So, I think this is a huge opportunity. Dave makes a great point. get in early on these things because then you have scale built in. >> And the question of which of the cyber taxi companies or robot taxi companies, we're going to call it going to win. It's the one with the lowest operating cost uh the lowest uh production cost and I haven't seen anything yet compete with uh with Tesla here

[02:15:01] >> or the one unfortunately that may be coziest with the municipalities that are approving them. >> True. It will be city by city. >> Usually it's pretty obvious who the winners are. Boston. >> It's so sad, Peter. So sad. >> It's so painful. >> Um, the the winners are usually pretty obvious is people are just so slow to react. They're not nimble enough. But it's not going to be a mystery who's going to win. It's going to be people underreacting to the urgency of the opportunity. >> Sorry. Go ahead. >> No, I just think it's becoming increasingly clear robots are the biggest investment class that we'll ever see. Um and so again like we'll see SPVS, we'll see funds, we'll see all this emerge and then as you reach that level of capability whereas Alex said they can take over a HVAC engineer or in this case do you need more than a two-seater robo taxi for just about anything? No, only when you want to have your air robo taxi. That's the only time you need anything better than that. Like these will have very long lives and they

[02:16:01] will earn money in just about any scenario. Yeah, you can find out more at tesla.com/roboaxi if you want to jump into this future economy and again um not promoting it and not uh giving investment advice doesn't get a commission on this one. >> I do not I do not um I do find it incredibly compelling. Um all right, I want to bring us to our our final story here today and it's uh uh it's an important one. Um, you know, you know, this is underlying much of what we discuss on moonshots. So, we've noted before, you know, people are having fewer babies. Uh, we're seeing a we're going to see a massive drop off in the human population and people are now living longer and healthier. And this is changing the global demographic. So uh here's a chart showing the growth in uh the number of people over 65 and the drop in the number of newborns 0 to age 5 on the planet. The global population

[02:17:02] over 65 is projected to grow from 852 million uh back in 2025 to 2 billion by60. That's more than half of all population growth over that period is coming from people over 65. The result, at least in the old economy, is that fewer working age people supporting more retirees is going to put huge economic pressure on pensions, on health care in every developed economy. The only way the math works out in this future is because of the topics we discuss on this pod, right? AI and robots doing the work for missing workers, people living longer and healthier, not needing to retire, maintaining themselves as economic contributors, right? What's the reason you retire? You're in pain. Uh you're feeling less energy or you're forced out uh because of policy. And this is why I say longevity is not a luxury. It's an e economic policy for the century ahead. Right? An 80-year-old with the body and

[02:18:01] mind of a 50-year-old isn't a pension liability. They're a founder in this future economy. So, you've all heard the saying, you know, demographics are destiny. Well, robots that do the work and therapies that add healthy decades are rewriting destiny. Um, gentlemen, uh, your thoughts. Selene, you want to jump in first? >> Uh, very simple followup. I mean, this is why we need the robo taxis and the robots and so on because we're going to need to redesign the entire concept of uh, lifespan to health span to jobs. You know, the concept of education, career, retirement essentially evaporates, right? you have to have repeated cycles of learning uh creation and then taking a sabbatical etc etc and longevity combined with AI totally disrupts uh who works, how they work, how long they work, what they do and so it's going to completely change the game. We need to rethink that whole thing from the bottom up. We've seen the preview with what's happening in in Japan and you you

[02:19:03] need it. China is moving to robots because they have to because of the one child policy and the population bomb that got coming towards them. >> Great point. So they have to do it. They don't have a choice. >> Yeah, these numbers can make the world much more interesting. Sorry. >> These numbers are hugely understated because they include India and parts of Africa that are still having babies like crazy. But if you look at China and you look at Europe, the numbers are much more acute than this. It's like that the there will be basically kindergartens and grade schools that are completely empty and massive numbers of over 65s. Uh and and it's not very far in the future. It's also America is largely immune because we have huge amounts of immigration right in the working age bracket. You don't tend to have a lot of immigrants that are 65 and over. And so so you won't notice it as much in the US, but other parts of the world this is this is basically going to completely rip the working class out of the economy. And then the voters are overwhelmingly not employed. And so it's

[02:20:00] going to create I mean it's already a mess, but it's going to create all kinds of strange things in those in those jurisdictions. >> I I would just add this is what victory looks like. That this is what we want to see. 150,000 people plus per day on Earth are dying. and putting an end to that, at least the beginnings of putting an end to that is what this looks like, where people are starting to live longer, uh, and people are starting to stop dying. And the the counterfactual looks more like the limits of growth from the Club of Rome, which I think was like early 1970s, potentially, depending on your vantage point, horribly racist perspective on what in its earliest form. >> Yeah. like based on the false premise that somehow humans are going to be overcrowding uh purportedly scarce surface area of the planet and that was going to lead to decrepit conditions of living for everyone. Utter nonsense. This inverted pyramid which some would

[02:21:00] call you know I I guess the opposite of of pro-atalism. It's not anti-proatalism. It's this is what a happy future looks like where we have ultimately far more AI agents than we do humans and longevity escape velocity is vanquished. We want this future. >> Yeah. So I think that there are two major opportunities here. One is integrated elder care with robots at a high level but the other one is integrated baby care. We should have more kids. Kids are wonderful and having a fully integrated care for kids throughout that can reduce the cost of child care and supporting and raising kids in the best way is probably the biggest thing anyone could do for humanity cuz there deserves to be more of us. So, let's have more kids. Let's figure out that problem. >> All right, gentlemen. I love this episode. You guys, I I love spending time with all you. This was brilliant. >> Listen, I need to say something about what your Matt said. >> Yeah. love my kid to death, but damn the work of being a parent is the biggest biological scam ever.

[02:22:01] Just is an unbelievable amount of work. >> It's very rewarding. It's very between those comments and then your prior comments about marriage o over longevity escape velocity. What's going on? >> Lily, I get permission from Lily to make some of these comments. >> All right, guys. Uh we'll be infinity away in 3 days. We'll see you again soon for our next episode. Dave, Alex, Salem, Immad, love you guys. >> Great conversation. >> Brilliant as always.