<p align="right"><font color="#3f3f3f">2025年09月30日</font></p> --- # 万亿美元AI建设内幕 | Dylan Patel 访谈 ## The OpenAI and Nvidia Deal: The Infinite Money Glitch ## OpenAI与Nvidia交易：无限金钱漏洞 **0:00** If the models don't improve, we're absolutely screwed. And in fact, the US economy will go into a recession. It's about the highest stakes like capitalism game of all time. Godsend in terms of like how much efficiency and value can be created and it doesn't ever have to get to like digital god level. Now, I do believe we're going to get to digital god level eventually. Eventually, if I could have an intelligence as smart as a Google senior engineer, that's $2 trillion of software value. Is that the main like bottleneck to be attacked? We're popping the bubble right now cuz the limit of AI is infinite. 如果模型不再改进，我们就完蛋了。事实上，美国经济将陷入衰退。这就像是有史以来最高赌注的资本主义游戏。从能创造多少效率和价值的角度来看，这简直是天赐之物，而且它永远不必达到数字上帝的水平。现在，我确实相信我们最终会达到数字上帝的水平。最终，如果我能拥有一个像谷歌高级工程师一样聪明的智能，那就是2万亿美元的软件价值。这是需要攻克的主要瓶颈吗？我们现在正在戳破泡沫，因为AI的极限是无限的。 **0:39** I was going to lay out this idea of going through the past, present, and future of compute as like the big big idea for our conversation, but since it just happened, I don't think you've heard you talk about it anywhere. I'd love to start by asking about this whole OpenAI Nvidia thing, which uh sounds exciting, seems vague, not really sure what's going on. and maybe you could explain it to us as you see it and what the strategic implications are of the big announcement. 我本来打算把过去、现在和未来的计算作为我们对话的重要主题，但既然它刚刚发生，我想你还没有在任何地方谈论过它。我想先问问这整个OpenAI Nvidia的事情，听起来很激动人心，但似乎很模糊，不太确定发生了什么。也许你可以从你的角度向我们解释一下，以及这个重大宣布的战略含义是什么。 **1:00** All right. So, I think it's I think it's very very simple, right? You've got OpenAI paying Oracle lots of money. You've got Oracle paying Nvidia lots of money. You've got Nvidia paying open lots of money meme. We've got we've got the infinite money glitch here. Uh no, no, no. I that's not actually what's happening, right? What's really happening is open air has an insatiable demand for compute. 好的。所以，我认为这非常非常简单，对吧？OpenAI向Oracle支付大量资金。Oracle向Nvidia支付大量资金。Nvidia向OpenAI支付大量资金的梗图。我们有无限金钱漏洞。呃不，不，不。我说那实际上不是正在发生的事情，对吧？真正发生的是OpenAI对算力有着永不满足的需求。 ## OpenAI's Compute Challenge and Capital Requirements ## OpenAI的算力挑战和资本需求 **1:26** Um the compute precedes the buildup of business. You have to have the cluster before you can rent it out for inference, right? Or rather run models on it for inference. You have to have the cluster to train the model that's good enough that it unlocks new use cases which then can be adopted and there's an adoption curve there for any new use case. 算力先于业务建设。你必须先拥有集群，然后才能将其出租用于推理，对吧？或者说在上面运行模型进行推理。你必须拥有集群来训练足够好的模型，以解锁新的用例，然后才能被采用，而任何新用例都有一个采用曲线。 **1:44** So you have to have all these things like sequenced given this is a game of the richest people in the world or rather the biggest tech giants in the world, right? It's Zuck. It's Google, you know, Larry and Sergey or Sergey is like constantly in the business now again, right? It's all the biggest people in the world. It's Elon, right? 所以你必须把所有这些事情按顺序排列，因为这是世界上最富有的人或者说世界上最大的科技巨头的游戏，对吧？扎克伯格、谷歌，你知道，拉里和谢尔盖，或者说谢尔盖现在又经常参与业务，对吧？都是世界上最大的人物。还有马斯克，对吧？ **2:02** There's there's very much a risk of OpenAI being too small to matter, right? You know, which is crazy to say because they've got 800 million users, but like where's the revenue? Where's the compute? They could easily get swamped in terms of having of of how much compute they have. OpenAI非常有可能因为太小而变得无关紧要，对吧？你知道，说这话听起来很疯狂，因为他们有8亿用户，但收入在哪里？算力在哪里？他们很容易在拥有多少算力方面被淹没。 **2:24** The magic of OpenAI was that they just spent way more compute on a single model run on GP3 and four and they had the foresight and the vision and the execution. Yeah. But they made that bet and they were able to secure it. And at the time it was like meh, right? It was a few hundred million, whatever, right? you know, that's a ton of money. But like now it's sort of like, well, Mark Zuckerberg sees how much compute he's going to have to get even though he has this insane cash flow. OpenAI的魔力在于他们在GPT-3和GPT-4的单次模型运行上花费了更多的算力，他们有远见、有愿景、有执行力。是的。但他们下了那个赌注，并且能够确保它。当时感觉还好，对吧？几亿美元，无论如何，对吧？你知道，那是一大笔钱。但现在就像，嗯，马克·扎克伯格看到他需要获得多少算力，尽管他有疯狂的现金流。 **2:50** That he's like, "Oh, wait. I need to go sign a deal with Apollo for, you know, $30 billion on this data center, right, in Louisiana, this mega data center I'm going to build." It's like, "Wait, why didn't you just fund this with cash flows? You have so much cash flow." It's like, "Because my plans, that's just the physical data center. Now, what am I going to put in it?" Is like so much money. 他就像，"哦，等等。我需要去和Apollo签一个协议，你知道，在路易斯安那州这个数据中心300亿美元，我要建造的这个超级数据中心。"就像，"等等，你为什么不直接用现金流来资助这个？你有这么多现金流。"就像，"因为我的计划，那只是物理数据中心。现在，我要往里面放什么？"需要这么多钱。 **3:09** The amount of capital that people are going to have and are dumping into this is is insane, right? Google was slow to wake up and then you know they were slow to pivot their data center operations. They were slow to do everything and so there while they could have way more compute than anyone by a humongous degree they haven't been able to deploy as fast. 人们将拥有并倾注到这里的资本数量是疯狂的，对吧？谷歌反应迟钝，然后你知道他们在转向数据中心运营方面很慢。他们做所有事情都很慢，所以虽然他们本可以拥有比任何人都多得多的算力，但他们无法快速部署。 **3:32** So if you have this like tremendous vision of what's going to happen with AI, you know that it takes a ton of compute to build them, you know pretty much the amount of compute you could dedicate to these models is limitless and they will get better. Now it's a log log scale, right? I.e. you need 10x more compute to get to the next tier of performance. 所以，如果你对AI将要发生的事情有这种巨大的愿景，你知道构建它们需要大量算力，你知道你可以投入到这些模型的算力数量几乎是无限的，它们会变得更好。现在是对数对数尺度，对吧？也就是说，你需要10倍的算力才能达到下一个性能层级。 **3:43** You might think of it as diminishing returns, but what if the next tier of performance is like, you know, you know, a six-year-old versus a 16-year-old. Like child labor is like quite like effective versus a six-year-old you can't get to do much, right? 你可能认为这是收益递减，但如果下一个性能层级就像，你知道，6岁孩子和16岁少年的区别呢？就像童工相比6岁孩子确实有效得多，6岁孩子你让他做不了多少事，对吧？ ## Oracle's $300 Billion Bet on OpenAI ## Oracle对OpenAI的3000亿美元赌注 **4:01** Um, they have to get more compute than anyone or at least among there. They have to race with the giants. These giants are trillion dollar businesses. So, how does OpenAI get there? Well, it's it's partnering with Microsoft. Well, that soured some, right? Um, it's partnering with Oracle. Well, Oracle can can do a lot, but Oracle doesn't even have a balance sheet like like Google and Microsoft and and Amazon and, you know, etc., right? 他们必须获得比任何人都多的算力，或者至少在其中之一。他们必须与巨头竞争。这些巨头是万亿美元的企业。那么，OpenAI如何到达那里？嗯，它与微软合作。嗯，那有点变味了，对吧？它与Oracle合作。嗯，Oracle可以做很多事情，但Oracle甚至没有像谷歌、微软和亚马逊那样的资产负债表，对吧？ **4:40** OpenAI needs allies, right? They need they need people to effectively spend the the capex ahead of the curve and trust that they'll be able to pay the rental income because that's what it is at the end of the day. OpenAI is committing to fiveyear deals. OpenAI需要盟友，对吧？他们需要有人在曲线之前有效地花费资本支出，并相信他们能够支付租金收入，因为归根结底就是这样。OpenAI承诺五年协议。 **4:51** These five-year deals cost X amount of money. It's 10 to 15 billion dollars per gigawatt of data center capacity that you pay a year. And then that 10 to 15 billion dollars of gig for a gigawatt of data center capacity. You're paying that for five years. Okay, that's 50 to 75 billion of of cash that goes out the door to OpenAI for one gigawatt of capacity. 这些五年协议花费X金额。每千兆瓦数据中心容量每年支付100亿到150亿美元。然后那100亿到150亿美元用于一千兆瓦的数据中心容量。你要为此支付五年。好的，那就是500亿到750亿现金流向OpenAI用于一千兆瓦的容量。 **5:08** And you talk about what Sam's saying is like, hey, I need I need 10 gigawatts. I need more than 10 gigawatts, right? Then you end up with this like really challenging aspect of like how do you pay for that? And hey, that's only the rental price. If I were to actually do the capex, it's or if I were to like, you know, because it's frontloaded, right? 你谈到Sam说的是，嘿，我需要10千兆瓦。我需要超过10千兆瓦，对吧？然后你最终面临这个非常具有挑战性的方面，比如你如何支付这笔费用？嘿，那只是租金价格。如果我实际进行资本支出，或者如果我要，你知道，因为它是前置的，对吧？ **5:26** That's the reason these deals are coming about. And so Oracle is making a massive bet, right, Larry? You know, hey, he's getting good margin off of it, but he's making a massive bet that this capex that he's going to pay for OpenAI will actually be paid cuz you know, he signed a $300 billion deal with OpenAI. 这就是这些交易出现的原因。所以Oracle正在下巨大的赌注，对吧，拉里？你知道，嘿，他从中获得了良好的利润，但他下了一个巨大的赌注，即他将为OpenAI支付的这笔资本支出实际上会被支付，因为你知道，他与OpenAI签署了3000亿美元的协议。 **5:44** It's like, where's that going to come from? Yeah. is like you you your revenue is like 15 billion ARR this month maybe right on a run rate basis it'll get to 20 by the end of the year uh pretty pretty clearly maybe it's like 16 now but how do you pay $300 billion of revenue u now if if the bet works out they've just made 100 billion of profit right like just pure cash profit it's crazy but if they if it doesn't work out they've got this huge and they they're starting to raise debt right they there was a small deal they signed recently um but they're going to start raising more and more debt 就像，那将从哪里来？是的。就像你你的收入大约是本月150亿美元的ARR，也许对吧，按运营率计算，到年底将达到200亿，非常清楚，也许现在是160亿，但你如何支付3000亿美元的收入？现在如果如果赌注成功，他们就赚了1000亿美元的利润，对吧，就像纯现金利润，这太疯狂了，但如果如果不成功，他们有这个巨大的，他们开始筹集债务，对吧，他们最近签署了一笔小交易，但他们将开始筹集越来越多的债务 ## Nvidia's Strategic Investment and Deal Mechanics ## Nvidia的战略投资和交易机制 **6:08** So this this game now now Nvidia's kind of got the same conundrum right it's like well Google and Amazon are doing these these deals whether it's to two other vendors for TPUs or for tranium whether it's anthropic or others they're trying to court openai uh they're trying to court other companies how do I get into this game right okay fine I can rely on Microsoft somewhat I can rely on Oracle somewhat but at the end of the day GPUs if I want GPUs to be king part of it is just like my chip is the best but part of it is also who's going to pay the capex upfront 所以这个游戏现在Nvidia也面临同样的困境，对吧，就像谷歌和亚马逊正在进行这些交易，无论是与其他两个供应商进行TPU还是Trainium的交易，无论是Anthropic还是其他公司，他们试图争取OpenAI，他们试图争取其他公司，我如何参与这个游戏，对吧，好吧，我可以在某种程度上依赖微软，我可以在某种程度上依赖Oracle，但归根结底，如果我想让GPU成为王者，部分原因是我的芯片是最好的，但部分原因也是谁将预先支付资本支出 **6:39** Google and Amazon will pay the capex up front if it's for TPUs or right they won't pay the capex up front necessarily for that same capacity of GPUs so you've got this like challenging aspect and so that's where this this Nvidia and open I deal comes 谷歌和亚马逊会预先支付TPU的资本支出，或者对吧，他们不一定会为同样容量的GPU预先支付资本支出，所以你面临这个具有挑战性的方面，这就是Nvidia和OpenAI交易的来源 ## Understanding the Demand Dynamics ## 理解需求动态 **6:53** from I want to dig into the underlying assumptions driving this on the training and inference side because obviously there's the willingness like Zuckerberg just needs to go down the hall to a CFO to get access to all this capital he doesn't even need to go down the hall he can just he can just make it so he's got the voting share Sam's got to fly to Norway way and you know Saudi and and and other places and we're we're at that tier of capital. 我想深入了解驱动训练和推理方面的基本假设，因为显然有意愿，比如扎克伯格只需要走到CFO那里就能获得所有这些资本，他甚至不需要走过去，他可以直接做到，所以他拥有投票权，Sam必须飞往挪威，你知道沙特和其他地方，我们处于那个资本层级。 **7:16** I think you're making it sound way easier than it is. I I don't mean to at all. I'm I'm just saying you know Zuckerberg is hold on. If it's this easy let's let's raise 100 bill dude. We should do it. We can compete. 我认为你让它听起来比实际容易得多。我完全不是这个意思。我只是说你知道扎克伯格等等。如果这么容易，让我们筹集1000亿美元吧，伙计。我们应该这样做。我们可以竞争。 ## Scaling Laws and Diminishing Returns Debate ## 扩展定律和收益递减辩论 **7:28** But I want to make sure I understand your thinking on the underlying two sides of this one which is like your view on the diminishing return curve on just like the the return on this. I I don't think it's a diminishing return right. I think that's important to recognize. Right. Start there. 但我想确保我理解你对这两个方面的基本思考，比如你对收益递减曲线的看法，就像对此的回报。我不认为这是收益递减，对吧。我认为认识到这一点很重要。对。从这里开始。 **7:45** I want to ask about inference too but and and like the growth in token you know inference token demand but given it's a log log chart right uh scaling laws are right given there's no model architecture improvements you just throw more compute data model size at it it gets better at this pace but you're confident that that will continue 我也想问关于推理的问题，但就像token的增长，你知道推理token需求，但考虑到这是一个对数对数图表，对吧，扩展定律是正确的，假设没有模型架构改进，你只是向它投入更多的计算数据模型大小，它以这个速度变得更好，但你有信心这将继续 **7:57** I I think everything has shown that it will continue and it's continued overd wasn't some like well GP5 is not not necessarily that much bigger than 4 right and 4 is smaller than four um what what what what's what's changing is sort of paradigm of how you spend the compute 我认为一切都表明它会继续，而且它已经持续了。GPT-5不一定比GPT-4大得多，对吧，GPT-4比GPT-4小。改变的是你如何使用算力的范式 ## Why Bigger Models Aren't Always Better ## 为什么更大的模型并不总是更好 **8:08** and also like if they made a bigger model, could they even serve it? No. Right? They did 4.5 and it was terrible. No one could no one could serve it, right? It was actually like quite a bit smarter. Uh but they couldn't actually serve it at any reasonable cost and speed. 而且，如果他们制作了一个更大的模型，他们甚至能服务它吗？不能。对吧？他们做了GPT-4.5，很糟糕。没人能服务它，对吧？它实际上要聪明得多。但他们实际上无法以任何合理的成本和速度提供服务。 **8:20** Um this is why Anthropic has the same issue, right? Or I wouldn't even call it an issue, but like all of their revenue comes from for Sonnet doesn't come from 4.1 Opus, right? Which is the better model. It's bigger, but it's slow because the hardware is not caught up. Yeah. In terms of inference speed for that and so no one wants to use a slow model, right? The user experience sucks. 这就是为什么Anthropic有同样的问题，对吧？或者我甚至不称它为问题，但就像他们所有的收入都来自Sonnet，而不是来自Opus 4.1，对吧？这是更好的模型。它更大，但很慢，因为硬件还没有跟上。是的。在推理速度方面，所以没人想使用慢速模型，对吧？用户体验很糟糕。 **8:48** But as far as like if the model gets better at each scale of hardware spend, I would say all the tech giants believe it. I believe it. I think a lot of people in the financial community are like this is freaking scary. Yeah. You know, because the moment it stops, you know, you wherever you were on the rung, right? If we went from $50 billion spend to $500 billion spend, well, that $500 billion spend is never going to have ROI, right? 但就像如果模型在每个硬件支出规模上都变得更好，我会说所有科技巨头都相信它。我相信它。我认为金融界的很多人都觉得这太可怕了。是的。你知道，因为一旦它停止，你知道，无论你在哪个梯级上，对吧？如果我们从500亿美元的支出增加到5000亿美元的支出，那么那5000亿美元的支出永远不会有投资回报，对吧？ **9:12** It was one thing if 50 billion didn't have ROI, but now this 500 doesn't have ROI. It's a big problem. So anyways, one could think of it as as diminishing returns because if you when you go from $50 billion of spend to $500 billion of spend, you only move up the let's call that one tier of model capabilities in absence of, you know, major algorithmic improvements, right? 如果500亿没有投资回报是一回事，但现在这5000亿没有投资回报。这是一个大问题。所以无论如何，人们可以将其视为收益递减，因为如果你从500亿美元的支出增加到5000亿美元的支出，你只提升了让我们称之为一个层级的模型能力，在没有重大算法改进的情况下，对吧？ **9:25** Um, and so I'm I'm holding those sort of off to the side for now. But that that iterative like performance improvement in the model is is like I like I mentioned earlier, right? It's like a six-year-old versus a 13-year-old maybe, right? The the amount of work you can get a 13-year-old to do is I mean, if if you do it, right, we we we we frown upon that now in this civilization. 所以我暂时把那些放在一边。但模型中的迭代性能改进就像我之前提到的，对吧？就像6岁孩子和13岁少年的区别，也许，对吧？你能让13岁少年做的工作量，我的意思是，如果你这样做，对吧，我们现在在这个文明中不赞成这样做。 **9:41** Um, but the amount of work you can get a 13-year-old to do is actually quite valuable relative to a six-year-old. And and the same applies to like a college intern versus someone who graduated and has even one year of work experience because there's a learning curve for kids coming out of college all the time. 但相对于6岁孩子，你能让13岁少年做的工作量实际上是非常有价值的。同样适用于大学实习生与毕业并有一年工作经验的人之间的比较，因为大学毕业的孩子一直都有学习曲线。 **10:10** So there's that learning curve and I think you know while it may be incrementally the same you know an order magnitude more of compute the amount of value like if we just had if we just if you made a company full of high schoolers and you had to refresh them every six months so they didn't learn too much right and become really good it would be really hard to create a valuable company 所以有那个学习曲线，我认为你知道，虽然它可能在增量上是相同的，你知道，一个数量级更多的计算，价值量就像如果我们只是有，如果你组建一个全是高中生的公司，你必须每六个月更新他们，这样他们就不会学得太多，对吧，变得非常好，创建一个有价值的公司会非常困难 **10:22** the most you could do is like dig trenches and and like do yard work but then all the time these kids wouldn't even show up right like as as a as as a function of like how valuable of a business could you do if you had unlimited high schoolers versus if you got a business that refreshed so they didn't build knowledge versus college students versus you know 25 to 30 year olds right the the the value of that business that you can build even though incrementally it's just 5 years between each of them yeah it's drastic it's it's a drastic value change 你最多能做的就是挖沟和做园艺工作，但一直以来这些孩子甚至不会出现，对吧，就像作为一个功能，如果你有无限的高中生，你能做多有价值的生意，与如果你有一个业务，定期更新他们，这样他们就不会积累知识，与大学生，与25到30岁的人，对吧，你可以建立的那个业务的价值，尽管增量上每个之间只有5年，是的，这是巨大的，是巨大的价值变化 **10:38** where do you think we are today like which level are we at do you think depends on the domain right like for software developers like I think we're we're we're we're really pretty good right um you know and that's where where we're seeing the most value creation happen right where you see anthropic have gone from like what a billion or less of revenue to seven to eight by you know already in 你认为我们今天处于哪个水平，你认为我们处于哪个层次，取决于领域，对吧，比如对于软件开发人员，我认为我们真的相当不错，对吧，你知道，这就是我们看到最多价值创造发生的地方，对吧，你看到Anthropic从10亿或更少的收入增长到七八十亿，你知道，已经在 ## The Economics of Tokens and Serving Capacity ## Token的经济学和服务容量 **10:55** it's the fastest revenue ramp we've ever seen for anything of this and it's basically all code related right I mean like you know some of it's their own cloud code product some of it's you know cursor some of it's GitHub copilot which also offers anthropic models and has since the beginning of the year um it's you know it's windurf it's it's it's all these different avenues to access the same thing 这是我们所见过的最快的收入增长，基本上都与代码相关，对吧，我的意思是，你知道，其中一些是他们自己的云代码产品，一些是Cursor，一些是GitHub Copilot，它也提供Anthropic模型，从今年年初开始，你知道，是Windsurf，是所有这些不同的途径来访问同样的东西 **11:19** and these companies aren't all doing the same thing there there's tweaks and nuances to how they're doing things differently But it's all code and you know in that sense it's like if I had 30-year-old senior engineer um at Google and if that was like if I had infinite of those all it costed was capex for chips and the operational cost is actually quite low then you could build businesses worth insane amounts. 这些公司并不都在做同样的事情，他们做事的方式有调整和细微差别，但都是代码，你知道在这个意义上，就像如果我有谷歌的30岁高级工程师，如果那就像我有无限的那些，所有成本只是芯片的资本支出，运营成本实际上相当低，那么你可以建立价值疯狂数量的企业。 **11:39** You could have a replacement for the $2 trillion of wages that go to all the software developers in the world today. or rather you could augment them and build, you know, twice as much or five times as much or 10 times as much if you could augment them because these things don't like just run on their own, right? It's more of like a force multiplier to the existing person. 你可以替代今天全世界所有软件开发人员的2万亿美元工资。或者说你可以增强他们并构建，你知道，两倍或五倍或十倍，如果你能增强他们，因为这些东西不会自己运行，对吧？它更像是对现有人员的力量倍增器。 **11:55** So the value creation potential is is there. It's obvious if you've coded at all in your life. I mean, it even works for VBA. It's not that great for VBA. So I know a lot of people in this audience probably know VBA, but like it's it's not even that terribly bad for making macros 所以价值创造潜力就在那里。如果你一生中编过代码，这是显而易见的。我的意思是，它甚至适用于VBA。对VBA来说并不是那么好。所以我知道这个受众中的很多人可能知道VBA，但就像它对于制作宏来说甚至不是那么糟糕 **12:17** but you know, anyways, like the value creation potential there is is incredibly high. So let's capture it. How do you how do you capture? And so sort of this draws back to the OpenAI Nvidia deal because I think most people in the market don't quite get it right. They're like, "Oh, this is just like round tripping." 但你知道，无论如何，那里的价值创造潜力非常高。所以让我们抓住它。你如何抓住？这就回到了OpenAI Nvidia交易，因为我认为市场上的大多数人并不完全理解它。他们说，"哦，这就像循环往复。" **12:29** It is to some extent, right? If OpenAI builds a gigawatt of capac they they they agreed 10 gawatts of capacity, Nvidia will do hundred billion dollars of equity investment into OpenAI in the form of cash, right? And and and Nvidia gets returned capital. The first chunk of the deal in the press release is one gawatt, $10 billion, right? 在某种程度上是这样，对吧？如果OpenAI建立一千兆瓦的容量，他们同意10千兆瓦的容量，Nvidia将以现金形式向OpenAI进行1000亿美元的股权投资，对吧？Nvidia获得返还资本。新闻稿中交易的第一部分是一千兆瓦，100亿美元，对吧？ **12:40** So pretty straight line but 10 g one a g one gigawatt to build as we established earlier is like $50 billion. So Nvidia is paying 10 billion open still has to come up with other 40 somehow. Yeah. Right now what they can do is go go to the markets get a loan or get someone else to put a loan. 所以相当直接，但正如我们之前确定的，建造一千兆瓦需要大约500亿美元。所以Nvidia支付100亿，OpenAI仍然必须以某种方式拿出另外400亿。是的。现在他们能做的就是去市场获得贷款或让其他人提供贷款。 **12:52** Right. There's these infrastructure funds that are trying to get into this. Um there's all these different you know all these commercial real estate people are trying to get in this. there's some way where they'll be able to figure out other people to front the capital right and then and then come up with like a deal much like like it is Oracle but open has to do more of the work in terms of setting up the cluster the software the networking etc. 对。有这些基础设施基金试图进入这个领域。有所有这些不同的，你知道，所有这些商业房地产人士试图进入这个领域。有某种方式，他们将能够找到其他人来预支资本，对吧，然后提出像Oracle那样的交易，但OpenAI必须在设置集群、软件、网络等方面做更多的工作。 **13:14** The nice thing for Nvidia is they sell you of that 50 billion they capture maybe 35 billion of that is capex that goes directly to Nvidia. So year zero openai its partner spends $50 billion on the data center. The timing is not exactly that. They spend $50 billion on the data center. 35 goes it to Nvidia. 对Nvidia来说好的是，在那500亿中，他们卖给你的可能有350亿是直接流向Nvidia的资本支出。所以第零年，OpenAI及其合作伙伴在数据中心上花费500亿美元。时间安排不完全是这样。他们在数据中心上花费500亿美元。350亿流向Nvidia。 **13:27** Nvidia's gross margin is 75%. You know again I'm going to make it simple numbers. Let's say it's 10 and 40 because 10 billion COGS 40 billion revenue $30 billion of gross profit. um if we fix the numbers it's effectively like half their gross profit from that deal is going directly to uh to open in the form of an equity investment Nvidia的毛利率是75%。你知道我再次要用简单的数字。让我们说是10和40，因为100亿销售成本，400亿收入，300亿毛利润。如果我们固定这些数字，实际上就像他们从那笔交易中获得的毛利润的一半将以股权投资的形式直接流向OpenAI **13:45** the 25% that's COGS is staying uh on in you know Nvidia is paying for that and then they keep the other half of the gross profit on their balance sheet or do buybacks whatever they want to do with it so so Nvidia is not necessarily like they are like roundtpping some of this um but open what effectively is happening is openi gets the opportunity to pay for a big chunk of it in equity 25%的销售成本留在，你知道，Nvidia正在为此付费，然后他们将另一半毛利润保留在资产负债表上，或者进行回购，无论他们想用它做什么，所以Nvidia不一定像他们在循环往复其中一些，但OpenAI实际上发生的是OpenAI有机会用股权支付其中的一大部分 **14:01** y and nvidia's lowering their prices without lowering their prices effectively but and they're getting owners ersship of a company who very likely could just like and and but Nvidia comes out great because they're they're they're getting the capex dollars up front. Yeah. 是的，Nvidia在不降低价格的情况下有效地降低了价格，但他们获得了一家公司的所有权，这家公司很可能会，Nvidia表现出色，因为他们预先获得了资本支出。是的。 **14:18** Right. So all they're really doing is they're saying half of my money that's in this sure it does make its way to me somehow but in reality I still made half of that gross profit and the other half is is is equity in a company that may or may not be worth something. A company that may or may not be able to pay hundreds of billions of dollars of compute deals that they've signed, right? 对。所以他们真正做的就是他们说我的钱的一半在这里，当然它会以某种方式回到我这里，但实际上我仍然赚了那一半毛利润，另一半是一家公司的股权，这家公司可能值钱也可能不值钱。一家可能或可能无法支付他们签署的数百亿美元计算协议的公司，对吧？ **14:29** In which case they'd be bankrupt, right? So this is this is like the the mechanics of that deal. It's about the highest stakes like capitalism game of all time. Um, and it's so interesting to think about when it might run out. 在这种情况下，他们会破产，对吧？所以这就像那笔交易的机制。这就像有史以来最高赌注的资本主义游戏。而且思考它何时可能耗尽是如此有趣。 **14:41** You mentioned like if we hit that final point and we don't see the return like we're we're kind of toast in a big hole. But I'm also curious about the other side of ability to serve and just demand for like today's models by inference. You know, the stat I last saw is token demands doubling every two months or something crazy. 你提到，如果我们达到那个最终点，而我们看不到回报，我们就像在一个大洞里完蛋了。但我也好奇关于服务能力的另一面，以及对今天模型的推理需求。你知道，我上次看到的统计数据是token需求每两个月翻一番或类似的疯狂数据。 **14:54** Obviously, there's all these reasoning tokens that are really exciting for some of the the longer thinking models. How do you think about the growth of the pool of demand for inference tokens themselves up just even in today's models like even if we just like stop things and fix things and we'll leave that other side of the equation just for a second. 显然，对于一些更长时间思考的模型来说，所有这些推理token都非常令人兴奋。你如何看待对推理token本身的需求池的增长，即使是在今天的模型中，即使我们只是停止事情并固定事情，我们暂时把方程的另一边放在一边。 ## Rate Limits and the Adoption Curve Problem ## 速率限制和采用曲线问题 **15:11** What's your model for thinking about that today? What most interests you in the in the growth of just broad? So so the thing I like to call it is tokconomics and I I stumbled upon the word actually it's like a crypto kill off crypto finally one once and for all. 你今天对此的思考模型是什么？在广泛增长中最让你感兴趣的是什么？所以我喜欢称之为tokonomics（token经济学），我实际上偶然发现了这个词，它就像一个加密货币，最终彻底杀死加密货币。 **15:23** So I'm I'm trying to make tokconomics uh SEO direct to you know us talking about tokconomics and then hopefully you talking about tokconomics hopefully like everyone using say tokconomics 20 more times it's the economics of the tokens right how much compute is being spent how much is the gross profit what's the value being created by these tokens 所以我试图让tokonomics直接SEO到你知道我们谈论tokonomics，然后希望你谈论tokonomics，希望每个人都多说20次tokonomics，这是token的经济学，对吧，花费了多少计算，毛利润是多少，这些token创造了什么价值 **15:41** that's that's the end of the day what's what's relevant here right Nvidia keeps saying AI factory which produces intelligence that intelligence has value let's say you have a gigawatt of capacity what can I serve well I could serve a thousand times times of a model that's really shitty. 这就是归根结底什么是相关的，对吧，Nvidia一直在说生产智能的AI工厂，那种智能有价值，让我们说你有一千兆瓦的容量，我能服务什么，我可以服务一千倍的非常糟糕的模型。 **15:59** I could serve, you know, amount, right? I could serve one one times amount that's of a model that's good. And I could serve like 0.1 times of a model that's amazing. Now, multiply that by whatever factor of like how many users, what's the number of tokens outputed, but you know, I could do X number of tokens, X time 100, X times a million tokens, right? 我可以服务，你知道，数量，对吧？我可以服务一倍数量的好模型。我可以服务0.1倍的令人惊叹的模型。现在，乘以任何因素，比如有多少用户，输出的token数量是多少，但你知道，我可以做X个token，X乘以100，X乘以一百万个token，对吧？ **16:16** Depending on the model quality. And so this is sort of where, you know, the whole GPD5 thing comes around, right? is open had had a had a challenging, you know, thing, right? They're like, "Hey, we have a couple gigawatts of capacity effectively, right? By the end of this year, roughly a couple gigawatts of capacity too. 取决于模型质量。所以这就是，你知道，整个GPT-5的事情出现的地方，对吧？OpenAI有一个具有挑战性的，你知道，事情，对吧？他们说，"嘿，到今年年底，我们实际上有几千兆瓦的容量，对吧？大约几千兆瓦的容量。 **16:35** Um, more or less a little bit less, but you know, right now, but you know, it's how how do they maximize their serving capacity with this?" Um one one avenue is we continue to serve big models and we make bigger models and the tokens are more expensive but this log scale is really challenging 或多或少少一点，但你知道，现在，但你知道，他们如何最大化他们的服务容量？一个途径是我们继续服务大模型，我们制作更大的模型，token更昂贵，但这个对数尺度真的很具挑战性 **16:50** because yes the value is an order magnitude you know value is way more but the cost is way more and then the the real whammy is the user experience is way worse right if I serve a massive massive model it's slow and users are fickle and you need that you need the response to be way faster than they can be 因为是的，价值是一个数量级，你知道价值要高得多，但成本也要高得多，然后真正的打击是用户体验要差得多，对吧，如果我服务一个巨大的模型，它很慢，用户是善变的，你需要响应比它们能够的快得多 **17:11** hard to calibrate yeah yeah so so there's this user experience challenge but really in the end it's like you know for a given model level I think there's a saturation point of how many how much demand of intelligence there is right you can only have such large child army right of of like people digging trenches or like con 2012 很难校准，是的，是的，所以有这个用户体验挑战，但实际上最终就像你知道，对于给定的模型级别，我认为有一个饱和点，有多少智能需求，对吧，你只能有这么大的儿童军队，对吧，就像挖沟的人或像2012年的骗局 **17:28** whatever it is like this is very cancelable but you know um but you could have a much larger army of you know or business of like the larger level of intelligence right and so when you think about hey what what could I have done with GPG3 GP GPD3 even if we paused there paused the model capabilities 无论它是什么，这是非常可取消的，但你知道，但你可以有一个更大的军队，你知道，或者更大级别的智能业务，对吧，所以当你想到嘿，我本可以用GPT-3做什么，即使我们在那里暂停，暂停模型能力 **17:44** right you know obviously the cost to serve a model quality of GPD3 has tanked 99% or more yeah it's like 2,000 times cheaper now it's so much cheaper now for and then GPD4 same thing right people were freaking out about Deepseek because it's like five six00 times cheaper 对吧，你知道，显然服务GPT-3模型质量的成本已经下降了99%或更多，是的，它现在便宜了2000倍，现在便宜得多，然后GPT-4也是一样的，对吧，人们对Deepseek感到恐慌，因为它便宜了五六百倍 **17:56** GPT OSS came out and that's even cheaper than that right and it's the same again like for roughly the same quality actually I would argue the GPT OS open source model is actually a little bit better than GPD4 OG um because it can do tool calling. GPT OSS出来了，那更便宜，对吧，再次是一样的，对于大致相同的质量，实际上我会说GPT OS开源模型实际上比原始GPT-4好一点，因为它可以进行工具调用。 **18:08** And anyways um the cost of these things tanks rapidly with algorithmic improvement, right? Not necessarily model getting bigger. Um and as these algorithms get better, you can but but at X level of intelligence, you can only serve so much demand. 无论如何，这些东西的成本随着算法改进而迅速下降，对吧？不一定是模型变大。随着这些算法变得更好，你可以，但在X级别的智能上，你只能服务这么多需求。 ## The Tokenomics of AI ## AI的Token经济学 **18:24** And then the the flip side is you know what that demand it takes time for people to realize how to use it. So when GPD3 launched, no one cared. When GBD3.5 launched, it was like still most people didn't care. Chat GPT launched with GBD3.5 people cared a little bit. 另一方面是，你知道那种需求，人们需要时间来意识到如何使用它。所以当GPT-3推出时，没人在意。当GPT-3.5推出时，大多数人仍然不在意。ChatGPT用GPT-3.5推出，人们稍微在意了一点。 **18:41** uh GPD4 launched on chat GPD then people cared a lot but a model tier of GPT 3.5 or three still can be very useful in a lot of world a lot of the world now it's not useful for like a lot of use cases right like for coding it was terrible right for for copyrightiting it's okay right like there's there's but there's some level of use case and it happens to four but it takes time for that adoption to happen GPT-4在ChatGPT上推出，然后人们非常在意，但GPT-3.5或GPT-3的模型层级在世界上很多地方仍然非常有用，现在它对很多用例不有用，对吧，比如对于编码它很糟糕，对吧，对于文案写作它还可以，对吧，就像有一些用例级别，它发生在GPT-4上，但采用需要时间 **19:06** um and so you've kind of got this challenge of like if I pause on a model capability then I end up like taking way too long for adoption and also like how can Can I get people to adopt it if I don't let people use it? And so, so open had this tremendous problem with GPD 40, right? 4 and then 4 turbo was smaller than 4 and 4 was smaller than 4 turbo. 所以你有这个挑战，如果我在模型能力上暂停，那么我最终会花太长时间进行采用，而且如果我不让人们使用它，我如何让人们采用它？所以，OpenAI在GPT-4上有这个巨大的问题，对吧？GPT-4，然后GPT-4 turbo比GPT-4小，GPT-4比GPT-4 turbo小。 **19:18** What open basically did was they made the model as much smaller as possible while keeping roughly the same quality or slightly better, right? So 4 to 4 turbo was like the model was less than half the size and four turbo to 40 like 40's cost is way lower than four. OpenAI基本上做的是他们尽可能使模型变小，同时保持大致相同的质量或稍好一点，对吧？所以GPT-4到GPT-4 turbo就像模型小于一半的大小，GPT-4 turbo到GPT-4o，GPT-4o的成本比GPT-4低得多。 **19:30** Um and they just kept shrinking the cost. Now five, what could they have done? They could have gone, "Oh, we'll go big step." They actually tried that with 4.5. They they screwed up some things cuz it was really hard to get, you know, 100,000 GPUs to work properly. There's challenges there. 他们只是不断降低成本。现在GPT-5，他们本可以做什么？他们本可以说，"哦，我们会迈出一大步。"他们实际上用GPT-4.5尝试了那个。他们搞砸了一些事情，因为让10万个GPU正常工作真的很难。那里有挑战。 **19:42** Also, they hadn't figured out the whole reinforcement learning paradigm at that time. So, the C, so they ran out of, you know, it's the the scaling laws are like it's a chart of quality versus compute, but that compute breaks down into how much bigger do I make the model, how much more data do I put in the model, and if the internet only has so many tokens, you're kind of screwed, right? 此外，他们当时还没有弄清楚整个强化学习范式。所以，他们用完了，你知道，扩展定律就像是质量与计算的图表，但那个计算分解为我把模型做多大，我在模型中放入多少数据，如果互联网只有这么多token，你就有点完蛋了，对吧？ **20:07** So it took you know there was there was potentially a cliff until reinforcement learning happened you know where you can generate data and train the model to be better without the internet having that data. Um but anyway so so they kind of had this problem of you have x amount of compute you can service your users but hey today um if people want to use my API I rate limit them because I can't actually serve them all. 所以它花了，你知道，可能有一个悬崖，直到强化学习发生，你知道，你可以在互联网没有那些数据的情况下生成数据并训练模型变得更好。但无论如何，他们有这个问题，你有X数量的计算，你可以为用户提供服务，但嘿，今天，如果人们想使用我的API，我会限制他们的速率，因为我实际上无法为他们所有人提供服务。 **20:27** Yeah. Oh if I want to use um you know I have to I have to rate limit the people who have chat GPT free pro and max whatever the whatever the $2 $200. There's like different rate limits. You can only do deep research so much. Um I have multiple Chad GPT accounts because I, you know, use deep research. 是的。哦，如果我想使用，你知道，我必须限制使用ChatGPT免费、专业和最大版本的人的速率，无论是2美元还是200美元。有不同的速率限制。你只能做这么多深度研究。我有多个ChatGPT账户，因为我，你知道，使用深度研究。 **20:38** It's like you you kick off a bunch, you read it, and you're like, "Wow, I learned a ton. Move on." Right? So you have this challenge of like you can't actually serve your user base enough. So how are they ever going to move up this adoption curve? 就像你启动一堆，你阅读它，你说，"哇，我学到了很多。继续前进。"对吧？所以你有这个挑战，就像你实际上无法为你的用户群提供足够的服务。那么他们如何在这个采用曲线上上升？ **20:50** So then as OpenAI, what's your choice? Do you make go from 40 to 5? Do you make the model way bigger and not be able to serve anyone? And plus, because you can't serve anyone and it's slow to serve, the adoption curve doesn't really get going. 那么作为OpenAI，你的选择是什么？你是从GPT-4到GPT-5吗？你是否让模型变得更大而无法为任何人提供服务？而且，因为你无法为任何人提供服务，而且服务很慢，采用曲线并没有真正开始。 **21:00** Um, or do you make the model the same size, which is what they did for GBD5. It's basically the same size as 40 and and roughly the same cost. That's actually a little bit cheaper potentially, and then you just serve way more users. Um, and get everyone up the adoption curve more. 或者你是否让模型保持相同的大小，这就是他们对GPT-5所做的。它基本上与GPT-4o大小相同，成本大致相同。实际上可能稍微便宜一点，然后你只是为更多用户提供服务。并让每个人在采用曲线上上升更多。 **21:17** And then you can instead of putting them on a bigger model, you put them on models that do thinking that can do uh, you know, so if you've used GP5 thinking or GP5 Pro, there's more intelligence there. Um, and so this is the whole conundrum they have and this is where the whole tokconomics thing comes into play. 然后你可以不是把他们放在更大的模型上，而是把他们放在可以思考的模型上，可以做，你知道，所以如果你使用过GPT-5 thinking或GPT-5 Pro，那里有更多的智能。所以这就是他们面临的整个难题，这就是整个tokonomics事情发挥作用的地方。 **21:30** the question you had, I wanted to level set it, right, which is how do you serve these users? The demand is growing so much. I'm not doubling my hardware every two months, right? Right. Yes, this capex is crazy, but I'm not doubling my hardware every two months, but I'm doubling my tokens every two months. 你提出的问题，我想设定一个基准，对吧，就是你如何为这些用户提供服务？需求增长如此之多。我不会每两个月加倍我的硬件，对吧？对。是的，这个资本支出很疯狂，但我不会每两个月加倍我的硬件，但我每两个月加倍我的token。 **21:46** So, so there has to be enough of a cost decrease and and there is, right, with with at a given level of intelligence. If you could like in if you could snap your fingers and change change a dial somehow that would most unlock and unleash more development, is it just is it just inference latency? because then we could do bigger models and serve them much faster in a way that consumers would enjoy. 所以，必须有足够的成本下降，而且有，对吧，在给定的智能水平上。如果你可以像在如果你可以打个响指并以某种方式改变一个刻度盘，那将最大程度地解锁和释放更多的开发，它只是推理延迟吗？因为那样我们可以做更大的模型，并以消费者喜欢的方式更快地为他们提供服务。 ## Inference Latency vs Cost Trade-offs ## 推理延迟与成本权衡 **22:04** Is that the main like bottleneck to be attacked? Inference is like always it's it's it's a curve again, right? Like all of these things are curves and it's a trade-off, right? Everything in engineering is a trade-off. So So you have inference latency versus cost on any given hardware. 这是需要攻克的主要瓶颈吗？推理总是，它是一条曲线，对吧？就像所有这些东西都是曲线，这是一个权衡，对吧？工程中的一切都是权衡。所以你在任何给定的硬件上有推理延迟与成本。 **22:16** Um GPUs can do lower latency to a certain extent, but then the cost is way higher or you can do really really high throughput and the cost is way uh lower, right? and and you know the company just kind of yolo they set the dial where they think it makes the most sense GPU可以在一定程度上做到更低的延迟，但成本要高得多，或者你可以做真正非常高的吞吐量，成本要低得多，对吧？你知道公司只是有点冒险，他们设定他们认为最有意义的刻度盘 **22:28** and there's other types of hardware which kind of aim for their curve to be at a different spot. Maybe the GPU curve is here uh but latency you know over here you know you're in very diminishing returns and so actually someone made a little curve right here. 还有其他类型的硬件，旨在让他们的曲线处于不同的位置。也许GPU曲线在这里，但延迟你知道在这里，你知道你处于非常递减的回报中，所以实际上有人在这里做了一条小曲线。 **22:40** It's like okay maybe that's a useful point but actually the market cares about this point. So anyways there's there's there's a curve of like who cares about latency. I think if I could just press a magic button. Yeah that's is it is it capacity? Is it latency? What is it? 就像好的，也许那是一个有用的点，但实际上市场关心这一点。所以无论如何，有一个像谁关心延迟的曲线。我认为如果我能按一个魔法按钮。是的，那是容量吗？是延迟吗？是什么？ **22:53** I think that I think that's a that's a tremendous like question. I'd probably still say capacity/cost is more important than latency really. I think existing levels of latency are fast enough for a lot. Um now now if the if the latency was 10x lower for GBD5 then they could have made a model that was 10x bigger and served it at this qual served it at this speed. 我认为那是一个巨大的问题。我可能仍然会说容量/成本比延迟更重要。我认为现有的延迟水平对很多人来说已经足够快了。现在，如果GPT-5的延迟降低10倍，那么他们本可以制作一个大10倍的模型，并以这种质量以这种速度提供服务。 **23:16** Yeah, that's what I'm wondering about. But but then you would have the same capacity issue, right? Um, so I guess like if I was if you could have your cake and eat it, which is all the capacity in the world and the lowest latency in the world. Yeah. Well, then you would just make the best you'd make the models way better, right? 是的，这就是我想知道的。但那样你会有同样的容量问题，对吧？所以我想如果我是如果你能鱼与熊掌兼得，也就是世界上所有的容量和世界上最低的延迟。是的。那么你只会制作最好的，你会让模型好得多，对吧？ **23:27** Like I think I think it's the physical realities of like if I'm at OpenAI, what do I choose to do? Um, do I invest more in the model that people can use or do I invest more in the fast? Do I invest a lot in the model that most people, you know, won't use because it's expensive first of all and even those that can afford it will often go back to the regular one, right? 就像我认为这是物理现实，如果我在OpenAI，我选择做什么？我是投资更多在人们可以使用的模型上，还是投资更多在快速上？我是否在大多数人不会使用的模型上投入很多，因为首先它很昂贵，即使那些能负担得起的人也经常会回到常规模型，对吧？ **23:45** Um, I have access to Cloud 4.1 Opus. I still use Sonnet more way more just because it's a better experience, right? It's dumber. It's it's it's objectively dumber, but it's slow. Yeah. And like I don't I don't know. Like my time's worth something, right? 我可以访问Claude 4.1 Opus。我仍然更多地使用Sonnet，只是因为它的体验更好，对吧？它更笨。它客观上更笨，但它很慢。是的。我不知道。我的时间值点钱，对吧？ **23:58** I think Openi wouldn't have been afraid to like make a model way way way bigger in a terrible user experience. Yeah. And as a result, we're just going to probably have to wait a little bit longer to see what the bigger models are in practice in a way that to to see what consumers actually do with them because it's just going to be too hard. 我认为OpenAI不会害怕在糟糕的用户体验中制作一个更大的模型。是的。结果，我们可能只需要等待更长一点时间才能看到更大的模型在实践中是什么，以一种方式来看看消费者实际上用它们做什么，因为这太难了。 ## Over-Parameterization and Model Learning ## 过度参数化和模型学习 **24:10** It's not necessarily even bigger, right? Like there's this whole concept of um overparameterization i.e. if you just throw more parameters in a neural network and even when humans I'll equate it to humans right when you had a vocab test or you had some test you memorized before you understood and it wasn't until you did multiple repetitions and in different forms that you actually understood the content rather than just memorized. 甚至不一定更大，对吧？就像有这整个过度参数化的概念，也就是说，如果你只是在神经网络中投入更多参数，即使当人类时，我会把它等同于人类，对吧，当你有词汇测试或你有一些测试时，你在理解之前记住了，直到你做了多次重复并以不同形式，你才真正理解了内容而不仅仅是记住。 **24:36** Um it takes it takes cycles. Um and when you when you do an LLM it's the same thing right? If you throw some data at it, it will memorize it before it generalizes. It's this concept called groing, right? You grocked a subject, i.e. it's like the aha moment, trick of understanding. 它需要周期。当你做LLM时，情况是一样的，对吧？如果你向它投入一些数据，它会在泛化之前记住它。这就是所谓的grokking的概念，对吧？你领悟了一个主题，也就是说，这就像顿悟的时刻，理解的诀窍。 **24:53** Yeah. Yeah. And the models do the same thing. They memorize it up until then they understand it at some point. And if you make the model bigger and bigger and bigger without the data changing, you just memorize everything. And actually, it starts to get worse again because it never had the opportunity to generalize because the model was so big and there's so many weights and there's so much capacity for information. 是的。是的。模型也做同样的事情。它们记住它，直到它们在某个时候理解它。如果你让模型越来越大而数据没有改变，你只是记住一切。实际上，它又开始变得更糟，因为它从来没有机会泛化，因为模型太大了，有这么多权重，有这么多信息容量。 **25:06** You know the challenge today is not necessarily make the model bigger. The challenge is how do I generate and create data that is in useful domains so that the model gets better at them. Nowhere on the internet to show you how to fly through a spreadsheet you know using only your uh mouse or not using your mouse using only your keyboard and all these like you know functions and all these things right like that's that's a repetition that's that's bars but there's no data on the internet about this. 你知道今天的挑战不一定是让模型变大。挑战是如何在有用的领域生成和创建数据，以便模型在这些领域变得更好。互联网上没有地方向你展示如何在电子表格中飞行，你知道只使用你的鼠标或不使用你的鼠标只使用你的键盘和所有这些你知道的功能和所有这些东西，对吧，就像那是重复，那是标准，但互联网上没有关于此的数据。 **25:33** So, how do you teach a model that? It's not going to learn it from reading the internet over and over and over again, which you and I could never do. And so, it hasn't a level of intelligence that we can't do. We can't read the whole internet, but it can't do basic stuff, which is like play with a spreadsheet. 那么，你如何教模型那个？它不会通过一遍又一遍地阅读互联网来学习，而你和我永远做不到。所以，它没有我们做不到的智能水平。我们不能阅读整个互联网，但它不能做基本的事情，就像玩电子表格。 **25:52** Um, so, so, so how do you get it to learn these things? And so, that's that's where this whole reinforcement learning paradigm kind of happened, which is giving it environments, specific environments to learn it and then fold back in. Right. Exactly. And that's that's where there's sort of a a challenge in terms of building those environments in terms and so there's like 40 startups now in the bay doing these environments 那么，你如何让它学习这些东西？所以，这就是整个强化学习范式发生的地方，也就是给它环境，特定的环境来学习它，然后折回去。对。确切地。这就是在建立这些环境方面存在一种挑战的地方，所以现在湾区有大约40家初创公司在做这些环境 **26:12** and you know questionable whether or not they'll any of them will make it or what will happen but like there's 40 and then these companies are also making their own environments but these environments can be anything and everything. Give me an example just like of one of the startups or something just to get 你知道他们中的任何一个是否会成功或会发生什么是值得怀疑的，但就像有40家，然后这些公司也在制作自己的环境，但这些环境可以是任何东西和一切。给我一个例子，就像其中一家初创公司或什么，只是为了得到 ## Building Environments for AI Training ## 为AI训练构建环境 **26:25** these startups are like like they're they're just making environments for open anthropic and others right so it's like as simple as like here is a fake Amazon right because Amazon terms of service ban chat models and all these things but here's a fake Amazon full of items um figure out how to click around and purchase items right uh figure out how to compare the two items and pick you know 这些初创公司就像，他们只是为OpenAI、Anthropic和其他公司制作环境，对吧，所以就像这样简单，这是一个假的亚马逊，对吧，因为亚马逊的服务条款禁止聊天模型和所有这些东西，但这是一个充满物品的假亚马逊，弄清楚如何点击并购买物品，对吧，弄清楚如何比较两个物品并选择，你知道 **26:41** I I've generated a list of deodorants three of them are fake one of them's real one of them is not the one I want here's the prompt figure how to buy it and if and and you know it tries many things and you know vary the prompt and all these things but eventually you know it's bought the right deodorant and you've succeeded and you fold it back in. 我生成了一份除臭剂清单，其中三个是假的，一个是真的，其中一个不是我想要的，这是提示，弄清楚如何购买它，如果，你知道它尝试了很多事情，你知道改变提示和所有这些事情，但最终你知道它买了正确的除臭剂，你成功了，你把它折回去。 **26:53** That's a simple thing. Or it could be, hey, clean this data, right? Here's this table. Ton ton of dirty data in there. Oh, there's like colons and stuff. The there's an address in one column. You know, I'm going to, you know, how do how do I separate out the columns? So, the address is like, you know, it's it's street address, city, zip code, and it'll try a bunch of stuff, but like, hey, maybe it can't do that yet. 这是一件简单的事情。或者可能是，嘿，清理这些数据，对吧？这是这张表。里面有大量脏数据。哦，有冒号之类的东西。一列中有一个地址。你知道，我要，你知道，我如何分开列？所以，地址就像，你知道，它是街道地址、城市、邮政编码，它会尝试一堆东西，但就像，嘿，也许它还做不到。 **27:12** So, really, you just drop it like you teach you give it, you know, iterative like here's here's addresses, here's different formats, and you slowly iteratively teach it. So, there's all this like challenge. So that's one that's another example. Another example is like you're in a game and like whether it's a tic-tac-toe or Call of Duty or you know a math puzzle, whatever the game is. 所以，实际上，你只是放弃它，就像你教它，你给它，你知道，迭代地就像这是地址，这是不同的格式，你慢慢迭代地教它。所以，有所有这些挑战。所以那是一个，那是另一个例子。另一个例子是你在一个游戏中，就像无论是井字游戏还是使命召唤，或者你知道数学谜题，无论游戏是什么。 **27:32** And that's what a lot of these environments initially have been is like math puzzles. It's like do this math puzzle. Oh well I can't do this one because it's too hard. Here's an easier one. Oh, okay. I can I can spin on this one. Okay, I'm better enough. Okay, now I can learn this one. 这就是很多这些环境最初的样子，就像数学谜题。就像做这个数学谜题。哦，我不能做这个，因为它太难了。这是一个更容易的。哦，好的。我可以在这个上旋转。好的，我足够好了。好的，现在我可以学习这个。 **27:44** Right? and and and it has iteratively stepped through those to where you know basically from this you know Q4 of last year to Q2 of this year these things hill climbed up math puzzles like crazy. Yeah. Um and a lot of that was not hey I just know the math. A lot of that was here's how I use Python to uh write something that does the math for me. 对吧？它迭代地通过这些步骤，你知道基本上从去年第四季度到今年第二季度，这些东西在数学谜题上疯狂地爬山。是的。其中很多不是嘿我只是知道数学。其中很多是这是我如何使用Python来编写为我做数学的东西。 **28:07** Um and now these things are actually quite good at math. But you know so so it's like these the environments can be super varied. Um and it doesn't need to be something that's like clear-cut and dry. It can be here's a medical case, what's wrong with it? And then you have another model say, well, here's here's your instructions on how you would grade the result of a case. 现在这些东西实际上非常擅长数学。但你知道，所以就像这些环境可以非常多样化。它不需要是像明确和干燥的东西。它可以是这是一个医疗案例，有什么问题？然后你有另一个模型说，嗯，这是你如何评分案例结果的说明。 **28:19** What looks like they didn't even try this or didn't even look up this. Okay, you did that wrong. And you know, you can you can feed these models into so these environments can be very very complicated. So building those out is is a challenge, right? It was one thing to say, I'm taking all the internet data. I'm going to filter it some. I'm going to throw it to the model, right? 看起来他们甚至没有尝试这个或甚至没有查找这个。好的，你做错了。你知道，你可以将这些模型输入，所以这些环境可以非常非常复杂。所以构建这些是一个挑战，对吧？说，我正在获取所有互联网数据是一回事。我要过滤一些。我要把它扔给模型，对吧？ **28:37** There's tons of engineering challenges there for sure. There's a different set of engineering challenges that take time to build out in those two like in pure raw internet pre-training world and in this new like environments world like what inning are we in in each of those would you say like how far into the potential benefits have we have we eaten 那里肯定有大量的工程挑战。有一组不同的工程挑战需要时间来构建，在这两个方面，就像在纯粹的原始互联网预训练世界和这个新的环境世界中，你会说我们在每个方面都处于哪个阶段，我们在潜在利益中吃了多远 **28:55** this is where like the whole like oh well then you know Dylan what you're saying is you never need to make models bigger again right because you've already run out of data and until you figure out how to generate tons and tons of data that's great but actually we haven't right like you know we've seen another angle where it's mostly just been pre-training scaling, right, is is V3 and Banana Nano, right? 这就是整个，哦，那么你知道Dylan你说的是你永远不需要再让模型变大了，对吧，因为你已经用完了数据，直到你弄清楚如何生成大量大量的数据，那很好，但实际上我们还没有，对吧，就像你知道我们看到了另一个角度，主要只是预训练扩展，对吧，是V3和Banana Nano，对吧？ **29:14** These Google image and video models um and Genie and like all these Google uh image and video models and that's that's purely like scaling on on multimodality, right? The models still aren't that great at video and audio and images. They're fine, but they could be a lot better. 这些谷歌图像和视频模型，Genie和所有这些谷歌图像和视频模型，那纯粹是在多模态上扩展，对吧？模型在视频、音频和图像方面仍然不是那么好。它们还可以，但可能会好得多。 **29:31** Um so there's like angles of scaling there, right? Cuz when I said we've run out of internet, we've run out of the text, tons of video and image and audio, right? We just it's just so expensive. So, you know, like we we didn't get to that. So, like maybe late innings on text, mid innings on pre-training. 所以那里有扩展的角度，对吧？因为当我说我们用完了互联网时，我们用完了文本，大量的视频、图像和音频，对吧？我们只是太贵了。所以，你知道，就像我们没有达到那个。所以，就像也许在文本上是后期阶段，在预训练上是中期阶段。 **29:43** I think we're early on text. Yeah, we're quite early. And then the other angle is just because you've used the text doesn't mean you can't learn faster, right? You take a class, you give them all a book, you tell them to read it once, and you test them all. It's like, well, one kid's going to get 100 and one kid's going to get a 40, right? It's just the reality of life. 我认为我们在文本上是早期的。是的，我们相当早期。然后另一个角度是，仅仅因为你使用了文本并不意味着你不能学得更快，对吧？你上课，你给他们所有人一本书，你告诉他们读一次，你测试他们所有人。就像，嗯，一个孩子会得100分，一个孩子会得40分，对吧？这就是生活的现实。 **30:06** And maybe maybe if you if you said if you read the book out loud to them, the kid who got a 100 might get a 30 and the kid who got a 40 might have got a 60, right? So there's like these different parameters and and when we talk about model architecture, the same thing happens there. 也许如果你，如果你大声给他们读书，得100分的孩子可能得30分，得40分的孩子可能得60分，对吧？所以有这些不同的参数，当我们谈论模型架构时，同样的事情发生在那里。 **30:17** So it's not like you stop training new models. It's not like you don't have algorithmic improvements or smarter kids, right? You know, it's not like pre-training is done. Yeah. In fact, it's it's the base of everything. So you want to keep having gains because any gains on pre-training, right? I.e. the model learns a little faster or the model's a little bit smaller for the same quality Yeah. feeds into the next stage which is this whole post-training side 所以不像你停止训练新模型。不像你没有算法改进或更聪明的孩子，对吧？你知道，不像预训练完成了。是的。事实上，它是一切的基础。所以你想继续获得收益，因为预训练的任何收益，对吧？也就是说，模型学得更快一点，或者模型对于相同的质量更小一点，是的。进入下一个阶段，这就是整个后训练方面 **30:35** um which will subsume the majority of the compute at some point and inning wise is are we in the second inning of that like how is I think we've like thrown the first ball wow cuz like you know like think about how we so many environments 这将在某个时候包含大部分计算，而阶段明智的是，我们是在第二阶段吗，就像我们如何，我认为我们已经扔了第一个球，哇，因为就像你知道，想想我们有多少环境 **30:48** I think I think my favorite thing my brother just had a baby this baby will literally stick his hand in his mouth and I'm like you I thought about it and then and then it's like wait he's like he's like calibrating the senses on his fingers by sticking his hand in his mouth cuz his tongue is the most sensitive thing. 我认为我最喜欢的事情是我兄弟刚生了一个孩子，这个孩子会真的把他的手放进嘴里，我就像你，我想了想，然后就像等等，他就像他在通过把手放进嘴里来校准手指上的感觉，因为他的舌头是最敏感的东西。 **31:07** He doesn't know he's doing it, but like that's how he's calibrating. He's like, "Oh, that's me. Oh, I can touch and feel, right?" It's like, how does the model learn these sorts of things, right? It's like you just have to try stuff and fail. And we're so so early in like, you know, think about how much we see throughout our life and how much of that information we throw away, right? 他不知道他在做什么，但就像那是他如何校准的。他就像，"哦，那是我。哦，我可以触摸和感觉，对吧？"就像，模型如何学习这些东西，对吧？就像你只需要尝试东西并失败。我们在这方面还很早期，你知道，想想我们一生中看到多少东西，我们扔掉了多少信息，对吧？ **31:29** We throw all of this information away. I don't remember anything about like, you know, like, do I remember what I had for lunch yesterday? No. But if it was amazing or bad, I would have remembered that. Oh, I don't like this or I like this. Right? It's sort of like, you know, there's all this information we throw away and these models, these environments. Yeah, we're generating tons of data and throwing most of away and training the model, but it's like infantessimal compared to what humans have done. 我们扔掉所有这些信息。我不记得任何关于，你知道，就像，我记得我昨天午餐吃了什么吗？不。但如果它很棒或很糟糕，我会记得那个。哦，我不喜欢这个或我喜欢这个。对吧？有点像，你知道，有所有这些我们扔掉的信息和这些模型，这些环境。是的，我们正在生成大量数据并扔掉大部分并训练模型，但与人类所做的相比，这就像微不足道。 **31:47** And so, I think there's so many environments you can put the model in. There's people who even think you don't get to the magical AGI until you embody it, i.e. you put the model in something that can interact in the real world, right, as a in a robot. I think like Elon and XAI like they're they're a bit more along that angle of like they think embodiment is required to get to artificial general intelligence 所以，我认为你可以将模型放入这么多环境中。甚至有人认为你不会达到神奇的AGI，直到你具体化它，也就是说，你把模型放在可以在现实世界中交互的东西中，对吧，作为机器人。我认为像马斯克和XAI，他们更倾向于那个角度，他们认为具体化是达到人工通用智能所必需的 **32:04** because you need the model to be able to say like pick this up or like oh wow this is like a rotating thingy which you could never get from like just watching a video about it. You you wouldn't get the concepts of it even. Yeah. Um and so I think we're so early in the reinforcement learning because that's what humans are. We're reinforcement learners 因为你需要模型能够说像拿起这个或像哦哇这就像一个旋转的东西，你永远无法从只是观看关于它的视频中得到。你甚至不会得到它的概念。是的。所以我认为我们在强化学习方面还很早期，因为那是人类是什么。我们是强化学习者 ## AI in Everyday Life ## 日常生活中的AI **32:22** and and the so what of let's say we fast forwarded we're in the seventh inning of that or something like this. What do you think the way that the average person will most feel that difference in terms of the utility of the model? 那么如果我们快进，我们处于那个或类似的第七局。你认为普通人最能感受到模型效用差异的方式是什么？ **32:33** It'll be very different like motus of using it right. It's one thing to like ask for information or ask it to organize information versus it just doing things. Those those 12-year-olds, you need to really direct them how to dig a hole cuz a lot of them haven't dug a hole. But you're talking about order me this vitamin and just like it's just done, right? 使用它的方式会非常不同，对吧。请求信息或要求它组织信息与它只是做事情是一回事。那些12岁的孩子，你需要真正指导他们如何挖洞，因为他们中的很多人没有挖过洞。但你说的是给我订购这种维生素，就像它刚完成，对吧？ **32:50** And and and we're actually like not too far away from that. I think if you try and research electric toothbrushes, like this is something cuz you know your electric toothbrush, I lose it. I leave it at a hotel all the time. And I've been obsessive about this. Like in 2021, I I like made a spreadsheet of all the electric toothbrushes cuz based on how many IC's were in each one of them, right? 我们实际上离那不太远。我认为如果你尝试研究电动牙刷，就像这是因为你知道你的电动牙刷，我弄丢了它。我一直把它留在酒店。我对此很痴迷。就像在2021年，我制作了一个所有电动牙刷的电子表格，因为基于每个牙刷中有多少IC，对吧？ **33:09** Like this one has a Bluetooth IC. Why? I don't know. This one has a display IC. Like it has a color display IC. Like what's going on, right? Like so I made a spreadsheet of all this. And so like I don't know. It's like this weird like little thing that I do. I've been finding like every every you know how I research which toothbrush I want to buy now 就像这个有蓝牙IC。为什么？我不知道。这个有显示IC。就像它有彩色显示IC。发生了什么，对吧？所以我制作了所有这些的电子表格。所以我不知道。这就像我做的这个奇怪的小事情。我一直在寻找，就像每一个，你知道我现在如何研究我想买哪个牙刷 **33:21** I bought a oral B IO like series 9 or whatever right like whatever it's like but it's like comparing them like these models now can like actually like figure out exactly what you want and more than 10% of Etsy's traffic is straight from GPT wow Amazon blocks GPT but like otherwise it would be really high 我买了一个Oral-B iO系列9或什么的，对吧，就像无论它是什么，但就像比较它们，这些模型现在可以实际上弄清楚你到底想要什么，Etsy超过10%的流量直接来自GPT，哇，亚马逊封锁了GPT，但就像否则它会非常高 **33:40** people make purchasing decisions through GPTs they just don't make the purchase open's head of applications or co of applications was at Shopify and created the shopping agent, right? This is is very clear. This is how they monetize. 人们通过GPT做出购买决定，他们只是不进行购买，OpenAI的应用主管或应用联合主管曾在Shopify工作并创建了购物代理，对吧？这非常清楚。这就是他们如何赚钱。 **33:53** The models are going to purchase for you, right? They're going to do actions for you and the model and then the company that does those actions for you, the model that will be able to take some sort of take rate, right? Even if it's like 0.1%, even if it's 1%, it's 2%. It'll be like a credit card transaction. 模型将为你购买，对吧？它们将为你做事情，然后为你做这些事情的公司，能够采取某种抽成的模型，对吧？即使是0.1%，即使是1%，是2%。它会像信用卡交易一样。 **34:04** Visa is the most amazing business in the world because of this, right? And and chat could be that, too. If I'm making my decisions on purchasing all sorts of things, I mean, I already almost outsource like what am I going to eat to like the front page recommendation of like Uber Eats sometimes or I already outsource a lot of decisions. Visa因此是世界上最了不起的企业，对吧？ChatGPT也可以是那样。如果我在购买各种东西时做决定，我的意思是，我已经几乎外包了比如我要吃什么给Uber Eats的首页推荐，或者我已经外包了很多决定。 **34:21** It's not too much further till I've like completely outsourced a decision and a purchasing intent. That's what's made and Google such amazing companies is they figured out how to get the thing you want to purchase in front of you as best as possible, right? and and all their work on recommendation systems is figuring out what you like 直到我完全外包了决定和购买意图还有不远了。这就是使谷歌成为如此了不起的公司的原因，他们弄清楚了如何尽可能好地将你想购买的东西放在你面前，对吧？他们在推荐系统上的所有工作都是弄清楚你喜欢什么 **34:33** how to keep you on the platform longer whether it's YouTube or Instagram or you know or bite dance right with Tik Tok or it's hey here's the ad of the thing you'll probably click on and buy because that's how I get paid and and everyone likes to claim they don't like pay attention to ads but you do right 如何让你在平台上停留更长时间，无论是YouTube还是Instagram，或者你知道字节跳动的TikTok，或者是嘿，这是你可能会点击和购买的东西的广告，因为这就是我如何获得报酬，每个人都喜欢声称他们不关注广告，但你确实关注，对吧 **34:45** before asking even more holistically kind of your view on where we're going there there's a third category which is the reasoning part of the equation so we've got pre-training we've got RL and environments post- training what about just like raw time spent reasoning thing and where that going as its own independent part of the overall scaling law. 在更全面地问你对我们去向的看法之前，有第三个类别，那就是方程的推理部分，所以我们有预训练，我们有RL和环境后训练，那么像花费在推理上的原始时间以及作为整体扩展定律的独立部分它去向哪里呢？ ## The Future of Reasoning and Compute Scaling ## 推理和计算扩展的未来 **35:02** The scaling laws again like if you zoom out that's not actually what the like original paper is but in spirit sure scaling laws are more compute better intelligence and that could be bigger and bigger model each iterative token is better whatever like word garbage I spew out if I went back and I wrote about everything I talked about in this I could make it way more condensed it could be way more clear 扩展定律再次，如果你缩小，那实际上不是原始论文的内容，但在精神上，当然扩展定律是更多计算更好的智能，那可能是越来越大的模型，每个迭代token都更好，无论我吐出什么样的文字垃圾，如果我回去写我在这里谈到的一切，我可以让它更浓缩，可以更清楚 **35:25** um potentially right now the benefit of podcast is a lot of times people more fun this way driving it's fun yeah exactly right uh they're walking their dog and they're listening whatever it is but like the interesting like an important like thing here is that by putting in these environments, you're teaching it like humans, right? 潜在地现在播客的好处是很多时候人们开车时更有趣，这很有趣，是的，确切地说，他们在遛狗并听，无论是什么，但就像有趣的，一个重要的事情是，通过将这些环境放入，你像人类一样教它，对吧？ **35:50** If I asked you, you know, to go figure something out, right? You might not necessarily know the answer right away, but I know you could probably figure it out in a given amount of time. That's reasoning. You're spending more brain cycles. The magic again of like intelligence of humans of of people is not that they are information retrieval like the best at information retrieval, right? 如果我让你，你知道，去弄清楚某事，对吧？你可能不一定马上知道答案，但我知道你可能能在给定的时间内弄清楚它。那就是推理。你花费了更多的大脑周期。人类智能的魔力再次不是他们是信息检索，像最擅长信息检索，对吧？ **36:08** Like like GPTs are amazing at information retrieval. We're really good at because we've been trained in these environments which is our world at figuring out how to do things iteratively. And so reasoning and these oral environments are linked together, right? 就像GPT在信息检索方面很惊人。我们真的很擅长，因为我们已经在这些环境中接受了训练，这就是我们的世界，在迭代地弄清楚如何做事情。所以推理和这些口头环境联系在一起，对吧？ **36:18** If if I'm telling a model, hey, do this math puzzle, it's not it's not just spewing out like, oh, the answer's one, oh, the answer's two. Oh, the answer's three. Okay, the answer was actually seven. And when it got there, I trained it again. It's like, okay, now it knows next time. Oh, the answer is six, seven, or eight. Now it's like seven. Okay, great. 如果我告诉一个模型，嘿，做这个数学谜题，它不是只是吐出，哦，答案是一，哦，答案是二。哦，答案是三。好的，答案实际上是七。当它到达那里时，我再次训练它。就像，好的，现在它知道下次。哦，答案是六、七或八。现在就像七。好的，太好了。 **36:34** It's not like now it instantly knows the answer. It's actually like, oh, here's this puzzle. Oh, like these numbers. Oh, this line, it's sudoku. These numbers add up to this. Um, oh, it has one through nine, but it's missing eight. Okay, it's eight. Right? Like it's thinking through it, right? 不是说现在它立即知道答案。实际上就像，哦，这是这个谜题。哦，像这些数字。哦，这条线，是数独。这些数字加起来是这个。哦，它有一到九，但缺少八。好的，是八。对吧？就像它在思考它，对吧？ **36:46** Like you and I would solve a sodoku. Now, eventually when you get good enough at sodoku, you could probably just like spit out an answer. Um you could do it in your sleep but for a long time you can do it without like you know and so sort of like this reasoning time is a way of spending more compute more brain cycles on the task without actually you know scaling the model 就像你和我会解决数独。现在，最终当你足够擅长数独时，你可能可以吐出一个答案。你可以在睡梦中做，但很长一段时间你可以做而不像你知道，所以有点像这个推理时间是一种在任务上花费更多计算更多大脑周期的方式，而实际上你知道不扩展模型 **37:07** um and then the model becomes more versatile right because humans have a rate if I just held a match against you and you didn't notice it you'd immediately jerk right because you're the the the rate at which you operate is hundreds of hundreds of hertz right you your body can actually take actions at like hundreds of actions per second 然后模型变得更多才多艺，对吧，因为人类有一个速率，如果我只是对你举一根火柴，你没有注意到它，你会立即反应，对吧，因为你操作的速率是数百赫兹，对吧，你的身体实际上可以每秒采取数百次行动 **37:21** if you look at like a fighter pilot's reaction time, right? The the peak of human like reaction time. Now, what reaction can they do is like completely like primal instinctual, right? Very little thought is put into it. If you think about like this alien intelligence that we're trying to make, is it is it immediately going to oneshot the answer always? No. 如果你看战斗机飞行员的反应时间，对吧？人类反应时间的巅峰。现在，他们能做什么反应就像完全是原始本能的，对吧？很少有思考投入其中。如果你想想我们试图制造的这种外星智能，它是否会立即一次性得出答案？不。 **37:49** But like, you know, at times it needs to. Yeah. At times it needs to be able to, you know, tell me exactly the answer in like two seconds or half a second or um, you know, whatever action it needs to take immediately. But a lot of times also needs to think through the problem, go and do stuff. 但就像，你知道，有时它需要。是的。有时它需要能够，你知道，在两秒或半秒内准确地告诉我答案，或者，你知道，它需要立即采取的任何行动。但很多时候也需要思考问题，去做事情。 **38:02** That's why you hire students. That's why you hire interns cuz you're like, "Yeah, I know this data exists. Here's the format I kind of want it on and go figure it out." And then they they spend a whole summer doing something you could have done in like 3 days, but like great, they learned a [ __ ] ton, right? 这就是为什么你雇用学生。这就是为什么你雇用实习生，因为你就像，"是的，我知道这些数据存在。这是我想要的格式，去弄清楚它。"然后他们花了整个夏天做你本可以在3天内完成的事情，但就像很好，他们学到了很多，对吧？ **38:15** And it's like these models need to go through that progression. Um, and so when I think about, you know, reasoning RL, it's it's a lot about how the human psyche and and intelligence works. Uh, and and sort of like I wouldn't say, you know, there there's there's a caution of like trying to make it too much like humans cuz it's not the fundamental substrate is not like humans. 就像这些模型需要经历那个进程。所以当我想到，你知道，推理RL时，这很大程度上是关于人类心理和智能如何运作。而且有点像我不会说，你知道，有一个警告，就像试图让它太像人类，因为基本基质不像人类。 **38:34** The processing is not like humans. Our brain is very different from a, you know, how how these ALUs on a chip works. Like the scaling of these things is very different. The raw speed, the amount of words they can, everything is so different. But at the same time, it's important to like reckon back to what actually makes people, you know, smart. 处理不像人类。我们的大脑与，你知道，芯片上的这些ALU如何工作非常不同。这些东西的扩展非常不同。原始速度，它们可以的单词数量，一切都如此不同。但与此同时，重要的是回到真正使人们，你知道，聪明的东西。 ## Memory and Context in AI: Short-term vs Long-term ## AI中的记忆和上下文：短期与长期 **38:51** On the topic of like embodiment, uh, and continuing with the human analogy, how do you think about things like short and long-term memory in a human versus just like raw model capacity or something? Like what role does that analogy of memory, I don't mean literally like like semiconductor memory, but like memory in a model, how do you think about the importance that that will play and where are we in that? 关于具体化的话题，继续人类类比，你如何看待人类的短期和长期记忆与原始模型容量之类的东西？记忆的类比扮演什么角色，我不是指字面上的半导体内存，而是模型中的记忆，你如何看待它将扮演的重要性以及我们在哪里？ **39:07** The magic of transformers was uh attention right i.e. I calculate everything in my context length. I cont I I I calculate the attention to each other, right? Basically, in a vector space like king, queen, there's these vectors. There's like dozens of vectors for each number. And king and queen are actually exactly the same on a ton of stuff, but then it's the opposite on one number because one's a male, one's a female. Transformer的魔力是注意力，对吧，也就是说，我计算我上下文长度中的所有内容。我计算相互之间的注意力，对吧？基本上，在像king、queen这样的向量空间中，有这些向量。每个数字有几十个向量。King和queen在很多方面实际上完全相同，但在一个数字上相反，因为一个是男性，一个是女性。 **39:33** And then that that will have a lot of other like, you know, ramifications throughout other literary stuff like, you know, what adjectives do you put with a male of, you know, this vectors? it's like regal and you know like powerful and could be ruthless whereas a queen could be like dignitary or whatever like I don't I don't know like whatever stupid stupid analogy 然后那将在其他文学内容中产生很多其他影响，就像，你知道，你用什么形容词来描述这个向量的男性？就像高贵，你知道像强大，可能是无情的，而女王可能像政要或什么的，我不知道，无论什么愚蠢的类比 **39:50** but when you think about how that applies to you know humans what what we're terrible at like exact recall I could tell you a sentence and tell you to repeat it yeah it's like six numbers the average person can remember or something like that right but like you get the gist of the sent if I told you like if I told you a whole paragraph you'd get the gist of it and you could you could repeat the meaning of it to someone, you could translate that meaning. 但当你想到这如何适用于你知道人类，我们不擅长的是精确回忆，我可以告诉你一句话并告诉你重复它，是的，就像普通人可以记住六个数字或类似的东西，对吧，但就像你得到句子的要点，如果我告诉你，如果我告诉你整个段落，你会得到它的要点，你可以向某人重复它的意思，你可以翻译那个意思。 **40:13** So models very different, right? Fundamentally transformer attention has been, you know, calculating the attention to everything to each other and getting the models to actually be able to recall. That's been a training data problem. But like you can get the model to repeat exactly what you want, anything in its context length. 所以模型非常不同，对吧？从根本上说，Transformer注意力一直在计算对一切事物的相互注意，并让模型实际上能够回忆。那是一个训练数据问题。但就像你可以让模型准确重复你想要的任何东西，它上下文长度中的任何东西。 **40:25** It's like a needle in the haststack is the like problem that like you know it's a benchmark that people did for a while because models had to get good at that. But now models are just like amazing, right? Like tell me tell me like blah blah blah and random part of your context. But what they really suck at is having infinite context 这就像大海捞针是问题，就像你知道这是人们做了一段时间的基准，因为模型必须擅长那个。但现在模型就像惊人，对吧？就像告诉我告诉我像等等等等和你上下文的随机部分。但他们真正不擅长的是拥有无限上下文 **40:42** because you have infinite and it's sp what the real word is sparse right you have sparse you you've taken this entire world and you've encoded it in such a small amount of data that lives in your brain and it's so sparse but you understood how to like grab the fundamental reason and put it down there 因为你有无限的，真正的词是稀疏的，对吧，你有稀疏的，你已经把这整个世界编码在如此少量的数据中，这些数据存在于你的大脑中，它是如此稀疏，但你理解如何抓住基本原因并把它放在那里 **41:00** whereas models they haven't been able to create something sparse yet right what is the long how do you how do you how do you reason over the context of infinity. And you know, humans maybe we have like a short-term memory and a long-term memory. I think it's a lot more blurry than that. There's no like clear line. Oh, this was in my short-term memory. Oh, this is in my long-term memory. 而模型还没有能够创建稀疏的东西，对吧，长期是什么，你如何在无限的上下文中推理。你知道，人类也许我们有短期记忆和长期记忆。我认为这比那模糊得多。没有明确的界线。哦，这是在我的短期记忆中。哦，这是在我的长期记忆中。 **41:18** It's like it is much more blurry, but as we as we go back and back and back, it's more and more sparse, right? If we think about, hey, what do you remember as a kid? The most crazy thing in psychology, I remember when I learned it, I was like, wait, my memory of what I did as a kid with my, you know, dad at this like thing, right? Is fake. 就像它更模糊，但随着我们回到过去，它越来越稀疏，对吧？如果我们想想，嘿，你小时候记得什么？心理学中最疯狂的事情，我记得当我学到它时，我就像，等等，我作为一个孩子和我的，你知道，爸爸在这个事情上做了什么的记忆，对吧？是假的。 **41:37** It's me remembering it and inventing the picture and me remembering that picture like successively but like the actual memory of what happened is like morphed a little bit um over time um because it's a spark like we the way humans collapse information um is super super dense and and but we are able to extract all the relevant information out 这是我记住它并发明图片，我连续记住那张图片，但就像实际发生的事情的记忆随着时间的推移有点变形，因为它是一个火花，就像我们人类压缩信息的方式非常非常密集，但我们能够提取所有相关信息 **41:54** now models they have there's a there's a ton of research going on in this domain of long context right how do I get longer and longer context without blowing up my model cost. This is a big challenge with reasoning. This is why, you know, we had this HBM bullish uh pitch for a while, right? Is like, you know, you need a lot of memory when you extend the context, right? 现在模型，有大量关于长上下文领域的研究正在进行，对吧，我如何在不增加模型成本的情况下获得越来越长的上下文。这是推理的一个大挑战。这就是为什么，你知道，我们有一段时间看好HBM，对吧？就像，你知道，当你扩展上下文时，你需要大量内存，对吧？ **42:13** Simple thesis, right? But the fundamental algorithm needs to change and improve over time iteratively to get to something like this short and long context of memory. That doesn't necessarily mean the model has to work like we do, right? Why can't the model just reason and have a database that it writes stuff in or like a word document 简单的论点，对吧？但基本算法需要随着时间的推移迭代地改变和改进，以达到像这样的短期和长期记忆上下文。这不一定意味着模型必须像我们一样工作，对吧？为什么模型不能只是推理并拥有一个它写东西的数据库或像Word文档 **42:30** that it writes stuff in and then it like takes it out of its context, works somewhere and like Roful calls back. It's like, oh yeah, right? Like we don't do that, right? Like you and I refer to our notes, we refer to our calendar, we refer to our text, we refer to anything all the shopping list, right? Like great, I know I need food for dinner, I go to the store, I'm like I need a shopping list, right? 它在其中写东西，然后它就像把它从上下文中取出，在某处工作，就像Roful回调。就像，哦，是的，对吧？就像我们不这样做，对吧？就像你和我参考我们的笔记，我们参考我们的日历，我们参考我们的文本，我们参考任何购物清单，对吧？就像很好，我知道我需要晚餐的食物，我去商店，我就像我需要一个购物清单，对吧？ **42:53** Like cuz otherwise I'm going to buy like stupid [ __ ] right? is like it's like so so the model doesn't necessarily have to fundamentally work the same way as humans. But there is that challenge of like how do I how do I train the model to operate over the context length of a human? How do I train it to interact with these databases and these word documents that it writes to? 就像否则我会买一些愚蠢的东西，对吧？就像所以模型不一定必须从根本上以与人类相同的方式工作。但有那个挑战，就像我如何训练模型在人类的上下文长度上操作？我如何训练它与这些数据库和它写入的Word文档交互？ **43:10** Because it's never going to learn that from pre-training has to learn that from an environment. But these environments have to be like architected in a way where the model knows it can write stuff down and refer back. And so one of the first things Openi did was deep research, right? deep research is everything is not in deep research's context, right? 因为它永远不会从预训练中学到那个，必须从环境中学到。但这些环境必须以某种方式构建，模型知道它可以写下东西并参考回来。所以OpenAI做的第一件事之一是深度研究，对吧？深度研究是一切都不在深度研究的上下文中，对吧？ **43:29** Deep research is working for like 45 minutes. It's outputting millions and millions of tokens and it's creating this amazing like you know thing that it wrote, right? And it's like pretty good research. Um I would say a lot of like memos that you read from people are like on the on par with like deep research at least like a junior. 深度研究工作大约45分钟。它输出数百万个token，它创建了这个惊人的，你知道，它写的东西，对吧？就像相当好的研究。我会说你从人们那里读到的很多备忘录就像与深度研究相当，至少像一个初级。 **43:40** How did they do that? was they they they enabled it to be able to write something down elsewhere and have this recall and and you know and and and effectively use language to compress information that it looked at, put that off to the side, use language to compress other information off to the side, use language to compress other information off to the side and then looking at all this compressed information and writing something, right? 他们是如何做到的？是他们使它能够在其他地方写下东西并拥有这种回忆，你知道，有效地使用语言来压缩它看到的信息，把它放在一边，使用语言来压缩其他信息放在一边，使用语言来压缩其他信息放在一边，然后查看所有这些压缩的信息并写一些东西，对吧？ **44:02** And that's that's sort of what deep research is. So how do models get there? I'm not sure, right? Like I think it's a fundamental research challenge. It's why these companies need, you know, millions of GPUs to train on. Yeah. Not for, oh, I'm gonna make a million GPU model, but because I need to try a bajillion different things 这就是深度研究的本质。那么模型如何到达那里？我不确定，对吧？就像我认为这是一个基本的研究挑战。这就是为什么这些公司需要，你知道，数百万个GPU来训练。是的。不是为了，哦，我要制作一个百万GPU模型，而是因为我需要尝试无数不同的东西 **44:20** because I don't know what will work and what's going to be what's going to work for humans is so different from what works with models. There's like any number of parameters or things you could tweak that could end up like changing how it develops, right? Um, and how good is it at, you know, if I do it this way versus that way, right? That's that's the whole point of ML research is is you're constantly trying stuff out and and trying to get better and better. 因为我不知道什么会起作用，什么会对人类起作用与对模型起作用的东西如此不同。就像你可以调整的任何数量的参数或事物，最终可能会改变它如何发展，对吧？如果我这样做与那样做，它有多好，对吧？这就是ML研究的全部要点，你不断尝试东西并试图变得越来越好。 ## The Spectrum of AI Optimism ## AI乐观主义的范围 **44:39** If I add all of this up and you know hold the mirror up, it's it seems like I would put you in the category of like unbelievably bullish on what these things are going to be able to do in 10 years time or something like pick your time frame. Yeah. Am I calibrated the right way? Like amongst everyone you talk to who you respect and think is 如果我把所有这些加起来，你知道举起镜子，似乎我会把你归入像难以置信地看好这些东西在10年内能够做什么的类别，或者类似的选择你的时间框架。是的。我校准得对吗？就像在你交谈的每个你尊重并认为是 **44:57** I'm much more emarrassed than a lot of people actually which is the crazy thing. help help me understand that distinction. Like if you're where are you 1 through 10 amongst the people that you respect 10 being the most bullish and then like what is the difference between if you're not a 10. What's the difference between you and the person who's a 10? 我实际上比很多人更谨慎，这是疯狂的事情。帮助我理解那个区别。就像如果你在你尊重的人中从1到10，10是最看好的，那么如果你不是10，你和10之间有什么区别？ **45:09** I respect you, but I know I'm like way more bullish than you and I respect like Mark Zuckerberg, but I know he might be he's probably maybe I don't know if he's more bullish than me, but I know Sam Alman's definitely way more bullish than me, right? He says he says we have artificial general intelligence in in less than a thousand days, right? 我尊重你，但我知道我比你看好得多，我尊重马克·扎克伯格，但我知道他可能是，他可能也许我不知道他是否比我更看好，但我知道Sam Altman绝对比我看好得多，对吧？他说我们将在不到一千天内拥有人工通用智能，对吧？ **45:27** say, you know, like or Dario, like I respect him immensely, but he's way more bullish than me. Right. My roommates are like one of them is an anthropic ML researcher and one of them is is another podcaster, Dwarvesh, like they're they're both like way more bullish than I am. Really? Yeah. Yeah. Yeah. But like even they are not as bullish as like some researchers in this field. 说，你知道，像Dario，我非常尊重他，但他比我看好得多。对。我的室友就像其中一个是Anthropic ML研究员，其中一个是另一个播客主持人Dwarvesh，他们都比我看好得多。真的吗？是的。是的。是的。但就像即使他们也不像这个领域的一些研究人员那样看好。 **45:47** So it's like but then if I go talk to like uh you know someone someone I respect like I don't know like a famous investor right like uh you know any of these famous investors I don't want to name one because I'm scared you know like but there's all these famous investors right it's like well no they're not they're not more bullish than me 所以就像但如果我去和，你知道，我尊重的某人交谈，我不知道，像一个著名的投资者，对吧，你知道，任何这些著名的投资者，我不想点名，因为我害怕，你知道，但有所有这些著名的投资者，对吧，就像嗯不，他们不比我更看好 **45:54** and the stuff I'm saying sounds like crazy [ __ ] some of it though is timeline um I'm I'm actually even more curious about like the upper limit the extent to which there is the upper limit I think I'm among the most bullish you can get because that's what I mean the upper limit of this is that this will just be smarter than humans. 我说的东西听起来像疯狂的废话，尽管其中一些是时间表，我实际上更好奇上限，上限的程度，我认为我是最看好的，因为这就是我的意思，这个的上限是它将比人类更聪明。 ## Timeline to AGI ## 通向AGI的时间表 **46:11** I don't think that will happen like anytime soon. Even if that doesn't happen anytime soon, there's so much valuable stuff that can be done with these models that economically we will skyrocket. There's so much value that can be created in the world just by hey, if the models know how to do uh cobalt to like C and Python migration of like main frames, just migrate everything migrate everything from mainframes to cloud 我不认为那会很快发生。即使那不会很快发生，用这些模型可以做这么多有价值的事情，经济上我们将飙升。仅仅通过嘿，如果模型知道如何做从COBOL到C和Python的主机迁移，只需迁移所有东西，将所有东西从主机迁移到云，就可以在世界上创造这么多价值 **46:35** the world is how much more efficient? Hey, making making all these random applications and like automated reports and like stop using Excel as a database, but instead like you can make a real database and and manipulate stuff in Excel, but like you know there's all sorts of like humongous business efficiencies that could happen or automation that could happen 世界效率提高了多少？嘿，制作所有这些随机应用程序和自动化报告，停止使用Excel作为数据库，而是像你可以制作一个真正的数据库并在Excel中操作东西，但你知道有各种巨大的业务效率可能发生或自动化可能发生 **46:54** without the model ever being, you know, we could literally just pause it at like a 6 months from now time frame of like how good it is at software development and it would be like godsend in terms of like how much efficiency and value can be created for the economy 而无需模型永远是，你知道，我们可以从字面上在大约6个月后的时间框架内暂停它，就像它在软件开发方面有多好，在可以为经济创造多少效率和价值方面，这将像天赐之物 **47:04** and it doesn't ever have to get to like digital god Now, now I do believe we're going to get the digital god eventually. Eventually, is that 10 years? Is that 5 years? Is that 100 years? Is that a thousand years? I don't know. Cuz there's there's so many unknown unknowns. 它永远不必达到数字上帝的水平。现在，我确实相信我们最终会达到数字上帝。最终，是10年吗？是5年吗？是100年吗？是一千年吗？我不知道。因为有这么多未知的未知。 **47:22** Like I mentioned, right? Like these these babies are putting their freaking hand in their mouth to calibrate. And then later they put their foot in their mouth and they're like, "Oh, that's my foot. Oh, here's the senses on it." And then they can pick up stuff in their hand and they no longer have to put it on their most sensitive part of their body because they know what it is. 就像我提到的，对吧？就像这些婴儿把他们该死的手放进嘴里来校准。然后他们把脚放进嘴里，他们就像，"哦，那是我的脚。哦，这是上面的感觉。"然后他们可以用手拿东西，他们不再需要把它放在他们身体最敏感的部分，因为他们知道它是什么。 **47:39** Or they're like, "Oh, this is a speck on the ground. What is it? It's not food. But now I know what it feels like inside my hands and I've calibrated, right? It's like the models have not gotten there yet, right? Like it has no idea how to do this. 或者他们就像，"哦，这是地上的一个斑点。它是什么？它不是食物。但现在我知道它在我手里的感觉，我已经校准了，对吧？就像模型还没有到达那里，对吧？就像它不知道如何做这个。 **47:50** Digital god is like well one I like kind of believe in embodiment and like you need non-digital god. You need a physical and you need the capability of like having touch and feel and all that to truly be uh have have an experience like humans and be smarter than us in every way. But you know that's so far away. 数字上帝就像，嗯，我有点相信具体化，你需要非数字上帝。你需要物理的，你需要像拥有触摸和感觉以及所有这些的能力，才能真正拥有像人类一样的体验，并在各方面都比我们更聪明。但你知道那太遥远了。 ## Physical Intelligence and Embodiment ## 物理智能和具体化 **47:56** What do you think about what physical intelligence is doing? attacking the whatever you want to call it large movement model or large robot model or something. What are they actually doing today is like holy [ __ ] it's so simple in terms of like to a human. Yeah. It's like to to models it's like picking this up is freaking hard. 你如何看待物理智能正在做什么？攻击你想称之为大型运动模型或大型机器人模型或其他什么。他们今天实际做的是，对人类来说这太简单了。是的。就像对模型来说，拿起这个该死的难。 **48:13** Like how much do I squeeze my pinky versus this finger versus this finger versus finger? I don't know. Like but you pick up a glass of water and you tilt it and it's like this is impossible for a model today and it's likely like at the level of dexterity like you know if I if I if it was a wine glass and I was swishing it. 就像我小指挤多少与这个手指与这个手指与手指？我不知道。但你拿起一杯水并倾斜它，这对今天的模型来说是不可能的，它可能就像灵巧性的水平，你知道，如果我，如果是一个酒杯，我在摇晃它。 **48:30** Think about how simple that is. You don't even think about it, but like you instinctually pick up a wine glass and you swish it and it lets the aroma out and you smell it, but it's like, oh, that little swish is so much tactile feedback and movement and it's like these models can't do that [ __ ] yet. Like nowhere close. 想想那有多简单。你甚至不考虑它，但就像你本能地拿起一个酒杯，摇晃它，它释放出香气，你闻到它，但就像，哦，那个小摇晃是这么多触觉反馈和运动，就像这些模型还不能做那该死的。完全不接近。 **48:43** So, I mean, I think yes, we have, you know, but it doesn't need to be that good. It doesn't need to be able to swish a wine glass and not break the wine glass and put it back down and tilt it perfectly and not spill it. Doesn't need to be able to do any of that to be tremendously valuable. 所以，我的意思是，我认为是的，我们有，你知道，但它不需要那么好。它不需要能够摇晃酒杯而不打破酒杯并放回去并完美倾斜并不洒。不需要能够做任何这些就非常有价值。 **48:54** What it needs to be tremendously valuable is pick this up and put it down here after knowing what it is. Um, so there's so much value that could be created just by being, you know, really good at like getting data. Yeah. Yes. I like, you know, I think the robotics world is huge. I think we're, you know, we're we're like we literally warming up. We haven't even left the dug dugout, right? 要非常有价值，它需要在知道它是什么之后拿起这个并把它放在这里。所以仅仅通过，你知道，真正擅长获取数据就可以创造这么多价值。是的。是的。我喜欢，你知道，我认为机器人世界是巨大的。我认为我们，你知道，我们就像我们真的在热身。我们甚至还没有离开休息区，对吧？ ## Talent Wars ## 人才战争 **49:11** Like we're we're like nowhere close to, you know, the scaling on on robotics. There's a ton of like the data flywheel needs to get going there. One of the most interesting things, the subplots of this whole world is the talent wars. And a cool idea is that as these things get better, maybe we begin to automate some of the research function that people formally would have played. 就像我们离机器人的扩展还很远，你知道。那里有大量的数据飞轮需要启动。最有趣的事情之一，这整个世界的次要情节是人才战争。一个很酷的想法是，随着这些东西变得更好，也许我们开始自动化人们正式会扮演的一些研究功能。 **49:32** Do you see a world where like we're squeezing down the fewer and fewer number of people that really matter that will have all the impact on where we go uh in terms of like net new research and that means that all this crazy spending that's happening at Meta or elsewhere makes a lot of sense that like maybe even those numbers should be higher or something like this. 你看到一个世界，我们正在压缩越来越少的真正重要的人，他们将对我们去向产生所有影响，就像净新研究，这意味着Meta或其他地方发生的所有疯狂支出都很有意义，就像也许那些数字应该更高或类似的东西。 **49:43** I think it's like tremendously hilarious that people are like, "Oh my god, this person's getting paid a billion dollars." It is infeasible. It's like how could this person possibly be worth that much? Well, they're running the experiments on chips that cost, you know, hundred billion 我认为人们说，"哦，天哪，这个人得到了10亿美元的报酬"是非常滑稽的。这是不可行的。就像这个人怎么可能值那么多？好吧，他们正在价值，你知道，数千亿的芯片上运行实验 **50:01** if every wasted experiment they do, if they just used like a third of the compute and and their ideas and their impact on it wasted the compute. It was an idea that was already done or like you know like there's so much wasted compute. I'll say I call it wasted. It's trying stuff and failing. But like none of us know what to try and what not to try and and these things are so complicated. 如果他们做的每个浪费的实验，如果他们只使用像三分之一的计算，他们的想法和他们对它的影响浪费了计算。这是一个已经完成的想法，或者像你知道有这么多浪费的计算。我会说我称之为浪费。它在尝试东西并失败。但就像我们中没有人知道尝试什么和不尝试什么，这些东西如此复杂。 **50:23** There's like a group of people just trying different stuff on the existing data. How do you mix it? What order do you feed it into the model? Um how do you filter it? Like what's the architecture? There's different people working on long context. There's different people working on every single aspect of the model that like if you just make them a little bit more efficient that they come up with the idea that's 5% more efficient. 有一群人只是在现有数据上尝试不同的东西。你如何混合它？你以什么顺序将它输入模型？你如何过滤它？架构是什么？有不同的人在研究长上下文。有不同的人在研究模型的每个方面，就像如果你让他们稍微更高效，他们就会想出效率提高5%的想法。 **50:36** Well, fantastic. I just saved not only 5% of my compute time, training time, I also save 5% across my entire inference fleet. And then I do it again and again and again and again because we're so far away from like these models being anywhere near as efficient as a human brain and we know it can at least get as efficient as us. 好极了。我不仅节省了5%的计算时间、训练时间，我还在整个推理舰队中节省了5%。然后我一次又一次地这样做，因为我们离这些模型接近人脑的效率还很远，我们知道它至少可以达到和我们一样的效率。 **50:48** Maybe the compute substrate isn't the same, but like whatever, right? Adding more people to the problem doesn't make it faster, right? Because there's so many things you're trying. You run these experiments, you learn something, and then you implement it. You tweak the knobs in these ways in 100 different ways and then you see the trend line and you're like, "Oh, so actually I should tweak it this way. Let's implement that." 也许计算基质不一样，但无论如何，对吧？向问题添加更多人并不会让它更快，对吧？因为你正在尝试这么多事情。你运行这些实验，你学到一些东西，然后你实施它。你以100种不同的方式以这些方式调整旋钮，然后你看到趋势线，你就像，"哦，所以实际上我应该以这种方式调整它。让我们实施那个。" **51:08** Right? There's so much like gut feel. there's so much like reading data, understanding it, imple reimplementing it into these things that if you add people, you're going to slow it down. In a sense, a lot of like Meta's problems before they did the super intelligence thing was that they just had too many people that weren't led by leadership that was amazing. 对吧？有这么多直觉。有这么多阅读数据、理解它、将其重新实施到这些东西中，如果你添加人，你会减慢速度。从某种意义上说，Meta在做超级智能之前的很多问题是他们只是有太多人，没有被出色的领导领导。 **51:24** Um, and they had like a lot of failed experiments and wasted time doing things that didn't matter. Um, there's a there's a tweet from one of my friends uh at OpenAI. Um, he's pretty famous on Twitter. His name is Run. He's like, I get visibly viscerally angry every time I think about how many H100s Meta is wasting. 他们有很多失败的实验并浪费时间做不重要的事情。有一条来自我在OpenAI的一个朋友的推文。他在Twitter上很有名。他的名字是Run。他就像，每次我想到Meta浪费了多少H100，我就明显地本能地生气。 **51:42** It's such a funny tweet because it's like, well, yeah, they're wasting a ton of compute. They were, you know, maybe they still are, but you know, like, and everyone's wasting compute, right? Opening eyes wasting tons of compute cuz, you know, what's the paralo optimal model architecture? Who knows? 这是一条如此有趣的推文，因为它就像，嗯，是的，他们在浪费大量计算。他们是，你知道，也许他们仍然是，但你知道，每个人都在浪费计算，对吧？OpenAI在浪费大量计算，因为，你知道，什么是帕累托最优模型架构？谁知道呢？ **51:59** Another thing I saw Run say recently which was so interesting was uh why don't we just go make even more ridiculous offers around the people that have process knowledge for things that we want here in the US in other countries like why don't we if if we're getting pretty good at the Arizona you know fab that we've built and we we think that we can sort of extract the process knowledge from the people 我最近看到Run说的另一件如此有趣的事情是，我们为什么不对在其他国家拥有我们在美国想要的东西的流程知识的人提出更荒谬的报价，就像如果我们在我们建造的亚利桑那州工厂做得相当好，我们认为我们可以从人们那里提取流程知识 **52:11** why don't we like go aqua hire like all the best people in Shenzen or all the best people in like other places in the world do you think it starts to escalate to that level like so much is dependent on the process knowledge of a relatively small group of people and the and the war the talent war should actually be it shouldn't be meta and open AI it should be like the US maybe through meta and open AI and like people from all over the world 我们为什么不去收购雇佣深圳所有最优秀的人或世界其他地方所有最优秀的人，你认为它开始升级到那个水平吗，就像这么多取决于相对较小的一群人的流程知识，战争，人才战争实际上应该是，它不应该是Meta和OpenAI，它应该像美国也许通过Meta和OpenAI以及来自世界各地的人 **52:33** like do you think it starts to get that extreme and should it that's almost a function of why like Intel is has fallen off a lot right is like um you have all these geniuses in you know you know nanochemistry and PhDs and all these like random like you know like things whether it be chemistry physics all these incredibly smart people 你认为它开始变得那么极端吗，它应该吗，这几乎是为什么英特尔已经下降很多的一个功能，对吧，就像你有所有这些天才，你知道纳米化学和博士学位，所有这些像随机的，你知道，事情，无论是化学物理，所有这些令人难以置信的聪明人 **52:56** but there's a whole class of incredibly smart people that never went that way because they're like, "Oh, those guys are making like 200k." Like, why would I do that? I'm gonna go I'm gonna go to Google and make 800k and now I'm going to go to OpenAI and make 10 mill or no I'm going to go to Meta and go make 100 million, right? 但有一整类令人难以置信的聪明人从未走那条路，因为他们就像，"哦，那些人赚20万。"就像，我为什么要那样做？我要去谷歌赚80万，现在我要去OpenAI赚1000万，或者不，我要去Meta赚1亿，对吧？ **53:09** Like any like smart 18-year-old is going to be like, "Fuck that. I'm doing this, right?" Why do the smartest doctors, and I don't mean to say the smartest doctors in a general sense, but their skews really smart population of doctors that want to be dermatologists and anesthesiologists. It's like, is that the most valuable thing for them to do? No. 就像任何聪明的18岁孩子都会说，"去他妈的。我要做这个，对吧？"为什么最聪明的医生，我不是说最聪明的医生在一般意义上，但他们倾向于想成为皮肤科医生和麻醉师的非常聪明的医生群体。就像，那是他们做的最有价值的事情吗？不。 **53:28** But those are the two professions that give you like good working hours and great pay. Yeah. Not to say that, you know, the general doctor is not smart as them. But if you took the population of general, like family doctors, like just the random doctor, and you took the population of dermatologists, the newest coming out of school, the ones that are being dermatologist and anesthesiologist are way smarter or at least were scored better, were able to get into the field that was coveted. 但这是两个给你良好工作时间和优厚报酬的职业。是的。并不是说，你知道，普通医生不如他们聪明。但如果你取一般的人口，像家庭医生，像随机医生，你取皮肤科医生的人口，刚从学校出来的最新的，那些成为皮肤科医生和麻醉师的人要聪明得多，或者至少得分更好，能够进入梦寐以求的领域。 **53:45** And so, yeah, talent wars like it it is truly like, you know, we've sort of been through this process of like capital has, you know, it's it's always been human human capital and and capital goods sort of those two vying off of each other. And for a long time with mechanization, industrialization, we had the human capital decreasing as the industrial capital increased. 所以，是的，人才战争就像它真的是，你知道，我们已经经历了这个过程，资本有，你知道，它一直是人力资本和资本货物，这两者相互竞争。长期以来，随着机械化、工业化，随着工业资本的增加，我们的人力资本在减少。 **54:05** Um and sort of that got to a point where especially in the 70s it really started to tank as the ability to globalize and and all these things started to really hit the US and and that's why we have all a lot of the like population level dynamics and income inequality that we have today that like is very bad for the psyche of the US and and the stab stability of it. 这在某种程度上达到了一个点，特别是在70年代，随着全球化的能力和所有这些事情开始真正打击美国，它真的开始下降，这就是为什么我们今天有很多像人口水平动态和收入不平等，这对美国的心理和稳定性非常不利。 **54:23** But then you have now you now have like we're in such a age of like well actually like manufacturing things is pretty commodity like most of the value doesn't come from the manufacturing of it. comes from the creation of the idea. One thing Jensen told me which I thought was like amazing, right? He's like, you know, Dylan, he's like, the reason America is rich, like people have it all wrong. 但现在你有了，我们现在处于这样一个时代，实际上制造东西是相当商品化的，大部分价值不来自制造它。来自想法的创造。Jensen告诉我的一件事，我认为这太棒了，对吧？他就像，你知道，Dylan，他就像，美国富有的原因，人们都搞错了。 **54:41** The reason we're rich is because we've exported all the labor, but we've kept all the value. And that's what Nvidia does, right? They've exported the labor of making their chips and Apple, right, everyone. It's it's done in Asia. Um, and those those companies make money, not as much money as Nvidia and Apple, right? 我们富有的原因是因为我们已经出口了所有劳动力，但我们保留了所有价值。这就是Nvidia所做的，对吧？他们已经出口了制造芯片的劳动力，还有苹果，对吧，每个人。它在亚洲完成。那些公司赚钱，但不如Nvidia和苹果那么多，对吧？ **54:58** All the gross profits are going to them. Um, and then they're either reinvesting it or buying back stock or whatever. However they allocate the capital is like you know different concern. If like as you said the process knowledge is so valuable why aren't we why aren't we doing this? That's that's a that's a great idea. Run's idea not mine. Yeah. 所有毛利润都流向他们。然后他们要么再投资，要么回购股票，无论如何。然而，他们如何分配资本是，你知道，不同的关注点。如果像你说的流程知识如此有价值，我们为什么不这样做？那是一个好主意。Run的想法，不是我的。是的。 **55:18** No I mean I think I think the challenge is how to choose people is really difficult. So for some roles someone who can talk the talk they're great right? Like people just automatically assume they're great because they can talk the talk. But you know how many people suck at talking and are really freaking good at doing. Yeah. Yeah. But then you don't know. You don't know, right? 不，我的意思是，我认为挑战是如何选择人真的很困难。所以对于某些角色，能说会道的人很棒，对吧？就像人们自动假设他们很棒，因为他们能说会道。但你知道有多少人不擅长说话但真的非常擅长做事。是的。是的。但你不知道。你不知道，对吧？ **55:34** Because it's like, well, but then there's people who talk about being able to do better than the person who's doing and like and then like you know these tests are never as good, right? Like so work trials. It's like how do you select? Um and this was a big challenge for Meta. 因为就像，嗯，但有人谈论能比做事的人做得更好，就像，你知道，这些测试从来不如，对吧？就像工作试验。就像你如何选择？这对Meta来说是一个巨大的挑战。 **55:50** Um so some of the criticisms are like they didn't get all of the best people. They actually got a lot of like bad people. It's like the cope from like OpenAI and Enthropic and you know these kinds of companies are like no no no they didn't get our best people. That's what Sam said, right? He's like they didn't get our best people. It's okay. 所以一些批评是他们没有得到所有最好的人。他们实际上得到了很多坏人。这就像OpenAI和Anthropic以及你知道的这些公司的应对，就像不不不，他们没有得到我们最好的人。这就是Sam说的，对吧？他就像他们没有得到我们最好的人。没关系。 **56:01** Meanwhile, he did have to do counter offers internally, right? So it's like you know um so so as far as the process knowledge I think I think the ML researchers are an extreme of how much value one can do. But my favorite analogy that I came up with recently is that ML research is the exact same as semiconductor manufacturing. 与此同时，他确实必须在内部做反报价，对吧？所以就像你知道，就流程知识而言，我认为ML研究人员是一个人可以做多少价值的极端。但我最近想出的最喜欢的类比是ML研究与半导体制造完全相同。 **56:19** You know, there's a ton of jobs in in in semiconductor manufacturing that don't exist in ML research, but it is a ton of tune a thousand different knobs, right? Oh, you put the wafer in this tool. You're going to change the pressure of the chamber when you're doing the deposition. Oh, you're going to change the mix of the chemicals flowing in. 你知道，半导体制造中有大量在ML研究中不存在的工作，但它是调整一千个不同旋钮的一吨，对吧？哦，你把晶圆放在这个工具中。当你进行沉积时，你要改变腔室的压力。哦，你要改变流入的化学物质的混合。 **56:36** Which chemicals you're putting in, right? Like what what speed you do it at. Do you do it for 30 minutes? Do you do it for 31 minutes? Do you do it for you know, obviously it splices way down. There's so many knobs on every single tool and you have a thousand input and process knobs, right? 你放入哪些化学物质，对吧？就像你以什么速度做。你做30分钟吗？你做31分钟吗？你做，你知道，显然它向下拼接。每个工具上有这么多旋钮，你有一千个输入和流程旋钮，对吧？ **56:55** Process knobs on each tool plus it's like the sequence of them all and and so like you frankly cannot test everything, right? It's impossible. It's it's too large of a search space just like just like designing a chip is too large of a search space. You have 100 trillion transistors. How you going to possibly try every single thing? Impossible, right? 每个工具上的流程旋钮加上它就像它们所有的序列，所以像你坦率地不能测试一切，对吧？这是不可能的。搜索空间太大，就像设计芯片的搜索空间太大一样。你有100万亿个晶体管。你怎么可能尝试每一件事？不可能，对吧？ **57:07** You just have to have enough intuition like pick that point, pick that point, pick that point, see the data. Oh, okay. I think the answer is here, right? And then just yolo, right? Um and and you know obviously once you you think the answer is here you test here and you're like okay here but like a different person might have seen these three and then said okay the answer is actually here not here 你只需要有足够的直觉，比如选择那个点，选择那个点，选择那个点，看数据。哦，好的。我认为答案在这里，对吧？然后只是冒险，对吧？你知道，显然一旦你认为答案在这里，你在这里测试，你就像好的在这里，但就像一个不同的人可能看到了这三个，然后说好的答案实际上在这里而不是这里 **57:26** and like the data is like fuzzy it's like somewhere in the center but like you know it's like this this whole like idea of like ML research is you spend a lot of time on compute training doing what effectively were useless things besides teaching yourself what's the right thing to do and what's the wrong thing to do and and semiconductor manufacturing is the same way 就像数据是模糊的，就像在中心某处，但你知道这整个ML研究的想法是你花很多时间在计算训练上做什么有效的无用的事情，除了教你自己什么是正确的事情和什么是错误的事情，半导体制造也是一样的 **57:43** and actually all process Process manufacturing is the same way. If you're iterating super fast and you're trying to get better and better and better or you're optimizing a process on a chemistry or whatever it is, you you try, you fail, you learn, you do. In semiconductor manufacturing, maybe it's just running tens of thousands of wafers. 实际上所有流程制造都是一样的。如果你迭代得超级快，你试图变得越来越好，或者你在优化化学或其他什么的流程，你尝试，你失败，你学习，你做。在半导体制造中，也许只是运行数万个晶圆。 **58:02** And so your R&D cost of you know an of an Intel and and or or your your cost of your you know main fab that is running the R&D is very very high and it's producing zero economic value besides that it's teaching you how to do the next node which then you can deploy at volume and that is like like what actually makes the money. 所以你的研发成本，你知道，英特尔的，或者你的主要工厂运行研发的成本非常非常高，除了教你如何做下一个节点之外，它产生零经济价值，然后你可以大量部署，那就像实际上赚钱的东西。 ## Power Dynamics in the AI Ecosystem ## AI生态系统中的权力动态 **58:20** I want to go back to where all the way to where we started and and ask about what I'll call like the like the wellspring or the fountain of power in this whole ecosystem. So I want to understand how you think about who has the power and how to keep or generate power as a business. 我想回到我们开始的地方，问关于我称之为这整个生态系统中权力的源泉或喷泉的东西。所以我想了解你如何看待谁拥有权力以及作为企业如何保持或产生权力。 **58:32** I mean um because it seemed like talent maybe talent is like the very beginning of the chain and he who has the talent like on a long enough timeline has the power or something like that but also there's structural stuff like just the scale the industrial scale of some of these things which just takes forever to build or whatever. 我的意思是，因为似乎人才也许人才就像链条的开始，在足够长的时间线上拥有人才的人拥有权力或类似的东西，但也有结构性的东西，就像其中一些东西的规模、工业规模，这需要永远建造或其他什么。 **58:45** How do you think about um even smaller zoomed in examples like okay cursor is unbelievably popular the revenue is insane um so much of it goes back to anthropic like who is the power in that relationships how does that dynamic change over time 你如何看待甚至更小的放大例子，比如好的Cursor非常受欢迎，收入疯狂，其中很多回到Anthropic，那个关系中的权力是谁，那个动态如何随时间变化 **58:59** it just seems like the power dynamics are so um so fascinating in this world and I'm curious where you think it like kind of comes from in the first place like where where it exists today and where it will go in the future when we think about like the power structures right like you mentioned a really interesting one does anthropic hold all the cards in this cursor relationship. 似乎这个世界中的权力动态如此令人着迷，我很好奇你认为它从一开始就像是从哪里来的，就像它今天存在于哪里，未来它会去哪里，当我们想到权力结构时，对吧，就像你提到的一个非常有趣的，Anthropic在这个Cursor关系中掌握所有牌吗？ **59:14** Cursor has like I don't know like nearly a billion dollars of revenue now on a on a on if you do current month times 12. That's a ton. But like again like their margins are what they are and they're sending most of it back to Anthropic. You know some people say their margins may be negative. I think they're slightly positive but regardless they're sending most of it back to anthropic. Cursor有，我不知道，现在几乎有10亿美元的收入，如果你做当前月乘以12。那是一吨。但就像他们的利润率是什么，他们把大部分送回Anthropic。你知道有些人说他们的利润率可能是负的。我认为它们略微为正，但无论如何，他们把大部分送回Anthropic。 **59:31** The gross profit dollars are at Enthropic right now. And but then Enthropic is taking anthropic is taking all the gross profit dollars and putting them into compute y for training. Yep. So then all those gross profit dollars are going to like Jensen laughing hilariously. 毛利润现在在Anthropic。但Anthropic正在采取，Anthropic正在采取所有毛利润并将它们投入训练的计算中。是的。所以所有那些毛利润都流向了Jensen，他在疯狂地笑。 **59:44** Well, maybe Jensen or maybe like Amazon who's then sending it to like uh or or or or Google who's sending it to Broadcom, right? Like the gross profit dollars are going to the hardware layer um from all of this for sure. But like does anthropic have all the power? Like you know like the common view is yes from a lot of people but then it's like well but anthropic only makes the model that's generated the code. 嗯，也许Jensen或者也许像亚马逊然后把它发送给，或者谷歌把它发送给Broadcom，对吧？就像毛利润肯定从所有这些流向硬件层。但就像Anthropic拥有所有权力吗？就像你知道很多人的普遍看法是肯定的，但就像嗯，但Anthropic只制造生成代码的模型。 **1:00:09** There's a lot more in this system, right? cursor gets all of the data. Um they get all of the users. They get how do they interact with this. Enthropic doesn't get that. They get a prompt. They send a response. Prompt response. Now, now they have cloud code which is like taking share. 这个系统中还有很多，对吧？Cursor获得所有数据。他们获得所有用户。他们如何与此互动。Anthropic没有得到那个。他们得到一个提示。他们发送一个响应。提示响应。现在，他们有云代码，就像占据份额。 **1:00:20** Um and it's very different than cursor. But like you know anyways like they get prompt response and then like cursor is like oh well I'm training embedding models on your code database and I have there's actually multiple models that I've made. I've made the embedding model. I've made the autocomplete model. I've made, you know, oh, I can switch the enthropic model to open the eye model whenever I want to. 它与Cursor非常不同。但就像你知道无论如何他们得到提示响应，然后Cursor就像哦嗯我正在你的代码数据库上训练嵌入模型，实际上我制作了多个模型。我制作了嵌入模型。我制作了自动完成模型。我制作了，你知道，哦，我可以随时将Anthropic模型切换到OpenAI模型。 **1:00:39** I'm only using the entropic model because it's the best one. Oh, and because I have all this data, maybe I can train a model not for everything better than you, but for the segment better than you. And so it's like the power dynamics are, you know, it's weird. It's it's it's their frenemies, right? Everyone's a friend, 我只使用Anthropic模型，因为它是最好的。哦，因为我拥有所有这些数据，也许我可以训练一个模型，不是在所有事情上都比你好，但在细分市场上比你好。所以权力动态是，你知道，很奇怪。这是他们的亦敌亦友，对吧？每个人都是朋友， ## Microsoft and OpenAI: A Shifting Power Balance ## 微软和OpenAI：不断变化的权力平衡 **1:00:56** right? Same as with Open Eye and Microsoft, the most crazy power dynamic that's going on in the world. um where they signed aou that said they had an understanding of like what the deal would actually be for them converting to for-profit. Like what is going on here? Like this this sounds like the most non-announcement announcement ever. 对吧？就像OpenAI和微软一样，世界上最疯狂的权力动态。他们签署了一份谅解备忘录，说他们了解他们转为营利性的交易实际上是什么。这里发生了什么？这听起来像有史以来最不像公告的公告。 **1:01:12** The power dynamics of this all it's it's the most fascinating soap opera ever, right? Like uh there were one of my friends was telling me about like K-pop Demon Hunters. I don't know if you've heard of this. I haven't done a nine-year-old daughter, so it's all it's all I hear about. 所有这些的权力动态，这是有史以来最迷人的肥皂剧，对吧？就像我的一个朋友告诉我关于K-pop恶魔猎人的事。我不知道你是否听说过这个。我有一个9岁的女儿，所以这就是我听到的一切。 **1:01:23** You've seen it a lot. I I had just heard about it and they're like, "Oh, let's watch it." I'm like, "What? Whatever." And but like there's drama. There's like But like this this real world power drama is way cooler than this. Like at least for you and I. 你看到了很多。我刚听说它，他们就像，"哦，让我们看它。"我就像，"什么？无论如何。"但就像有戏剧。有像但就像这个现实世界的权力戏剧比这酷得多。至少对你和我来说。 **1:01:35** Which parts of the drama interest you personally the most? Like what where do you think the stakes are the highest in in the various like subplots? The Microsoft opening I1 is absurdly interesting because at one point right like 2023 it was like Microsoft's going to own the world. Yeah. Right. 2024 a lot of it too. 戏剧的哪些部分对你个人来说最感兴趣？你认为在各种次要情节中赌注最高的地方在哪里？微软OpenAI是荒谬地有趣，因为在某个时候，对吧，就像2023年，就像微软将拥有世界。是的。对。2024年也有很多。 **1:01:53** And then like H2 2024 Microsoft backed down a lot. Right. They pulled back because because uh Amy Hood and and whoever else at Microsoft, Mipundar, whoever were like, "Maybe we don't need to be on the hook for a $300 billion. We're not going to build out $300 billion worth of compute for Open AI." 然后就像2024年下半年微软退缩了很多。对。他们退缩了，因为Amy Hood和微软的其他人，Satya Nadella，任何人都说，"也许我们不需要承担3000亿美元。我们不会为OpenAI建造价值3000亿美元的计算。" **1:02:05** Like that's they can't pay for it. Yeah. Right. It was like like was at least had to go through their head when they cut back. And so they paused a bunch of data centers, right? And they said, "Oh, you know, we don't need to be the exclusive comput provider. You can go to Oracle. It's fine." Right? 就像他们无法支付。是的。对。至少当他们削减时必须经过他们的头脑。所以他们暂停了一堆数据中心，对吧？他们说，"哦，你知道，我们不需要成为独家计算提供商。你可以去Oracle。没关系。"对吧？ **1:02:17** Like and they like relinquish this power, right? Now Oracle has that deal. But then like OpenAI sends like 20% of their revenue to Microsoft or API revenue or something like this. And then you know they have Microsoft has this like 49% capped profit structure on OpenAI and then there's like this whole like IP sharing like this deal like you you it's like really hard to understand the mechanics of the OpenAI Microsoft deal even. 就像他们放弃了这种权力，对吧？现在Oracle有那笔交易。但就像OpenAI向微软发送大约20%的收入或API收入或类似的东西。然后你知道微软在OpenAI上有这个像49%的利润上限结构，然后有这整个像IP共享这样的交易，就像你你，就像真的很难理解OpenAI微软交易的机制。 **1:02:42** Um so you have this like whole power dynamic and they're trying to renegotiate this like OpenAI doesn't want and the whole deal is like oh one we have AGI you no longer have API rights or IP rights and it's like [ __ ] does that mean? Right? Like if you ask someone 20 years ago and you put them in front of Chad GPT, you know, AGI like this is [ __ ] AGI. 所以你有这整个权力动态，他们试图重新谈判这个，OpenAI不想要，整个交易就像哦，一旦我们有AGI，你不再拥有API权利或IP权利，这到底是什么意思？对吧？就像如果你20年前问某人，你把他们放在ChatGPT面前，你知道，AGI就像这就是该死的AGI。 **1:03:01** Like it knows everything and it could have a conversation. I can't tell it's not a human. Actually, I can tell it's way smarter than a human. Yeah. But now it's like ah whatever. I can't do XYZ. So the thing the bar always moves no matter what the level of intelligence is. 就像它知道一切，它可以进行对话。我分不出它不是人类。实际上，我可以说它比人类聪明得多。是的。但现在就像啊无论如何。我不能做XYZ。所以无论智能水平是什么，标准总是在移动。 **1:03:11** And for me it's not going to it's going to be like when the thing puts its hand in its mouth and it's like yeah, this is me cuz I'm a human. Right? Like you know that's sort of like the sentience the consciousness of it all. Right? That's one power dynamic that's like crazy what's going on there. 对我来说，它不会是，它会像当东西把手放进嘴里时，就像是的，这是我，因为我是人类。对吧？就像你知道那有点像它的意识、知觉。对吧？这是一个权力动态，就像疯狂地在那里发生什么。 ## Nvidia's Dominance and Balance Sheet Strategy ## Nvidia的主导地位和资产负债表策略 **1:03:24** U another power dynamic is the one around Nvidia and the hyperscalers, right? Nvidia is the king. All of the gross profit is going to them today, right? Pretty much all of it. Sure, TSMC makes some, sure, SKH makes some, but they have to invest a ton in capex. Sure, Broadcom makes a bunch and you know, Broadcom makes a ton of gross profit off of these companies, but like Nvidia makes by by far the most grow gross profit in the industry and it's not even close. 另一个权力动态是围绕Nvidia和超大规模计算提供商的，对吧？Nvidia是王者。所有毛利润今天都流向他们，对吧？几乎所有。当然，台积电赚一些，当然，SK海力士赚一些，但他们必须在资本支出上投资大量。当然，Broadcom赚一堆，你知道，Broadcom从这些公司赚大量毛利润，但Nvidia远远在行业中赚最多的毛利润，甚至不接近。 **1:03:52** And so going back to like the analogy of like well they're king and they want to continue to be king and they want to make sure GPUs continue to be most used, but also like they can't buy anything. Like they can't buy any companies. They weren't even allowed to buy ARM when they were like a nobody, right? 所以回到类比，就像嗯他们是王者，他们想继续成为王者，他们想确保GPU继续被最多使用，但也像他们不能买任何东西。就像他们不能买任何公司。当他们像无名小卒时，他们甚至不被允许购买ARM，对吧？ **1:04:03** I don't I don't mean nobody, but they weren't like they were like pretty much a nobody on the grand scheme of things and they weren't allowed to buy ARM, you know, in like 2020 or 2021, whatever the time frame was. They totally could not buy any major companies. Um, you know, they'll buy smart startups like you know, I bought a startup that I was like a seed investor in and like an adviser in like all these things like recently, but like they they can't buy a real company. 我不是说无名小卒，但他们不像，在大局中他们像几乎是无名小卒，他们不被允许购买ARM，你知道，在2020年或2021年左右，无论时间框架是什么。他们完全不能购买任何大公司。你知道，他们会购买聪明的初创公司，就像你知道，我购买了一家我是种子投资者和顾问的初创公司，像所有这些东西，最近，但他们不能购买一家真正的公司。 **1:04:28** So, what do they do with all this cash flow? And like, sorry, but you're a loser if you just do buybacks. Like that just that's admitting that's admitting you can't get higher returns. Yep. On your capital. On your capital, which is fine. Like, you know, Meta, Apple, Google. They were mature companies for a while. 那么，他们用所有这些现金流做什么？就像，对不起，但如果你只是进行回购，你就是失败者。就像那只是承认你无法获得更高的回报。是的。在你的资本上。在你的资本上，这很好。就像，你知道，Meta、苹果、谷歌。他们有一段时间是成熟的公司。 **1:04:45** Guess what? They're gonna those companies aren't going to do buybacks ever [ __ ] again, right? Or not like ever again, but for a while cuz like they have way more they think there's better ROI for their capital now. And Nvidia, like you know, if you look at Jensen, he's like he's like he's always like flirted with buybacks, but like mostly he's been like reinvesting in the business 猜猜怎么着？他们那些公司永远不会再做回购了，对吧？或者不像永远，但有一段时间，因为他们认为现在他们的资本有更好的投资回报。而Nvidia，就像你知道，如果你看Jensen，他就像他一直像调情回购，但主要他一直在重新投资业务 **1:05:02** but you can't reinvest that much into the business. So like how do you He's doing demand guarantees. He's doing like all this crazy stuff now. Yeah. Right. Right. He's using his balance sheet to win. Yeah. Try and win more. Right. Um which which is an interesting uh dynamic. 但你不能在业务上重新投资那么多。所以你如何，他正在做需求保证。他现在正在做所有这些疯狂的事情。是的。对。对。他正在使用他的资产负债表来获胜。是的。试图赢得更多。对。这是一个有趣的动态。 **1:05:16** I don't know if there's ever been anything like this in terms of the non non anti-competitive nature of this, right? Like where you backs stop clusters so that you know like Cor recently got a deal with Nvidia where it was like they backs stopped a cluster right now core would have never built this cluster because it's for like short-term demand and renting GPUs on short-term is like a terrible business model, right? 我不知道在这方面是否有过这样的非反竞争性质，对吧？就像你在哪里支持集群，所以你知道像CoreWeave最近与Nvidia达成了一项交易，就像他们支持了一个集群，现在CoreWeave永远不会建立这个集群，因为它是为了短期需求，短期租用GPU就像一个糟糕的商业模式，对吧？ **1:05:32** You want to do long-term contracts. you've and you want to do long-term contracts to people with balance sheets. That's the golden like goose, but that doesn't exist so much. So, you do long-term contracts of people who don't have a balance sheet like OpenAI. And if you can't do that, then you'll do, you know, short contracts with people who don't have a you who do have a balance sheet, right? 你想做长期合同。你想与拥有资产负债表的人做长期合同。这就是金鹅，但那并不太存在。所以，你与像OpenAI这样没有资产负债表的人做长期合同。如果你不能那样做，那么你会做，你知道，与拥有资产负债表的人做短期合同，对吧？ **1:05:44** Like there's this whole matrix of like who you rent GPUs to. But from Nvidia's interest, it's like, you know what I really love is when venture capitalists fund a company and then 70% of their round is spent on compute. They [ __ ] love that, right? And that's what's happening with all these companies like these and and like it's like whether it's physical intelligence who you know they're spending a lot on like robot arms and [ __ ] too but they're also spending a lot of compute 就像有这整个矩阵，你向谁租用GPU。但从Nvidia的利益来看，就像，你知道我真正喜欢的是当风险投资家资助一家公司，然后他们一轮的70%花在计算上。他们该死地喜欢那个，对吧？这就是所有这些公司发生的事情，就像无论是Physical Intelligence，你知道他们在机器人手臂和废话上花费很多，但他们也花费很多计算 **1:06:04** or it's like you know any other startup that's raising cursor whoever right and even if it's not directly it's indirectly going to GPUs um they love they love when people spend their entire round on a GPUs would be really good is if it wasn't like a two-year deal or three-year deal for that compute if it was oh yeah yeah you can spend 70% of your round on one training run. 或者就像你知道任何其他正在筹集资金的初创公司，无论是谁，对吧，即使不是直接的，间接地也流向GPU，他们喜欢当人们把整轮花在GPU上时，真的很好，如果不是两年或三年的计算协议，如果是哦是的，你可以在一次训练运行上花费你一轮的70%。 **1:06:27** You know, leave a company with these ideas, gather the data, do the training run, and then you have a product and like you try and you show how good the model is, then you try and raise again. That's what would be really great for Nvidia, but no one wants to build a cluster who's predicated on that as the business model. That's crazy. 你知道，带着这些想法离开公司，收集数据，进行训练运行，然后你有一个产品，你尝试，你展示模型有多好，然后你尝试再次筹集资金。这对Nvidia来说真的很棒，但没有人想建立一个以此为商业模式的集群。这太疯狂了。 **1:06:38** So, they have to back stop a cluster to do that. Or, hey, you know, OpenAI might go to, you know, their own chip. They might go to um some ASIC from another company, right? They they they might even buy TPUs. There's a there, you know, they might even like go to Amazon, right? Like they don't really care. 所以，他们必须支持一个集群来做到这一点。或者，嘿，你知道，OpenAI可能会去，你知道，他们自己的芯片。他们可能会去另一家公司的某些ASIC，对吧？他们甚至可能购买TPU。他们，你知道，他们甚至可能去亚马逊，对吧？就像他们真的不在乎。 **1:06:56** They're not beholdened to Microsoft anymore, trying to serve a product to a customer. Yeah. And they want to build the digital god and they want to serve a product, right? Make revenue, right? So they don't have to go to Nvidia. Nvidia is the best option. But you know, it' be really really helpful is if I could, you know, going back to the earlier part in this this in this discussion is if the first year I get the compute up front and I don't have to pay for the compute for the first year, right? 他们不再受制于微软，试图向客户提供产品。是的。他们想建立数字上帝，他们想提供产品，对吧？赚取收入，对吧？所以他们不必去Nvidia。Nvidia是最好的选择。但你知道，真的非常有帮助的是，如果我能，你知道，回到这次讨论中早期的部分，如果第一年我预先获得计算，我不必在第一年支付计算费用，对吧？ **1:07:18** like I was mentioning, you know, the $10 billion for the So, it's like it'd be really good if I could do that because then I can for a full year I can do training. I can I can subsidize inference. I can do all these things that build up a user base and then I can and then I can actually pay for it. 就像我提到的，你知道，100亿美元的。所以，如果我能那样做会真的很好，因为那样我可以在整整一年里进行训练。我可以补贴推理。我可以做所有这些建立用户群的事情，然后我可以，然后我实际上可以支付它。 **1:07:30** I have a year of a gigawatt to figure out a business model, right? Whether that is serving free tokens and then implementing uh this purchasing right of of you know purchasing stuff for the free user or it's and and so a lot of that is like almost no fee initially purchasing and then and then like slowly rising the fee over time right or it's hey you know I have to serve this model at worse gross margins or negative gross margins initially but then eventually I can serve it at positive gross margins because the models keep getting cheaper 我有一年一千兆瓦的时间来弄清楚一个商业模式，对吧？无论是提供免费token，然后实施这个购买权，你知道为免费用户购买东西，或者是，所以很多是像最初几乎没有费用购买，然后随着时间慢慢提高费用，对吧，或者是嘿你知道我必须以更差的毛利率或最初负毛利率提供这个模型，但最终我可以以正毛利率提供它，因为模型不断变得更便宜 **1:08:00** or it's I train the next generation model that's so much better than everyone else and then I'll win all the business for that level of intelligence because I'm the only one with a 18-year-old. You guys all have 14-year-olds, right? Like, you know, who are working for you. So, it's like, you know, this is this is a they can do whatever they want with this allocation. 或者是我训练下一代模型，比其他人好得多，然后我会赢得那个智能水平的所有业务，因为我是唯一拥有18岁的人。你们都有14岁的，对吧？就像，你知道，为你工作的人。所以，就像，你知道，这是这是一个，他们可以用这个分配做任何他们想做的事。 **1:08:11** It's not an allocation of capital, per se. Allocation of compute. They get to decide what they'd allocate that compute to. And Nvidia's helping them by effectively frontloading it if they can find a capital, you know, and and and that company's like, "Oh, yeah, yeah, Nvidia's backing this, too. Oh, you know, there's all these other things." 这不是资本的分配本身。是计算的分配。他们可以决定将那个计算分配给什么。如果他们能找到资本，Nvidia通过有效地预先加载来帮助他们，你知道，那家公司就像，"哦，是的，是的，Nvidia也在支持这个。哦，你知道，还有所有这些其他东西。" **1:08:29** It's much more reasonable for someone to say, "Oh, yeah. I'll back I I'll pay the capex because I know the first year is already going to be paid because you've got that investment from Nvidia. What about the next four years?" If you ask a bunch of investors who are students of economic cycles through history, like Carlo Perez type stuff, they'll say that the concern is that every shortage is followed by a glut 对某人来说，说"哦，是的。我会支持，我会支付资本支出，因为我知道第一年已经会被支付，因为你从Nvidia获得了那笔投资。接下来的四年怎么办？"更合理。如果你问一群研究历史经济周期的投资者，像Carlota Perez类型的东西，他们会说担心是每次短缺之后都会有过剩 **1:08:49** and we always overbuild on long lead time big capex projects and you've got multi-gigawatt, you know, power being installed. You've got all this crazy stuff in semiconductors and like at some point like it just gets overbuilt. All the stuff we talked about earlier feels like we're not really close to that. Like there's so much freaking demand. 我们总是在长交货期大资本支出项目上过度建设，你有多千兆瓦，你知道，正在安装的电力。你在半导体方面有所有这些疯狂的东西，就像在某个时候就会过度建设。我们之前谈到的所有东西感觉我们真的不接近那个。就像有这么多该死的需求。 **1:09:03** If the models don't improve, yes, we will overbuild, right? Like it's pretty simple. It's like yes, there will be like supply chain things where switches from one supplier to another and like that's a lot of the stuff that we like nitty-gritty stuff we focus on. At the end of the day, if the models don't improve, we're absolutely screwed. 如果模型不改进，是的，我们会过度建设，对吧？就像很简单。就像是的，会有供应链的东西，从一个供应商转到另一个，那是我们喜欢关注的很多细节的东西。归根结底，如果模型不改进，我们就完全完蛋了。 **1:09:22** In fact, the US like you know in another year if if this lasts another year and then it happens like the US economy will go into recession like straight up because of this and probably Taiwan as well and probably Korea as well, right? Because there's so much buildup and revenue flowing through to us for this, right? 事实上，美国，你知道，再过一年，如果这再持续一年，然后它发生，美国经济将因此直接陷入衰退，可能台湾也是，可能韩国也是，对吧？因为有这么多积累和收入因此流向我们，对吧？ **1:09:35** But you know when you look at these other things like the bubbles of the past, some of them were just silly nonsense, right? Like tulips, silly nonsense, right? Crypto complete Ponzi scheme, right? But then there's other stuff that's like no, this was real, right? like the the UK like spent like some absurd percentage of their GDP on railroads for like a decade. 6% or something crazy. 但你知道当你看这些其他东西，像过去的泡沫，其中一些只是愚蠢的废话，对吧？就像郁金香，愚蠢的废话，对吧？加密货币完全是庞氏骗局，对吧？但还有其他东西就像不，这是真实的，对吧？就像英国在铁路上花费了他们GDP的一些荒谬的百分比，大约十年。6%或类似的疯狂。 **1:09:53** Yeah, we're nowhere close to GD 6% of our GDP. Like holy [ __ ] Um but like that was like okay there's tangible but it's like oh well we over did overbuild because like how many goods are there to transport? But like also you must like reduce you must build these railroads to reduce the cost of transport so much because you have no clue when the demand stops and you've overbuilt and because there's 10 people trying to do it at once you're obviously going to overbuild at some point. 是的，我们离GDP的6%还很远。就像天哪。但那就像好的，有有形的东西，但就像哦嗯我们过度了，过度建设，因为有多少货物要运输？但就像你也必须建设这些铁路以大幅降低运输成本，因为你不知道需求何时停止，你已经过度建设了，因为有10个人试图同时做，你显然会在某个时候过度建设。 **1:10:10** Um same thing with fiber. A lot of the argument against this is like, well, no, but this time it's the strongest balance sheets in the world. It's the world's most profitable companies. They can all pull the plug at any point. Yeah. Microsoft pulled the plug at one point before they're like, oh [ __ ] no, no, plug it back in. Right. 同样的事情发生在光纤上。很多反对这个的论点是，嗯，不，但这次是世界上最强大的资产负债表。是世界上最赚钱的公司。他们都可以随时拔掉插头。是的。微软曾经拔掉插头，然后他们就像，哦该死不，不，把它插回去。对。 **1:10:27** They recently plugged it back in. They're like, "Oh, wait. We're starting. We're restarting this. We're going out into the market. We're signing deals with uh Nebius for GPUs." Like, I don't remember how big the deal was. It's like 10 plus billion. Yeah. It's like 19 billion for Nebius. 他们最近把它插回去了。他们就像，"哦，等等。我们开始了。我们重新启动这个。我们进入市场。我们与Nebius签署GPU协议。"就像，我不记得协议有多大。就像超过100亿。是的。Nebius是190亿。 **1:10:39** It's like, well, if they had just not pulled the plug on their data centers, they wouldn't have had to do that. they wouldn't have to pay those gross profit dollars to Nebius, right? But, you know, Nebius made the bet that the demand is there and they were right. Um, and so, you know, when you think about this, it's like, what is the level of demand where this stops, right? 就像，嗯，如果他们只是没有拔掉他们数据中心的插头，他们就不必那样做。他们不必向Nebius支付那些毛利润，对吧？但，你知道，Nebius打赌需求在那里，他们是对的。所以，你知道，当你想到这个，就像，这停止的需求水平是什么，对吧？ **1:10:57** If if scaling laws continue, right? How I mean, of course, there's a adoption curve, there's a pace, there's realities with capital, there's realities with supply chains, things take time. But if you like boil it down to it, it's like your demand for 30-year-old senior engineers at Google who know how to make and program anything is effectively like I don't I want to say infinite, but it's $2 trillion of value. Yeah. 如果扩展定律继续，对吧？我的意思是，当然，有一个采用曲线，有一个节奏，有资本的现实，有供应链的现实，事情需要时间。但如果你把它归结为，就像你对谷歌30岁高级工程师的需求，他们知道如何制作和编程任何东西，实际上就像我不想说无限，但它是2万亿美元的价值。是的。 **1:11:16** Right. If I could have an intelligence as smart as a Google senior engineer, that's $2 trillion of software value, right? Because that's how much I pay the world pays to software engineers today. Um and you just go down the list of every other use case, right? 对。如果我能有一个像谷歌高级工程师一样聪明的智能，那就是2万亿美元的软件价值，对吧？因为这就是我今天支付给世界软件工程师的金额。你只需列出每个其他用例，对吧？ **1:11:32** If you have just a simple, you know, phys physical intelligence robot that can do this, that can recognize headphone versus water and pick up or versus phone, right? And pick up the right thing and manipulate it properly and put it in the right spot and sort it, that's worth how much to the distribution uh supply chain, right? Like I I don't know, but a lot, right? 如果你只有一个简单的，你知道，物理智能机器人可以做这个，可以识别耳机与水并拿起或与电话，对吧？并拿起正确的东西并正确操作它并把它放在正确的位置并分类它，那对配送供应链值多少，对吧？就像我不知道，但很多，对吧？ **1:11:44** So, it's like there's you we don't need to get digital god for there to be immense value. But the interesting thing here is that, you know, it's like human capital, capital goods. All of these other revolutions have been human have been capital goods that reduce the amount of human capital you need. Whereas this is just creating human capital, right? In a sense. 所以，就像我们不需要达到数字上帝就有巨大价值。但这里有趣的是，你知道，就像人力资本、资本货物。所有这些其他革命都是减少你需要的人力资本的资本货物。而这只是创造人力资本，对吧？在某种意义上。 **1:12:01** In a sense, right? If I like sort of get everyone bowled up, right? And we're we're on this podcast, right? You know, there's like this I don't know if you've like uh heard the curse, right? It's like if you talk about the stock on in on this podcast, it goes down, right? I've heard word of it. 在某种意义上，对吧？如果我有点让每个人都激动起来，对吧？我们在这个播客上，对吧？你知道，有这个，我不知道你是否听说过这个诅咒，对吧？就像如果你在这个播客上谈论股票，它就会下跌，对吧？我听说过它。 **1:12:19** We went and did we went and did the math one time because I was sick of hearing about this [ __ ] curse and it's just market performance. It's not Oh, really? So, so, so I last time I was it wasn't your it wasn't this podcast, it was your other podcast. I talked about applied materials and the stock was up like 70% the six months after. I was like, there you go. Yeah, I broke the curse. I was like, hell yeah. 我们去做了，我们做了一次数学，因为我厌倦了听到这个该死的诅咒，这只是市场表现。不是，哦，真的吗？所以，所以，所以我上次不是你的，不是这个播客，是你的另一个播客。我谈到了应用材料，股票在六个月后上涨了70%。我就像，你看。是的，我打破了诅咒。我就像，太好了。 ## The Middle Layer ## 中间层 **1:12:37** What do you think about all the companies in the middle? We've talked a lot about Nvidia and then like people at the end serving applications. What about these companies like together and base 10 and fireworks and you mentioned Nebius like all these interesting middle middle layer players are there amazing businesses to be built there do you think are they temporary patchwork to make the system work and serve you know serve demand like what do you think about that this middle layer 你如何看待所有中间的公司？我们已经谈了很多关于Nvidia的，然后像最后提供应用程序的人。这些公司呢，比如Together、Baseten、Fireworks，你提到的Nebius，所有这些有趣的中间层玩家，你认为那里有令人惊叹的业务要建立吗，它们是临时的拼凑来使系统工作并服务，你知道，服务需求，你如何看待这个中间层 **1:12:54** the cloud business model right like uh let's let's say neocloud business model so there's you sort of you mentioned inference providers and neoclouds um the neocloud business model is absolutely amazing or terrible depending on how you do it right it's terrible if you like sign short-term contracts and you just hope and pray you have short-term contracts forever and actually initially your short-term profits have amazing cash flows, right? 云商业模式，对吧，让我们说新云商业模式，所以你有点提到了推理提供商和新云，新云商业模式绝对惊人或糟糕，取决于你如何做，对吧，如果你签短期合同并且只是希望和祈祷你永远有短期合同，这很糟糕，实际上最初你的短期利润有惊人的现金流，对吧？ **1:13:22** Because you're selling it at like you you bought a GPU and you put it in a data center and the power and all that. The cost per hour over a six-year period for Blackwell is $2. Let's just call it for simplicity sake, it's $2. It's not exactly that, but if I sold it for 6 months, I could get like north of I could get like 350 or $4. 因为你以像你买了一个GPU并把它放在数据中心和电力和所有那些的价格出售。Blackwell六年期间每小时的成本是2美元。让我们为了简单起见称它为2美元。不完全是那样，但如果我卖了6个月，我可以得到超过，我可以得到像350或4美元。 **1:13:34** Like, holy [ __ ] that margin's insane. But what happens two years from now, three years from now when I'm still selling six-month contracts or one-mon contracts and the next generation of Nvidia chip is out or and it's 10x faster for 3x the cost, right? Okay. So now, you know, naturally the price of this should tank. 就像，天哪，那个利润率疯了。但从现在起两年、三年后会发生什么，当我仍在销售六个月合同或一个月合同，而Nvidia的下一代芯片已经推出，或者它快10倍，成本是3倍，对吧？好的。所以现在，你知道，自然这个的价格应该暴跌。 **1:13:46** The other way to do it is is hey, I actually like have a long-term contract of well, I'm selling to opening I'm selling to micro the other end of the spectrum is what Nebius just signed. I'm signing [ __ ] $19 billion to Microsoft. They will stay no matter what. The market literally believes Microsoft will pay its obligations before the US government 另一种做法是嘿，我实际上有一个长期合同，嗯，我在卖给OpenAI，我在卖给微软，光谱的另一端是Nebius刚刚签署的。我正在签署该死的190亿美元给微软。无论如何他们都会留下。市场字面上相信微软会在美国政府之前支付其义务 **1:14:03** because it's like literally a cheaper bond rate, right? Which is like insane to me, but whatever. This $19 billion has like a huge gross profit cuz the price per hour and and it's not exactly $3 and it's not $2, but like the margins here are like really good. Like Nebius is going to make like at least $6 billion of gross profit off of this. 因为它就像字面上更便宜的债券利率，对吧？这对我来说太疯狂了，但无论如何。这190亿美元有巨大的毛利润，因为每小时的价格，不完全是3美元，不是2美元，但这里的利润率真的很好。就像Nebius将从中赚取至少60亿美元的毛利润。 **1:14:28** Um, and then obviously they have their operational cost, but like like $6 billion of gross profit off of this deal is like insane. So like I would do that all day and Cororeweave did until Microsoft stopped going to Cororeweave, right? Um and but like Cororee's turned around and they found other customers and all these things, right? Selling to Google and selling to OpenAI. 然后显然他们有运营成本，但就像这笔交易的60亿美元毛利润太疯狂了。所以我会整天这样做，CoreWeave也这样做，直到微软停止去CoreWeave，对吧？但CoreWeave转身找到了其他客户和所有这些东西，对吧？卖给谷歌和卖给OpenAI。 **1:14:47** Oh, but now Oh, OpenAI is definitely not like a real, you know, you can't rely on their balance sheet. Okay, I still have amazing margins when I sell to OpenAI, but they don't have a balance sheet. So how can I be sure that they're actually going to pay the thing that they've signed up to? Right? 哦，但现在哦，OpenAI绝对不像一个真实的，你知道，你不能依赖他们的资产负债表。好的，当我卖给OpenAI时我仍然有惊人的利润率，但他们没有资产负债表。那么我如何确定他们实际上会支付他们签署的东西？对吧？ ## The Risk Spectrum ## 风险范围 **1:15:03** So, so you know, I know in theory this contract is worth a ton of money and in Cory's books today are that that contra all the contracts they've signed are mostly Microsoft, mostly money in the bank, right? But the open egg contracts like what if they can't afford to pay for this? Okay, now there's like there's a bigger risk and there's a longer and longer tale of like these businesses. 所以，你知道，我知道理论上这份合同值一大笔钱，在CoreWeave今天的账簿上，他们签署的所有合同主要是微软，主要是银行里的钱，对吧？但OpenAI合同就像如果他们负担不起支付这个怎么办？好的，现在有更大的风险，有越来越长的尾巴，就像这些业务。 **1:15:23** So like yeah, you absolutely can make a ton of money. Um, you know, there have been more recent deals with crypto miners, Google, and Fluid Stack because Google's really short on data center capacity. Um, people want to use more TPUs. um they can't they can't serve them all themselves. So, they're going to sell TPU systems to uh providers. 所以就像是的，你绝对可以赚一大笔钱。你知道，最近有更多与加密矿工、谷歌和Fluid Stack的交易，因为谷歌真的缺少数据中心容量。人们想使用更多TPU。他们不能自己为他们所有人提供服务。所以，他们将向提供商出售TPU系统。 **1:15:37** Um you know, Google Yeah, they're backstopping the deals with uh Terowolf is one of the companies. Um I can't remember the other one, but there's two companies they've signed deals with where they're backstopping the data center plus like selling the the TPUs physically to another company and then they're being deployed and then they're getting rented. 你知道，谷歌，是的，他们正在支持与Terawulf的交易，这是其中一家公司。我记不起另一家了，但有两家公司他们签署了协议，他们在支持数据中心，加上像物理地将TPU出售给另一家公司，然后它们被部署，然后它们被租用。 **1:15:49** Um and Google still makes all the money. like you know there's there's those companies you know yeah that's great as well but then there's a long tale of like is the enterprise demand there is the who's taking the risk right and it's like open taking the risk because they're betting their entire company could go bankrupt if it doesn't come 谷歌仍然赚所有的钱。就像你知道有那些公司，你知道，是的，那也很棒，但还有一个长尾巴，就像企业需求在那里，谁在承担风险，对吧，就像OpenAI在承担风险，因为他们在赌他们整个公司如果不来可能会破产 **1:16:06** Oracle's taking a risk because they're they're signing up for you know $300 billion of contract okay $200 billion of hardware spend across data centers and and chips and of that like they're going to have to go get debt right um so so they're on the hook and they they'll probably be able to pay for it if it happens but they'll just be like their their EV will like tank Oracle在承担风险，因为他们正在签署你知道3000亿美元的合同，好的，跨数据中心和芯片的2000亿美元硬件支出，其中他们将不得不获得债务，对吧，所以他们在承担，他们可能能够支付，如果它发生，但他们会像他们的企业价值会暴跌 **1:16:22** if opening I can't pay for all the hardware that they bought right and and luckily for them it phases in over time and whatever right but like and then you go to the inference providers and it's like yeah there is there's a business to be made here too right like I'm serving models maybe Roblox comes to me and they want to put an LLM in their in their game right because of XYZ reasons 如果OpenAI无法支付他们购买的所有硬件，对吧，幸运的是，它随着时间分阶段进行，无论如何，对吧，但就像然后你去推理提供商，就像是的，这里也有一个业务要做，对吧，就像我在服务模型，也许Roblox来找我，他们想在他们的游戏中放一个LLM，对吧，因为XYZ原因 **1:16:41** okay Roblox is a good customer or like hey you know another company like uh Shopify wants to put an LLM for customer service and you know yes they could do it themselves but actually there's inference is a hard thing especially as you get to larger and larger models and more complicated models and all the things 好的，Roblox是一个好客户，或者像嘿你知道另一家公司像Shopify想放一个LLM用于客户服务，你知道，是的，他们可以自己做，但实际上推理是一件难事，特别是当你得到越来越大的模型和更复杂的模型和所有的东西 **1:16:54** or you know there's all these different use cases um where people want to serve models and maybe it's maybe it's just open source models and maybe it's fine-tuning of those open source models which those companies can help you do or you can do and they can serve for you and they have scalable reliable capacity like it's like there's businesses to be made here 或者你知道有所有这些不同的用例，人们想服务模型，也许它只是开源模型，也许是那些开源模型的微调，那些公司可以帮你做或你可以做，他们可以为你服务，他们有可扩展的可靠容量，就像这里有业务要做 **1:17:13** but there's also like yolo I'm selling tokens to like random people who are trying to build SAS apps in NSF and maybe they run out of runway right and and and like okay that funding doesn't directly go to Nvidia, but you go through some steps and it's going to Nvidia after some value chain. Um, and and Nvidia's holding no risk. Everyone in the middle's got a lot of risk. 但也有像冒险我在向试图在NSF中构建SaaS应用的随机人出售token，也许他们用完了跑道，对吧，就像好的，那笔资金不直接流向Nvidia，但你经过一些步骤，它在一些价值链之后流向Nvidia。而Nvidia没有风险。中间的每个人都有很多风险。 **1:17:29** I'd love to hear your thoughts on like going back to the other side of the equation, the the app side, you know, stuff we're going to use these models to do at the significance of this switch from like deterministic code to a much different thing. And it seems like what we're doing is the thing we always do, you know, Apple used to call this like the schemorphic era where you just basically use the new technology to do the old thing you used to do. 我很想听听你对回到方程的另一边，应用方面的想法，你知道，我们将使用这些模型做的东西，从确定性代码切换到非常不同的东西的重要性。似乎我们正在做的是我们总是做的事情，你知道，苹果过去称之为拟物化时代，你基本上只是使用新技术来做你过去做的旧事情。 **1:17:55** So, we're making engineers better. You know, that would be like an an obvious current example, but we seems like we haven't yet gotten into the world where we're going to start using this this technology to do things that we couldn't do before with deterministic code. I'm curious how you think about like that side of like pushing the envelope. 所以，我们让工程师变得更好。你知道，那将是一个明显的当前例子，但似乎我们还没有进入我们将开始使用这种技术来做我们以前用确定性代码做不到的事情的世界。我很好奇你如何看待像推动极限的那一面。 ## AI for Material Science and Hard Tech ## 材料科学和硬科技的AI **1:18:06** Why is that? Like I feel like that's exactly what we do with it, right? Is like the cost to develop things is so high that you can't do it, right? like or the cost to like you know have someone go buy stuff for you. It's like, okay, great. You might have an executive assistant and you can tell them to like, you know, do this, but like the vast majority of people don't. 为什么是这样？就像我觉得这正是我们用它做的，对吧？就像开发东西的成本如此之高，你做不到，对吧？就像或者像你知道让某人为你买东西的成本。就像，好的，太好了。你可能有一个执行助理，你可以告诉他们，你知道，做这个，但绝大多数人没有。 **1:18:30** And now GPT is on the cusp of doing that, right? Go buy go do this, go buy this for me, and they'll find the best thing and they'll buy it, right? And you just trust them enough, right? Um it takes time to trust them, but like it's like these things are proliferating across, you know, the massive, you know, tech tech is the most deflationary thing in the world ever, right? 现在GPT即将做到这一点，对吧？去买，去做这个，为我买这个，他们会找到最好的东西并购买它，对吧？你只是足够信任他们，对吧？信任他们需要时间，但就像这些东西正在扩散到，你知道，庞大的，你知道，科技是世界上最具通货紧缩的东西，对吧？ **1:18:48** Uh in terms of quality of life, it's it's so it gets cheaper way faster than the revenues go up, right? But the revenues still go up. That's sort of like the fundamental basis of semiconductors, of tech, everything, right? Are we doing things that we can't do before with tech, with AI? Sure. I mean, like the COVID vaccine was like created with AI. 就生活质量而言，它变得便宜的速度比收入增长的速度快得多，对吧？但收入仍在增长。这有点像半导体、科技、一切的基本基础，对吧？我们正在用科技、用AI做我们以前做不到的事情吗？当然。我的意思是，COVID疫苗就像是用AI创造的。 **1:19:01** Like it was like AI drug discovery. Like you can there's like there's like entire briefs about like how it was done with AI. And guess what? If like another pandemic happened, I bet it'd be even faster to discover the vaccine if there's a vaccine for it or whatever, right? Like there's all these protein folding things. There's all these like optimization things. 就像AI药物发现。就像你可以有整个简报关于它是如何用AI完成的。猜猜怎么着？如果像另一场大流行发生，我敢打赌发现疫苗会更快，如果有疫苗或其他什么，对吧？就像有所有这些蛋白质折叠的东西。有所有这些优化的东西。 **1:19:23** There's AI for material science and AI for like, you know, all these other like aspects of society. There's optimization. Maybe it's not in your face, right? It's like, oh my god, AI just made this drug, right? It's like, no, I mean, AI worked with the researchers who made the COVID vaccine and so we didn't have to all like, you know, be stuck inside forever or whatever, right? 有材料科学的AI和AI用于，你知道，社会的所有这些其他方面。有优化。也许它不在你面前，对吧？就像，哦，天哪，AI刚刚制造了这种药物，对吧？就像，不，我的意思是，AI与制造COVID疫苗的研究人员合作，所以我们不必像，你知道，永远被困在里面或其他什么，对吧？ **1:19:40** Point being, we're it's already happening. And the whole like use the new thing to make the old thing faster. It's like sure, but like if I go back three years, how many people would it have taken to deploy a image recognition model that looks at every data center in the world and and looks at like what's the pace it's of constructions in and what equipment they have and like assuming this is something you do. 重点是，我们已经在发生了。整个像使用新事物使旧事物更快。就像当然，但就像如果我回到三年前，需要多少人来部署一个图像识别模型，查看世界上每个数据中心，并查看像建设的速度是多少，他们有什么设备，就像假设这是你做的事情。 **1:20:00** This is something we do, right? It's like it's like how many people would that have taken? I don't think it would have been possible. my business model like this is the second highest revenue product for us would not have been possible if it weren't for AI like vibe coding like you know being able to dig through permits and regulatory filings being able to run image recognition on satellite photos 这是我们做的事情，对吧？就像需要多少人？我不认为这是可能的。我的商业模式，这是我们第二高收入的产品，如果不是因为AI，就像氛围编码，就像你知道能够挖掘许可证和监管文件，能够在卫星照片上运行图像识别，这是不可能的 **1:20:25** like this would not be possible this business is not possible without AI and like am I using it directly like oh yeah sure I'm scraping through the regulatory filings and permits through with LLMs and then manually reviewing it with people and like you know and or like doing it doing that the same with like the the images, satellite images. 就像这不可能，没有AI这个业务是不可能的，就像我直接使用它吗，哦，是的，当然，我正在用LLM抓取监管文件和许可证，然后手动与人审查它，就像你知道，或者像对图像、卫星图像做同样的事情。 **1:20:36** Yes, there's a lot of stuff that you know the image recognition model does. We just also look at them a lot and then it's like compiling them and selling a spreadsheet that you get like bi-weekly reports on like all the data centers or what's changed or like hey actually this Amazon data center the fans are starting to spin so actually there's revenue going on from this Amazon data center so we can forecast Amazon's revenue 是的，有很多东西，你知道图像识别模型做。我们也经常查看它们，然后就像编译它们并出售一个电子表格，你得到像双周报告，关于所有数据中心或什么改变了，或者像嘿，实际上这个亚马逊数据中心的风扇开始旋转，所以实际上这个亚马逊数据中心正在产生收入，所以我们可以预测亚马逊的收入 **1:20:54** right it's like oh okay like this is like relevant right like you know but like I don't think this would have been possible just a few years ago um at least like off the proof right now and especially like you know there's demand for it because everyone wants to track this and it's so important but it's like it begets each other 对吧，就像哦好的，这是相关的，对吧，就像你知道，但就像我不认为这几年前是可能的，至少就像现在的证明，特别是你知道有需求，因为每个人都想追踪这个，这太重要了，但就像它相互促进 **1:21:05** and I think like at least in my daily life it's like I don't think I could have taken that step from where I was in a business which was still a research provider but like that is a monumental jump and like being able to do it with three people out of the gate versus like 50 or 100 like I don't know how many people it would have taken but I don't think it's possible 我认为至少在我的日常生活中，我不认为我本可以从我所在的业务迈出那一步，那仍然是一个研究提供商，但就像那是一个巨大的飞跃，就像能够用三个人从一开始就做到，与像50或100，我不知道需要多少人，但我不认为这是可能的 **1:21:24** and it's like mainframe migration is something people have always wanted to do Amazon leaving Oracle took [ __ ] 20 years right and they wanted to do it 20 years ago and they had their highest revenue products after EC2 were like the next four were database products at AWS and yet they still freaking used Oracle's database because it's hard. 就像主机迁移是人们一直想做的事情，亚马逊离开Oracle花了该死的20年，对吧，他们20年前就想这样做，他们在EC2之后的最高收入产品就像接下来的四个是AWS的数据库产品，但他们仍然该死地使用Oracle的数据库，因为这很难。 **1:21:42** Now there's like mainframe migration can be way faster or like migration from one tech stack to another can be way faster. You can make your business more efficient. You add more automation. Yes, the tech exists. Go to all the businesses around the world and it's like they aren't using the leading edge of what they could. They aren't using, you know, what a 2020 company could have done without AI, 现在就像主机迁移可以快得多，或者像从一个技术栈迁移到另一个可以快得多。你可以使你的业务更高效。你添加更多自动化。是的，技术存在。去世界各地的所有企业，就像他们没有使用他们可以使用的前沿技术。他们没有使用，你知道，一个2020年的公司在没有AI的情况下本可以做的， **1:22:06** right? No one is doing that. And if they did, they'd be so much more efficient, right? But like all of these things just take too long to build. They're too expensive to build. You have your existing processes. How do you hand them over? How do you switch them over? How do you teach people to do this? AIS can help you with all of this, right? 对吧？没人在做。如果他们做了，他们会高效得多，对吧？但就像所有这些东西只是需要太长时间来构建。它们构建起来太贵了。你有你现有的流程。你如何移交它们？你如何转换它们？你如何教人们做这个？AI可以帮你解决所有这些，对吧？ **1:22:17** So it's sort of like you can take the pessimistic view of like, oh, we're just doing the same things, but it's like the value here is humongous. If it's tokens on one end, we haven't talked much about like watts at the very beginning and power. 所以有点像你可以采取悲观的观点，哦，我们只是在做同样的事情，但就像这里的价值是巨大的。如果一端是token，我们在一开始还没有谈论太多关于瓦特和电力。 ## Building Infrastructure ## 建设基础设施 **1:22:28** What are your thoughts on like what is going on here and how like humanity is responding to this crazy new demand for just raw power? The first approximation is that like we're being a bunch of pansies and it's not that much power yet, right? Like AI data centers are like 3 4% of the US economy. Power, not economy. Um or just data centers period of that like two is regular data centers 你对正在发生的事情以及人类如何应对对原始电力的这种疯狂新需求有什么想法？第一近似是，我们是一群懦夫，还没有那么多电力，对吧？就像AI数据中心大约占美国的3-4%。电力，不是经济。或者只是数据中心，其中两个是常规数据中心 **1:22:51** and two is like AI data centers, you know, that's nothing, dude. Like that's literally nothing. Uh it's just we haven't built power in like 40 years, right? Or like we've transitioned from coal to natural gas more and more over 40 years. It's like so mostly we just don't know how to and there's these regulations and like there's not enough labor and like the supply chains for like Evanova and their dual combine cycle gas reactors are not there yet 两个就像AI数据中心，你知道，那什么都不是，伙计。那真的什么都不是。只是我们大约40年没有建造电力了，对吧？或者像我们在40年里越来越多地从煤炭过渡到天然气。所以主要是我们只是不知道如何，有这些法规，就像没有足够的劳动力，就像Enavon和他们的双联合循环燃气反应堆的供应链还不存在 **1:23:10** and same for like Mitsubishi and you know you you know oh like this random e UV curing process for transformer coils is like you know is like there's only this much capacity and it takes two years to build them. It's like it's just like it's a supply chain thing. It's a like lack of labor thing. It's not that it's like actually that much yet, but like at the end of the day, it's like, okay, wait, wait, you're telling me open is making a data center with 2 gigawatts 三菱也是一样，你知道，你知道，哦，像这个随机的变压器线圈的UV固化过程就像，你知道，就像只有这么多容量，建造它们需要两年。就像这只是一个供应链问题。这是一个像缺乏劳动力的问题。还没有那么多，但就像归根结底，就像，好的，等等，等等，你告诉我OpenAI正在建造一个2千兆瓦的数据中心 **1:23:31** and that's like the entirety of the power consumption of like Philadelphia. Like that is real. Yeah, that's insane. That's insane, right? But like we used to get like excited about finding like a couple hundred megawatts new data center. Now it's like if it's not a gigawatt like I I remember like the the the guy who leads that team, he he was like, "Oh, it's just 500 megawws, whatever." 那就像费城的整个电力消耗。那是真实的。是的，那太疯狂了。那太疯狂了，对吧？但就像我们过去会因为找到几百兆瓦的新数据中心而兴奋。现在就像如果不是一千兆瓦，我记得领导那个团队的家伙，他就像，"哦，只是500兆瓦，无论如何。" **1:23:50** I was like I like immediately opine I was like I I I also agreed immediately then afterwards I was like wait a second dude that's like a lot of power that's like how much wait 500 megawatts is $25 billion of capex like come on like once you put in the GPUs and everything right it's like that's a ton of money 我就像我立即发表意见，我就像我我我也立即同意，然后之后我就像等一下，伙计，那就像很多电力，那就像多少，等等，500兆瓦是250亿美元的资本支出，拜托，就像一旦你放入GPU和所有东西，对吧，那是一大笔钱 **1:24:02** but like snore because there's so many of it happening right we're learning how to build power again right um we're we're getting the the supply chains to do it again we're reshaping the grid uh there's all these challenges with these AI data centers that with regards the demand response and making grids unstable, right? 但就像打鼾，因为有这么多正在发生，对吧，我们正在学习如何再次建造电力，对吧，我们正在获得再次这样做的供应链，我们正在重塑电网，有所有这些AI数据中心的挑战，关于需求响应和使电网不稳定，对吧？ **1:24:26** Um, you know, you could, you know, AI workloads because they change so much so fast, especially training, you can just blow, you could just you can just cause like brownouts or blackouts. Um, especially if the grid doesn't have enough inertia or if you're not putting enough like things to dampen it in between the workload and and the grid. 你知道，你可以，你知道，AI工作负载，因为它们变化如此之快，特别是训练，你可以，你可以只是导致像停电或断电。特别是如果电网没有足够的惯性，或者如果你没有在工作负载和电网之间放置足够的缓冲。 **1:24:45** And even if it's not destroying it, uh, the grid runs at like 59 hertz or whatever, right? If you you skew it up and down too much, um you these transient power responses, your refrigerator will break down sooner the motors in it and and you might not even know it because the data center's nearby. 即使它没有破坏它，电网以像59赫兹或其他什么运行，对吧？如果你上下倾斜太多，这些瞬态功率响应，你的冰箱会更快地分解其中的电机，你甚至可能不知道，因为数据中心在附近。 **1:25:02** So like there's like this there's all these things. There's so many like third order effects here with like AI data centers. But like the funnest one is just that like we're building power, right? And it's like whether it's, you know, gas, which is a lot of it, whether it's through efficient dual combine cycle reactors or it's like, you know, random generators that are like not nearly as efficient, single cycle or even worse. Um, diesel generators. 所以就像有这个，有所有这些东西。这里有这么多像AI数据中心的三阶效应。但最有趣的是，我们正在建造电力，对吧？无论是，你知道，天然气，其中很多，无论是通过高效的双联合循环反应堆，还是像，你知道，不那么高效的随机发电机，单循环或更糟。柴油发电机。 **1:25:20** There's there's a company that's putting a bunch of truck engines in parallel like like diesel truck engines because the capacity the industrial capacity for diesel truck engines is huge. Like and no one's tapped it yet. So why don't we just put a ton of them in parallel and create this power generation thing right here, right? 有一家公司正在并联一堆卡车发动机，就像柴油卡车发动机，因为柴油卡车发动机的工业容量是巨大的。没人开发它。所以我们为什么不把一吨它们并联在一起，在这里创建这个发电东西，对吧？ **1:25:31** and and and then you're generating power with a bunch of diesel truck engines in parallel and then you're able to power a data center, right? Like, okay, great. Because I can't get turbines, right? And it's like or like, you know, there's there's all these like crazy things people are doing. Uh Elon buying some power equipment from Poland and shipping it to America because he needed that power equipment, but like whatever, couldn't get it here because the supply chains were weird. I'll just get it over there. 然后你用一堆并联的柴油卡车发动机发电，然后你能够为数据中心供电，对吧？就像，好的，太好了。因为我得不到涡轮机，对吧？就像或者，你知道，有所有这些人们正在做的疯狂的事情。马斯克从波兰购买一些电力设备并将其运送到美国，因为他需要那个电力设备，但无论如何，不能在这里得到它，因为供应链很奇怪。我只是从那里得到它。 **1:25:55** Any lacks capacity in the supply chain is being eaten up immediately and then everyone's like, "Okay, let's invest." So G is like, I'm going to double my turbine production. It's like, holy crap. Okay, that's awesome. And Mitsubishi is doing the same thing. And you know, you move, you go down the list, it's like, my my transformer supply chain is expanding like crazy. 供应链中的任何缺乏容量都被立即吃掉，然后每个人都说，"好的，让我们投资。"所以GE就像，我要加倍我的涡轮机生产。就像，天哪。好的，太棒了。三菱也在做同样的事情。你知道，你移动，你列下去，就像，我的变压器供应链正在疯狂扩张。 **1:26:07** And like, you know, they're fully sold out, so I'm going to go to the Korean guys. And that's fully sold out, so I'm going to figure out how to get the Chinese stuff in, even though it's not exactly like, you know, what people want to do, right? It's like there's all these like weird things like electrician wages have like doubled for mobile electricians that can work on data center stuff or rather contract. 就像，你知道，他们完全售罄，所以我要去找韩国人。那也完全售罄，所以我要弄清楚如何获得中国的东西，即使它不完全像，你知道，人们想做的，对吧？就像有所有这些奇怪的事情，比如电工工资翻倍，对于可以在数据中心工作的移动电工或承包商。 **1:26:25** Like if you're down to move to West Texas, it's like it's like it's like 2015 again and like being a fracking guy, right? You don't need to be super duper skilled. You can go to West Texas and make a shitload load of money off of fracking. But there's not enough of those people. That's why, right? 就像如果你愿意搬到西德克萨斯，就像又回到2015年，就像成为一个水力压裂工人，对吧？你不需要超级熟练。你可以去西德克萨斯，从水力压裂中赚一大笔钱。但没有足够的这些人。这就是为什么，对吧？ **1:26:37** Like if there were enough electricians in West Texas, if there were enough electricians in America, we could build these data centers faster. So there's like all these like little supply chain quirks and weirdy weirdities. Everyone's supply chain is different because the way Google makes their data centers is different from the way Vantage makes their data center which is different from the way that Edge Connects makes their data center which is different from the way QTS makes their data centers which is different from the way Amazon makes their data centers. 就像如果西德克萨斯有足够的电工，如果美国有足够的电工，我们可以更快地建造这些数据中心。所以有所有这些小供应链怪癖和奇怪之处。每个人的供应链都不同，因为谷歌制造数据中心的方式与Vantage制造数据中心的方式不同，与Edge Connex制造数据中心的方式不同，与QTS制造数据中心的方式不同，与亚马逊制造数据中心的方式不同。 **1:27:05** So their supply chains are not exactly the same. Um and so you get all these weirdnesses in all these different supply chains. No one really knows it because everyone who tracked the supply chain or like knew it. Like you go talk to like power people, it's like it's like on one end of the spectrum is like Daario and then you take a few steps and it's like it's like ML researchers, the average ML researcher, then it's like me and then it's like you 所以他们的供应链并不完全相同。所以你在所有这些不同的供应链中得到所有这些奇怪之处。没人真正知道它，因为追踪供应链或知道它的每个人。就像你去和电力人士交谈，就像在光谱的一端就像Dario，然后你走几步，就像ML研究人员，普通ML研究人员，然后就像我，然后就像你 ## Grid Regulations and Backup Power Challenges ## 电网法规和备用电源挑战 **1:27:24** in terms of how bullish we are on AI and the guy at the power utility is like over here, right? Um like only there's like a few more people. There's like the standard New York stock, you know, New York investor, semi-investor, then there's the New York like, you know, um not semiism investor. And then there's like, you know, the Sequoia guy who thinks that A has been a bubble since 2023. 就我们对AI的看好程度而言，电力公司的人就像在这里，对吧？只有像多一些人。有像标准的纽约股票，你知道，纽约投资者，半投资者，然后有纽约的，你知道，不是半导体投资者。然后有，你知道，红杉的人，他认为AI自2023年以来一直是泡沫。 **1:27:42** And then there's this this utility guy, right? Um you know, this utility guy is like, "I'm not building power. Power doesn't go up, you know, whatever." And then you have like the regulations around it. Um you know, it's like, "How can I build a data center in this density?" Because um okay, well then great. Like I'll build the data center in this density. I'll have all this backup generators. 然后有这个公用事业人员，对吧？你知道，这个公用事业人员就像，"我不建造电力。电力不上升，你知道，无论如何。"然后你有围绕它的法规。你知道，就像，"我如何在这个密度下建造数据中心？"因为好的，那么好。就像我会在这个密度下建造数据中心。我会有所有这些备用发电机。 **1:28:00** Great. Now all of a sudden the grid's like um yeah. So what we're going to do is we're going to So this has happened in Texas or it's happening in PJM um which is the main you know the sort of northeast kind of areaish. um these two grids are putting these like rules where hey we're gonna actually say hey big loads we can tell you 24 hours or 72 hours beforehand we're going to cut off half your power so that we can like which is fine right 太好了。现在突然电网就像，是的。所以我们要做的是，这在德克萨斯发生了，或者它正在PJM发生，这是主要的，你知道，有点像东北地区。这两个电网正在制定这些规则，嘿，我们实际上会说，嘿，大负载，我们可以提前24小时或72小时告诉你，我们将切断你一半的电力，以便我们可以，这没问题，对吧 **1:28:23** because like we need to because we need for something else yeah like people people need to have their homes powered we're not [ __ ] like Taiwan where if we're in a drought we limit people's power us water usage and not the fat right which is which is a real story right I think there was there was it was like 2022 2021ish there was like multiple like cities where they were like okay yeah we're going to limit the showers you can take to three a day or three a week, which is fine because they're East Asian and they have they don't have the smelly gene 因为我们需要，因为我们需要其他东西，是的，就像人们需要为他们的家供电，我们不像台湾那样该死，如果我们处于干旱中，我们限制人们的电力，我们的用水量，而不是工厂，对吧，这是一个真实的故事，对吧，我认为有，大约是2022年2021年左右，有多个城市，他们就像好的，是的，我们将限制你可以洗澡的次数为一天三次或一周三次，这没问题，因为他们是东亚人，他们没有臭味基因 **1:28:48** like you know you know like if you did this in in India like it'd be it'd be cooked. I mean it's already cooked but like you know like you know but anyways like you know they'll limit the water to these people before they'll limit the water to TSMC because and it makes sense the economic value of TSMC is way above the economic value of people showering three times a week. 就像你知道，如果你在印度这样做，就会被煮熟。我的意思是它已经被煮熟了，但就像你知道，但无论如何，就像你知道，他们会在限制台积电的水之前限制这些人的水，因为这是有道理的，台积电的经济价值远高于人们每周洗澡三次的经济价值。 **1:29:05** But like the US grid is not going to work that way. We're not that authoritarian or you know people have more say. Okay. So, anyways, like it's like in Texas and in PGM, you can cut half the power if you give them a notice. And if you do that, then you need to turn on the generators that are there on the site. 但美国电网不会那样工作。我们不是那么专制，或者你知道人们有更多发言权。好的。所以，无论如何，就像在德克萨斯和PJM，如果你给他们通知，你可以切断一半的电力。如果你这样做，那么你需要打开现场的发电机。 **1:29:16** You know, it's often diesel generators, maybe it's gas, um maybe it's like hydrogen stuff. There's all sorts of weird stuff people try to do just to ramp up power uh for that period of time. But then all of a sudden, oh crap, the density of my generators means that I fail the air permit if I run the generators for more than eight hours a month. 你知道，通常是柴油发电机，也许是燃气，也许是像氢的东西。有各种奇怪的东西人们试图做，只是为了在那段时间内提升功率。但突然，哦该死，我的发电机密度意味着如果我每月运行发电机超过八小时，我就会不通过空气许可。 **1:29:32** So now what do I do? Right? It's like there's all these like weird regulations. Even if it's like Texas, it's it's really fun that uh we get to watch it. you get to see watch it and then watch the supply chain and try and like you know at least from my perspective provide the data so people can trade on it or provide the data so people can adjust their supply chains 那么现在我该怎么办？对吧？就像有所有这些奇怪的法规。即使是像德克萨斯，这真的很有趣，我们可以观察它。你可以看到它，然后观察供应链，并尝试，你知道，至少从我的角度提供数据，以便人们可以基于它进行交易，或提供数据，以便人们可以调整他们的供应链 **1:29:50** you know industry-wise right people who go to your audience they can trade on it or they can like see and invest and make money and like allocate capital more efficiently right if if I were to line up all the stages of this between the US and China so you know power semis models applications etc where do you think the most interesting differences are 你知道行业方面，去你的受众的人可以基于它进行交易，或者他们可以看到并投资和赚钱，并更有效地分配资本，对吧，如果我要排列美国和中国之间所有这些阶段，所以你知道电力半导体模型应用程序等等，你认为最有趣的差异在哪里 ## US vs China: Who Really Needs AI to Win? ## 美国与中国：谁真正需要AI才能获胜？ **1:30:03** like what are the what are the story lines between us and China at those various layers of like the AI stack that are the most interesting to you? When you look at China, it's like they're a very formidable competitor. Um I think if we didn't have the AI boom, the US probably would be behind China and no longer the world hegeimon by the end of the decade if not sooner. 就像美国和中国之间在AI堆栈的各个层次上最有趣的故事线是什么？当你看中国时，他们是一个非常强大的竞争对手。我认为如果我们没有AI繁荣，美国可能会在十年结束前，甚至更早，落后于中国，不再是世界霸主。 **1:30:30** And a world where the US is not the hegeimon is is a bad one for Americans at least. Um, you know, I I I, you know, I'm sort of like a, you know, sure, [ __ ] bald eagle like carrying like American. I'm like, it's bad for the world. Without AI, like we're definitely just going to lose, right? 一个美国不是霸主的世界，至少对美国人来说是糟糕的。你知道，我我我，你知道，我有点像，你知道，当然，该死的秃鹰，像携带美国人。我就像，这对世界不好。没有AI，我们肯定会失败，对吧？ **1:30:47** Our supply chains are slower. They cost too much. Uh, we're we're sliding. Our debt is like unsustainable. Like, you know, our economy is not growing fast enough to maintain the level of debt. Like our we're over consuming relative to what we produce. um the financialization there's like this all this like darth and like human like in like the US in terms of like social instability 我们的供应链更慢。它们成本太高。我们正在下滑。我们的债务是不可持续的。就像，你知道，我们的经济增长不够快，无法维持债务水平。就像我们相对于我们生产的东西过度消费。金融化有这所有这些像黑暗和像人类在美国的社会不稳定 **1:31:08** partially because of income inequality but also largely because of um the visualness the visual like nature of of income inequality and the tendency of people to flaunt their wealth more because of social media and how that hacks people's brains and then also like because the algorithm serves people different content we're drifting further and further apart in culture 部分是因为收入不平等，但也很大程度上是因为收入不平等的可视化性质，以及人们因社交媒体而更倾向于炫耀财富，以及这如何侵入人们的大脑，然后也因为算法为人们提供不同的内容，我们在文化上越来越疏远 **1:31:30** right monoculture of everyone watching watching the same movies in the ' 50s and 40s and 30s versus like now like you and I are pretty similar and our feeds are completely different. So think about someone who's not in this world like in in our like you know similar worlds like their feed is like insanely different. 对吧，50年代、40年代和30年代每个人都看同样电影的单一文化，与现在相比，就像你和我很相似，我们的信息流完全不同。所以想想不在这个世界的人，在我们类似的世界中，他们的信息流非常不同。 **1:31:44** I think the US would literally fall apart if we don't do something like and and by do something I mean like AI has to dramatically accelerate GDP growth. Once you start talking about dividing the pie you're screwed, right? Um it has to be growing the pie and you know this this whole thing, right? 我认为如果我们不做点什么，美国会真的崩溃，我说做点什么是指AI必须大幅加速GDP增长。一旦你开始谈论分蛋糕，你就完了，对吧？它必须是增长蛋糕，你知道这整个事情，对吧？ **1:31:56** Um, so I like I'm I'm like, you know, the US really really needs AI. China's view is like I think it's like a little bit different, right? They don't necessarily need AI to win. They've always played this long game. They did it with steel. Um, they've done it with like, you know, rare earth minerals. They've done it with solar panels. 所以我就像我就像，你知道，美国真的非常需要AI。中国的观点是，我认为有点不同，对吧？他们不一定需要AI才能获胜。他们一直在玩这个长期游戏。他们用钢铁做到了。他们用稀土矿物做到了。他们用太阳能电池板做到了。 **1:32:14** They've done it for, you know, producing phones. They've done it for PCBs. They've done it for so many freaking industries. Incrementally, they're just going to continue to do that. And then they're going to win because they they work harder and they're on average smarter. 他们为生产手机做到了。他们为PCB做到了。他们为这么多该死的行业做到了。渐进地，他们只会继续这样做。然后他们会赢，因为他们工作更努力，平均更聪明。 **1:32:25** If we don't have super powerful AI systems, we'll run out of like easily accessible like nickel and cobalt and oil and natural gas and we won't be able to make solar panels efficient and fast enough and everything will start to get more expensive and the pies will reduce and we'll also tear each other apart in that way 如果我们没有超强大的AI系统，我们将用完像容易获取的镍和钴、石油和天然气，我们将无法使太阳能电池板足够高效和快速，一切都将开始变得更贵，蛋糕将减少，我们也会以那种方式互相撕裂 **1:32:42** like uh sort of the so I have like a very pessimistic view that if we don't accelerate we die. If that's your world view then like we really need to win AI. Um, and China's worldview is sort of like, you know, they want to be the world hedgeimon. Like, I mean, who doesn't want to be the world hedgeimon? 就像有点所以我有一个非常悲观的观点，如果我们不加速，我们就会死。如果那是你的世界观，那么我们真的需要赢得AI。中国的世界观有点像，你知道，他们想成为世界霸主。我的意思是，谁不想成为世界霸主？ **1:33:01** But there's only two countries in the world that can legitimately do it and legitimately are trying, right? The US and China. The way like the Chinese AI ecosystem thinks about this is, well, we don't necessarily need to have the biggest compute cluster. When OpenAI is trying to make a 2 G data center full of GB200s and GB300s, like all these different chips, and those chips are way faster than the chips that will sell China/ the chips China can make themselves. 但世界上只有两个国家可以合法地做到并合法地尝试，对吧？美国和中国。中国AI生态系统对此的看法是，嗯，我们不一定需要拥有最大的计算集群。当OpenAI试图制造一个充满GB200和GB300的2千兆瓦数据中心时，就像所有这些不同的芯片，这些芯片比将出售给中国/中国可以自己制造的芯片快得多。 **1:33:17** and China is deploying less of them. You know, the girth of compute is huge. You know, we're kind of doing what China's done historically, which is dumping tons of capital into something and the market becomes interesting. And the beneficiary is like, oh, if OpenAI, you know, they have 800 million users today when they have three, four, five billion users across the world, which is possible, right, of of chat, GPT, and whatever applications they come up with, then they're on our system 中国部署得更少。你知道，计算的规模是巨大的。你知道，我们有点在做中国历史上做过的事情，就是向某事物倾注大量资本，市场变得有趣。受益者就像，哦，如果OpenAI，你知道，他们今天有8亿用户，当他们在世界各地拥有三四五十亿用户时，这是可能的，对吧，ChatGPT和他们想出的任何应用程序，那么他们在我们的系统上 **1:33:42** and then then they can start to make money, right? It's sort of like YouTube lost money forever, but now it's the platform for watching videos across the world, right? and Chad GPT will be the same thing. So sort of that like there there's that like aggregation theory. China doesn't necessarily think of it the same way. 然后他们可以开始赚钱，对吧？有点像YouTube永远亏损，但现在它是世界各地观看视频的平台，对吧？ChatGPT也会是一样的。所以有点像那有那个聚合理论。中国不一定以同样的方式思考它。 **1:33:54** Um but they are still incredibly pilledled on like well we want to be able to make everything ourselves, right? So make all of the chips ourselves. We're not necessar we don't actually care that much about making all the chips ourselves. Sure Trump's doing the tariffs, sure we had the chips act, but those were drops in the bucket compared to how much money China's releasing into the semiconductor ecosystem and have has been for the last 10 years. 但他们仍然非常坚定地像嗯我们想能够自己制造一切，对吧？所以自己制造所有芯片。我们不一定，我们实际上不那么关心自己制造所有芯片。当然特朗普在征收关税，当然我们有芯片法案，但与中国在过去10年中向半导体生态系统释放的资金相比，这些只是杯水车薪。 **1:34:18** they've dumped, you know, at least like $4500 billion into this ecosystem through SOE's through um who, you know, through certain tax policies, through certain like land grants, through provincial governments, through uh the big funds, uh which is like government venture funds almost. 他们倾注了，你知道，至少像四五千亿美元到这个生态系统，通过国有企业，通过，你知道，通过某些税收政策，通过某些像土地赠款，通过省政府，通过大基金，这几乎像政府风险基金。 **1:34:29** So, they've dumped so much more capital into semiconductors than we have in an unprofitable way because they want to build that ecosystem. Um and and over time, you know, it's like, well, if you take any country in isolation, China is the one that has everything at the highest level on average, right? 所以，他们以无利可图的方式向半导体投入的资本比我们多得多，因为他们想建立那个生态系统。随着时间的推移，你知道，就像，嗯，如果你孤立地看任何国家，中国是平均拥有最高水平的一切的国家，对吧？ **1:34:43** Sure, they're like 30 years behind on jet engines, but they don't actually need to go for or 20 years or 10 years, whatever it is, but they don't need to go outside of China for any of the materials besides like raw materials. Whereas like the US needs like titanium from here and like, you know, blah blah blah from there, right? 当然，他们在喷气发动机方面落后30年，但他们实际上不需要或20年或10年，无论是什么，但他们不需要离开中国获得任何材料，除了像原材料。而美国需要从这里获得钛，就像，你知道，等等等等从那里，对吧？ **1:35:03** And the same applies to their semiconductor ecosystem, right? Sure, the US and Taiwan and Korea are way ahead, but then they also have the accumulated capital base of all of the existing equipment and all of the existing fabs, but then they they don't have um you know the cap like they they need to import from all these different places because it's a global supply chain. 同样适用于他们的半导体生态系统，对吧？当然，美国、台湾和韩国遥遥领先，但他们也拥有所有现有设备和所有现有工厂的累积资本基础，但他们没有，你知道，资本，他们需要从所有这些不同地方进口，因为这是一个全球供应链。 **1:35:20** And so China is like much more concerned today about being insular than being the best in doing this like sort of aggregation theory. But because they're so talented and they have an insular supply chain, um yes, they purchase some stuff from the foreign world. They rent stuff. They have Bite Dance who's I think the third largest user of GPUs in the world. 所以中国今天更关心封闭而不是在做这个像聚合理论方面做到最好。但因为他们如此有才华，他们有一个封闭的供应链，是的，他们从外国世界购买一些东西。他们租用东西。他们有字节跳动，我认为是世界第三大GPU用户。 **1:35:37** Um after OpenAI and um you know probably Meta, although bite dance may be bigger than Meta, but third largest user of GPUs in the world or second maybe even Bite Dance is um they they have they have all the other major Chinese tech companies. They have all of these amazing graduates. 在OpenAI之后，你知道，可能是Meta，尽管字节跳动可能比Meta大，但是世界第三大或第二大GPU用户，甚至可能字节跳动是，他们有所有其他主要中国科技公司。他们有所有这些令人惊叹的毕业生。 **1:35:49** They don't have a talent pool. companies don't poach from each other, right? Deepseek engineers make a lot more than other engineers, but they're not making $10 million even though they may be worth it, right? Um there's this like real big perception difference. And China could build way faster than us. 他们没有人才库。公司不互相挖角，对吧？Deepseek工程师赚的比其他工程师多得多，但他们没有赚1000万美元，即使他们可能值得，对吧？有这个真正大的感知差异。中国可以比我们建得快得多。 **1:36:08** If they wanted to build, you know, a 2 gawatt or 5 gawatt data center, they could probably smuggle a lot of chips. It's not like a pure derog a derivative of them wanting to smuggle shitloads of chips because hey if they wanted to build a 10 gawatt data center I bet they could build it in like a few years. 如果他们想建造，你知道，2千兆瓦或5千兆瓦数据中心，他们可能可以走私很多芯片。这不像是他们想走私大量芯片的纯粹贬义衍生品，因为嘿，如果他们想建造一个10千兆瓦数据中心，我敢打赌他们可以在几年内建成。 **1:36:25** Um whereas the US is not going to build a single 10 gawatt data center for for a while right like the total capacity of an open AI will be like 10 gawatts in a few years right um optimistically you know they don't have the best chips speed rating up trying to get better and better and faster and faster. 而美国在一段时间内不会建造一个10千兆瓦数据中心，对吧，就像OpenAI的总容量将在几年内达到10千兆瓦，对吧，乐观地说，你知道，他们没有最好的芯片速度评级，试图变得越来越好，越来越快。 **1:36:39** Um they don't have the best memory. Um they're trying to get better and faster there. they do have the most power. They can build stuff way faster, right? We're impressed at how fast Elon does stuff. Elon's slow compared to China. Um, and I think he knows that. 他们没有最好的内存。他们正在努力在那里变得更好更快。他们确实拥有最多的电力。他们可以更快地建造东西，对吧？我们对马斯克做事的速度印象深刻。与中国相比，马斯克很慢。我认为他知道这一点。 **1:36:50** Um, which is why he's maybe like the one who's like actually using the Chinese ecosystem more in terms of like, you know, the the battery facilities making in China and, you know, all these things, right? He's he probably recognizes it too. Um, you so there's there's like these these major differences in like viewpoint and approach 这就是为什么他可能像那个实际上更多地使用中国生态系统的人，就像，你知道，在中国制造的电池设施，你知道，所有这些东西，对吧？他可能也认识到这一点。所以有这些在观点和方法上的主要差异 **1:37:08** because China wants an insular supply chain. They want to have supply chain security. We talk about wanting that, but we don't actually put the money behind it. We're, you know, where's the, you know, where's the the Russian roulette of like where's the American uh or the slot machine of where the American capital is being allocated. Um it's building the biggest data centers. It's training the best models. 因为中国想要一个封闭的供应链。他们想拥有供应链安全。我们谈论想要那个，但我们实际上没有投入资金。我们，你知道，美国资本分配的俄罗斯轮盘赌或老虎机在哪里。它在建造最大的数据中心。它在训练最好的模型。 **1:37:31** Whereas in China, the capital's being allocated into um growing the EV supply chain, growing the semiconductor supply chain, catching up in all these areas. And like the US, sure, we want to catch up, but actually we're just gonna give like terrible. Maybe Jensen was right that like what you want to own is the end customer thing. 而在中国，资本被分配到发展电动汽车供应链、发展半导体供应链、在所有这些领域赶上。就像美国一样，当然，我们想赶上，但实际上我们只会给予像糟糕的。也许Jensen是对的，你想拥有的是最终客户的东西。 **1:37:48** And yeah, export export to production and and import doing the same thing they've done forever, which is like prepare at the base level and be behind at the customer side and the value the value happens close to the customer. But then like you get to the point of like okay well what happens in like three four years even if the US AI is amazing 是的，出口到生产并进口做他们永远做的同样的事情，就像在基础层面准备并在客户侧落后，价值发生在客户附近。但就像你到达这样的点，好的，即使美国AI很惊人，三四年后会发生什么 **1:38:05** we have no you know like the doomsday scenario of like you know China decides to blockade Taiwan or even invade it or create some political instability. People talk about like Cambridge analytics and like Russian trolls whatever like China could do a billion times that into Taiwan especially with AI with how good AI is now 我们没有，你知道，就像世界末日场景，你知道中国决定封锁台湾或甚至入侵它或制造一些政治不稳定。人们谈论像剑桥分析和俄罗斯喷子，无论什么，中国可以对台湾做十亿倍，特别是用AI，AI现在有多好 **1:38:22** and and somehow subvert it or coup or blockade or whatever and we no longer have Taiwan US economy kind of free falls, right? Because we can't make refrigerators without Taiwanese chips. We can't make, you know, cars. We can't make AI data centers. We can't grow any of the cloud. We can't That means we can't deploy any more SAS applications. Like, what the hell can we do? 以某种方式颠覆它或政变或封锁或其他什么，我们不再拥有台湾，美国经济有点自由落体，对吧？因为没有台湾芯片，我们做不了冰箱。我们做不了，你知道，汽车。我们做不了AI数据中心。我们不能发展任何云。我们不能，这意味着我们不能再部署任何SaaS应用程序。我们到底能做什么？ **1:38:40** Back to going to acquire all the talent, get them over here, do that. Right. I I think like that's sort of like the the like catch 22 of this all is like if you push China too hard, they totally will. Like they're going to start swinging. They have the talent, they could go crazy, they could, you know, if if we no longer have Taiwan, actually, China could build a way bigger cluster than us. 回到获取所有人才，把他们带过来，做那个。对。我认为这有点像这一切的两难境地，如果你太用力推中国，他们完全会。他们会开始挥舞。他们有人才，他们可以疯狂，他们可以，你知道，如果我们不再拥有台湾，实际上，中国可以建立一个比我们大得多的集群。 **1:39:05** And if comput is all that matters, right? Like they could do all of these things and they own the means of production for everything, right? So sort of like it's like, you know, there's this like challenging aspect of like geopolitical risk is like, you know, that's why people don't want to invest in TSMC, but it's like almost like you can't invest into Amazon or Apple or Google or like Microsoft if you have geopolitical risk. 如果计算是唯一重要的，对吧？他们可以做所有这些事情，他们拥有一切的生产资料，对吧？所以有点像，你知道，有这个像地缘政治风险的挑战方面是，你知道，这就是为什么人们不想投资台积电，但几乎就像如果你有地缘政治风险，你就不能投资亚马逊或苹果或谷歌或微软。 **1:39:24** if you believe Taiwan has risk and so it's like yolo invest in TSMC. I know a lot of people's PMs are like oh you can't invest in TSMC because geopolitical risk and it's like no dude you can't invest in [ __ ] Apple. 如果你相信台湾有风险，所以就像冒险投资台积电。我知道很多人的投资组合经理就像哦你不能投资台积电因为地缘政治风险，就像不，伙计，你不能投资该死的苹果。 ## Favorite AI Bears ## 最喜欢的AI看空者 **1:39:31** Who is your favorite AI bear? Like someone that is is far distant from you on just their perspective on the direction of this whole thing that you nonetheless like and respect. 谁是你最喜欢的AI看空者？就像某人在他们对这整个事情方向的看法上与你相距甚远，但你仍然喜欢和尊重他们。 **1:39:43** There's some of the like AI researcher like gods like Yan Lun and like these kind of people who are AI bears. I I respect them. I like what they like their ideas. I think they're completely wrong, but you and their argument is what? Like if you had to sum it, you know, the ways we're doing this won't work, right? 有一些像AI研究人员之类的神，比如Yann LeCun和这些看空AI的人。我尊重他们。我喜欢他们的想法。我认为他们完全错了，但他们的论点是什么？就像如果你必须总结，你知道，我们做这个的方式不会起作用，对吧？ **1:40:00** LLM's on scale or right, but and it's like, okay, yeah, yeah, auto reggress like it's like, okay, auto reggressive pre-training on the internet doesn't work to get you to AGI. So, he's completely right on that, but then like he'll turn around and be like, well, no, no, no, but like RL systems and all these things are not the right way either, right? LLM在规模上或对的，但就像，好的，是的，是的，自回归就像，好的，互联网上的自回归预训练不能让你达到AGI。所以，他在那方面完全正确，但就像他会转身说，嗯，不，不，不，但像RL系统和所有这些东西也不是正确的方式，对吧？ **1:40:16** Like, you know, it's sort of like, you know, it's like the no butts. Um, I think there's also like there's some investors that I know who like think this is [ __ ] but they're just making tons of money on it anyways. I think it's [ __ ] in what sense? Like it's an overspend and like, you know, this is a dumb way to do it. 就像，你知道，有点像，你知道，就像没有但是。我认为还有一些我认识的投资者，他们认为这是废话，但无论如何他们只是从中赚很多钱。我认为在什么意义上是废话？就像这是过度支出，就像，你知道，这是一个愚蠢的方式。 **1:40:30** And like, yes, I bought Oracle before earnings because, you know, you know, we see we see these deals happening, whether it's through our data or some other means. They saw these deals happening um with OpenAI, but we don't think OpenAI can pay for it, but we know the market's perception will be this and therefore the stock will go up and so we'll own it, right? 就像，是的，我在盈利前买了Oracle，因为，你知道，我们看到这些交易发生，无论是通过我们的数据还是其他方式。他们看到这些交易与OpenAI发生，但我们不认为OpenAI能支付它，但我们知道市场的看法会是这样，因此股票会上涨，所以我们会拥有它，对吧？ **1:40:55** Um, and so there's like I guess I would respect them to some sense, but it's it's it's it's um I think more and more the level of evidence that's there that this stuff is going to get super powerful, it's hard to not, right? Again, like this AI bubble is going to pop because this podcast, man, I assure you it's just a market return. It's a coin toss. 所以有点像我想我会在某种意义上尊重他们，但它是，我认为越来越多的证据表明这些东西将变得超级强大，很难不这样想，对吧？再次，像这个AI泡沫将破裂，因为这个播客，伙计，我向你保证这只是市场回报。这是抛硬币。 **1:41:07** What startups interest you the most? So, one of the startups is um that I've I've you know, it's the most recent investment I've made. It's called periodic labs. Um it's mostly open AI people. It's a Google guy and you know the couple material scientists. 什么初创公司最让你感兴趣？所以，其中一个初创公司是，你知道，这是我最近的投资。它叫Periodic Labs。主要是OpenAI的人。有一个谷歌的人，还有几个材料科学家。 **1:41:20** Um the area of AI that we've all been talking about is like large scale web training RL all text all digital god right. You know I you know we want to make digital god. Yeah. But you know what would drive a shitload of value for the economy besides you know automating programming of everything 我们一直在谈论的AI领域就像大规模网络训练RL所有文本所有数字上帝，对吧。你知道我们想制造数字上帝。是的。但你知道除了自动化编程一切之外，什么会为经济带来大量价值 **1:41:37** is like if we just like came up with like a battery chemistry that was like 25% more efficient. Mhm. Like holy [ __ ] You know, like the main main cap against like us all having like face glasses, you know, things like that is like batteries are not good enough, right? And the power dissipation, but the battery is like terrible. 就像如果我们想出一种电池化学，效率提高25%。嗯。就像天哪。你知道，像我们所有人都拥有像脸部眼镜的主要限制，你知道，像那样的东西是电池不够好，对吧？还有功耗，但电池就像很糟糕。 **1:41:54** So, you have to do make all these compromises. But if I could have the processing power of like, you know, a laptop on my face, we'd be way further ahead. And like and then and then if we all had like these like super powerful machines attached to our face, we could do inference on things and recognize and interact with the AI at much higher speed and velocity 所以，你必须做出所有这些妥协。但如果我能在脸上拥有像笔记本电脑的处理能力，我们会遥遥领先。就像然后如果我们都有这些像超级强大的机器连接到我们的脸上，我们可以对事物进行推理并以更高的速度和速度识别和与AI交互 **1:42:14** and and that would like dramatically improve our productivity, right? Like things like this are like so like gated by hard tech moving faster. And so what Periodic is trying to do is they're taking this RL paradigm, but they're trying to do it with like real world, right? test chemistry for something, right? A here's a here's a chemistry um here's an optimization, here's something that the model spit out, but then you also want to test it in the real world and then feed that feedback back into the model. 这会大大提高我们的生产力，对吧？像这样的东西就像被硬科技更快发展所限制。所以Periodic试图做的是他们采用这种RL范式，但他们试图用现实世界来做，对吧？测试某物的化学，对吧？这是一种化学，这是一个优化，这是模型吐出的东西，但你也想在现实世界中测试它，然后将反馈反馈回模型。 **1:42:35** And so you do this like chain of of circles, right? But instead of purely being, you know, digital, right? Which is which is why like RL is like really hard because you need to generate a bunch of responses, test, and then train the model. So the flywheel is so freaking fast. Yeah. Right. the flywheel in the physical world is so slow, right? 所以你做这个像圆圈链，对吧？但不是纯粹是，你知道，数字的，对吧？这就是为什么RL真的很难，因为你需要生成一堆响应，测试，然后训练模型。所以飞轮如此快速。是的。对。物理世界中的飞轮如此缓慢，对吧？ **1:42:52** Oh, if I you mean I need to make a chemistry, I need to try this. I need to test the the thing. Um I need to input it back in and and you know, I need to keep calibrating and keep doing this. It's so much more expensive. It's so much harder to do. But actually, there's a ton of low hanging fruit there, I bet. 哦，如果我你的意思是我需要制作一种化学，我需要尝试这个。我需要测试这个东西。我需要把它输入回去，你知道，我需要不断校准并继续做这个。这要贵得多。做起来要难得多。但实际上，我敢打赌那里有很多低垂的果实。 ## Hardware Innovation Beyond Accelerators ## 加速器之外的硬件创新 **1:43:10** What about in the hardware world? Like just in the pure hardware space attacking some other interesting bottleneck when we talk about like where we are in tech, right? Like it's like um semiconductor manufacturing is super space age, right? It's like the most complicated tools we make in the world. 硬件世界呢？就像在纯硬件领域攻击一些其他有趣的瓶颈，当我们谈论我们在科技中的位置时，对吧？就像半导体制造是超级太空时代，对吧？这是我们在世界上制造的最复杂的工具。 **1:43:27** Um, including like tools that cost like half a billion dollars, right? Like, and and they're super super like amazing feats of engineering. Then the software behind them all is like really [ __ ] right? So, it's like, you know, you could you can accelerate all that. 包括像花费半十亿美元的工具，对吧？就像，它们是超级超级惊人的工程壮举。然后它们背后的软件就像真的很糟糕，对吧？所以，就像，你知道，你可以加速所有这些。 **1:43:43** Um, but really it's like um in the hardware world, the biggest challenge is that like I'm not really a big bull on the accelerator companies. I've never been. Companies make competing with Nvidia. Yeah, I got it. um competing with Nvidia, with TPUs, with Tranium, you know, with with AMD. Not a big bull on those kinds of companies because it's too hard. 但实际上在硬件世界中，最大的挑战是我对加速器公司不是很看好。我从来没有。与Nvidia竞争的公司。是的，我明白。与Nvidia、TPU、Trainium、AMD竞争。对这类公司不是很看好，因为太难了。 **1:44:01** It's just too many things to do. It's too capo intensive. It's too there's not enough of a revolutionary leap. There's too many predicated things. Um you know, I'm I I wish it could happen, right? It'd be fun. Um maybe it does happen, but it would take hell of a badass thing. But I think there's a lot of individual parts of the supply chain which are not spaceaged, right? 要做的事情太多了。资本密集度太高。没有足够的革命性飞跃。有太多前提条件。你知道，我希望它能发生，对吧？那会很有趣。也许它会发生，但那需要非常厉害的东西。但我认为供应链中有很多单独的部分不是太空时代的，对吧？ **1:44:24** Um, Nvidia space age, yes, it's the biggest value value owner today, but their supply chain has so much old [ __ ] right? And whether it's their supply chain or the hyperscale supply chain, transformers have not changed in like 50, 100 years, right? Like there's a guy building a company in that space. Solid state transformers, right? Like things like this. Nvidia是太空时代，是的，它是今天最大的价值所有者，但他们的供应链有这么多旧东西，对吧？无论是他们的供应链还是超大规模供应链，变压器在50、100年内没有改变，对吧？就像有一个人在那个领域建立一家公司。固态变压器，对吧？像这样的东西。 **1:44:29** Yeah. So, there's all sorts of interesting things there. There's so many interesting companies in that space because there's so much innovation to be done and there wasn't that much of a need to do innovation before. Another area is like networking between chips because as we extend context length the memory requirements become bigger and bigger and yes new memory technologies would be awesome 是的。所以那里有各种有趣的东西。那个领域有这么多有趣的公司，因为有这么多创新要做，以前没有那么多创新的需求。另一个领域是芯片之间的网络，因为当我们扩展上下文长度时，内存需求变得越来越大，是的，新的内存技术会很棒 **1:44:50** but DM is an industry has so much invested capital goods so much so existing factories it's really hard to attack. Um but you know networking is less so and there's more breakthroughs that can be done in networking that okay maybe you don't have better memory technologies but you've tied the chips closer together so you can use each other's memory on the problem. 但DRAM作为一个行业有这么多投资的资本货物，这么多现有工厂，真的很难攻击。但你知道网络不那么多，在网络中可以做更多突破，好的，也许你没有更好的内存技术，但你把芯片绑得更紧密，所以你可以在问题上使用彼此的内存。 **1:45:07** There's so much more that you can do in terms of the optics space bridging the gap between electrical connectivity and optical connectivity because look you know Nvidia created Blackwell they had a ton of manufacturing problems and challenges with it um for their supply chain um balance sheets went up for you know various companies in the supply chain were building servers and stuff 在光学领域，在电连接和光连接之间架起桥梁，你可以做更多的事情，因为你知道Nvidia创建了Blackwell，他们在供应链方面遇到了大量制造问题和挑战，资产负债表上升了，你知道，供应链中的各种公司正在建造服务器和东西 **1:45:24** because they're trying to figure it out AI server AI data center deployments were slowed because of these challenges there's reliability challenges because these things are connecting to each other at you know at absurd bandwidths right every chip in the rack and connect to every other chip in the rack at 1.8 terabytes a second, right? 因为他们试图弄清楚AI服务器AI数据中心部署因这些挑战而放缓，有可靠性挑战，因为这些东西以荒谬的带宽相互连接，对吧，机架中的每个芯片以每秒1.8太字节连接到机架中的每个其他芯片，对吧？ **1:45:40** That's that's if you think about how much data that is, right? Like like the amount of bandwidth is so high for connecting these chips together. Like I like you can't fathom what a terabyte a second is. You can't fathom what a gigabyte a second is. You know, it's like okay, a gigabyte a second is like a video, right? Like or like less than a video, right? Or like a megabyte a second 那就是如果你想想那有多少数据，对吧？就像连接这些芯片的带宽如此之高。就像我喜欢你无法理解每秒一太字节是什么。你无法理解每秒一千兆字节是什么。你知道，就像好的，每秒一千兆字节就像一个视频，对吧？就像或少于一个视频，对吧？或像每秒一兆字节 **1:46:00** but actually that's a million bits of information. What's what's a kilobyte a sec? a bite a sec. Okay, you can understand what a bite a second is because that's eight bits. Okay, I'm transmitting eight bits to you back and forth every second. That's pretty fast. That's what it used to exist. And it's like where we are now, there's still tons of innovation left to be done there. 但实际上那是一百万比特的信息。每秒一千字节是什么？每秒一字节。好的，你可以理解每秒一字节是什么，因为那是八比特。好的，我每秒向你来回传输八比特。那相当快。那就是它过去存在的样子。就像我们现在在哪里，那里仍然有大量创新要做。 **1:46:17** Like I think part of the reason Intel is behind is also that uh data sharing internally was terrible and just within the fab the lithography team doesn't want to share their data with um the etch team and they can't use and and that data can't leave the fab and go to an AWS data center to run you know uh you know correlations and all these other things. 就像我认为英特尔落后的部分原因也是内部数据共享很糟糕，仅在工厂内，光刻团队不想与蚀刻团队共享他们的数据，他们不能使用，那些数据不能离开工厂去AWS数据中心运行你知道相关性和所有这些其他东西。 **1:46:37** So you don't learn from the experiments you do fast enough. Right now TSMC is not perfect here either. They won't send their data to a cloud either. But like you know this experimentation experiment analyze the data figure out the new experiments cycle is slow and how you break that is actually like you know partially it's it's changing these companies culture which I think lian is trying to do. 所以你不能从你做的实验中学得足够快。现在台积电在这方面也不完美。他们也不会将数据发送到云。但你知道这个实验实验分析数据弄清楚新实验周期很慢，你如何打破它实际上就像你知道部分是改变这些公司的文化，我认为C.C. Wei正在尝试这样做。 **1:46:58** Um but also it's a lot of it is like building better simulators simulating the world more accurately. So world models generally are like hey I'm going to simulate the world. I'm going to walk around in, you know, the common one I think is G3 that Google made, right? Where you can walk around and at at Cape State and you walk around the world and you can like see cars driving and like 但也有很多是像建造更好的模拟器，更准确地模拟世界。所以世界模型通常就像嘿我要模拟世界。我要在，你知道，我认为常见的是谷歌制作的Genie，对吧？你可以在开普敦走来走去，你在世界各地走来走去，你可以看到汽车行驶 **1:47:18** they're talking about like but it's like actually what a world model could also be is just like simulating molecules or simulating um but not through classical methods, right? It's not computational fluid dynamics. It's it's the model experiencing this enough and then running and training a model on physics and then feeding that back through and doing it through a AI method instead of 他们在谈论，但实际上世界模型也可以只是像模拟分子或模拟，但不是通过经典方法，对吧？它不是计算流体动力学。它是模型足够经历这个，然后运行并在物理学上训练模型，然后将其反馈并通过AI方法而不是 **1:47:35** and so world models can be doing any sorts of things, right? You can put make a world model to train robots how to pick up cups, right? Or you can make a world model that is simulating some chemistry in a chemical reaction, right? Or um a fire, right? Like you could do all sorts of different things. 所以世界模型可以做各种事情，对吧？你可以制作一个世界模型来训练机器人如何拿起杯子，对吧？或者你可以制作一个模拟化学反应中某些化学的世界模型，对吧？或者火，对吧？你可以做各种不同的事情。 **1:47:48** So there's a lot of world model companies out there. Some of them are really interesting, especially when they're targeting the physics and reality of the world. Most of the cool innovation is just happening at big companies or already existing companies, right? That's just the nature of it all, right? 所以外面有很多世界模型公司。其中一些真的很有趣，特别是当他们针对世界的物理和现实时。大多数酷的创新只是发生在大公司或已经存在的公司，对吧？这就是一切的本质，对吧？ **1:48:05** Like actually TSMC is doing the most cool innovation and Nvidia is doing the most cool innovation and like you know Amphanol is doing cool innovation. It's like all these companies are doing cool innovation. C could we do like a uh a quick speed round where like I say some company and you just give me like you know a sentence or two on like your impression of them just like just like how how you feel about them in this moment. 实际上台积电正在做最酷的创新，Nvidia正在做最酷的创新，就像你知道Amphenol正在做酷的创新。就像所有这些公司都在做酷的创新。我们能做一个快速回合吗，就像我说一些公司，你只给我一两句话，就像你对他们的印象，就像你现在对他们的感觉。 ## Speed Round: Company Impressions ## 快速回合：公司印象 **1:48:23** Yeah, start with Open AI. Oh, yeah. Super awesome. That's it. I mean, we've talked about them all day. Anthropic. I'm actually more optimistic on anthropic than I'm Open AI. Why? their revenue is accelerating way faster because what they're focused on is more relevant to that two trillion dollar software market 是的，从OpenAI开始。哦，是的。超级棒。就这样。我的意思是，我们整天都在谈论他们。Anthropic。我实际上对Anthropic比对OpenAI更乐观。为什么？他们的收入加速得更快，因为他们关注的东西与那个2万亿美元软件市场更相关 **1:48:38** versus OpenAI is split between yeah they're going to do that but they're also going to do these other things but they're also going to do like target AI for you know science and they're going to also target AI for um you know the consumer app and doing the like take rate thing which all of these businesses could be amazing and open maybe executes on all of them but Anthropic is definitely executing on the software side better 而OpenAI在是的他们要做那个之间分裂，但他们也要做这些其他事情，但他们也要做像针对科学的AI，他们也要针对消费应用程序并做像抽成的事情，所有这些业务都可能很棒，OpenAI可能在所有这些上执行，但Anthropic肯定在软件方面执行得更好 **1:48:55** yeah AMD I love them but they're pretty mid why do you love them if when you grow up like like building computers and like liking computers and like AMD's innovating and they're always like fostered this underdog mentality against Intel and against Nvidia, evil Intel and evil Nvidia, you know, and like AMD is like, you know, the nice company that's like the underdog 是的，AMD我爱他们，但他们相当中等，如果你长大时喜欢建造计算机和喜欢计算机，AMD正在创新，他们总是培养这种对抗英特尔和Nvidia的弱者心态，邪恶的英特尔和邪恶的Nvidia，你知道，就像AMD是，你知道，那个弱者的好公司，你为什么爱他们 **1:49:17** and like they're always they've always got the oh, they're going to take share from them thesis. It's like it's like it's hard not to love them, you know, like and I know so many people there and I like the I like all these major hardware companies, right? There's not one that I don't like as in terms of the people, but like AMD's got a soft spot because like I think it was my first multibagger as well. 就像他们总是有哦，他们要从他们那里抢占份额的论点。就像很难不爱他们，你知道，我认识那里这么多人，我喜欢所有这些主要硬件公司，对吧？就人而言，没有一个我不喜欢的，但AMD有一个软肋，因为我认为它也是我的第一个多倍投资。 **1:49:36** Like my first multibagger. I can't own stocks anymore because compliance. Sorry for the rant, but I [ __ ] love AMD, you know? I also love Nvidia, but mid but mid XAI. They're in a real danger of not being able to raise capital. U Elon's the best C. 就像我的第一个多倍投资。我不能再拥有股票了，因为合规。抱歉发牢骚，但我该死地爱AMD，你知道吗？我也爱Nvidia，但中等，但中等。XAI。他们处于真正无法筹集资金的危险中。马斯克是最好的CEO。 **1:49:47** Of course, everyone's going to give Elon capital, but like the scale of capital required uh for him to keep up. He can get the next bet. He can get to Colossus 2, right? uh this mega data center that he's building, largest data center in the world when he builds it, uh 300,000, you know, black wells, 500,000 black wells, right? 当然，每个人都会给马斯克资本，但就像他需要跟上的资本规模。他可以得到下一个赌注。他可以到达Colossus 2，对吧？他正在建造的这个超级数据中心，当他建造它时是世界上最大的数据中心，30万，你知道，Blackwell，50万Blackwell，对吧？ **1:50:04** Like it's going to be really great, but if he doesn't figure out a business model besides like Pornbot, um which is what Annie is, which also I think he's monetizing the wrong way. Like I think he could monetize it so much better. how you've captured the zeitgeist with a cute anime girl that talks to you in a cute voice 就像它会非常棒，但如果他不弄清楚除了像色情机器人之外的商业模式，这就是Grok是什么，我也认为他在以错误的方式赚钱。我认为他可以更好地赚钱。你如何用一个用可爱的声音和你说话的可爱动漫女孩捕捉时代精神 **1:50:22** and like will rz you up and and you've got like these users who actually fall for it and it's like not realistic enough yet, but it will slowly get more realistic. And you're not like you're selling like outfits for the same price. You should make it a random like, "Hey, you have a chance to buy the outfit that is actually her being nude." 就像会让你兴奋，你有这些用户实际上会爱上它，就像还不够现实，但它会慢慢变得更现实。你不像你在以相同价格出售服装。你应该让它随机，"嘿，你有机会购买她实际上裸体的服装。" **1:50:41** Or like, "Hey, you have the chance to buy the outfit of her like looking like this one anime girl from this one anime." Or like, "Hey, you have the chance to buy this outfit that's her in a nun suit." I mean, obviously like, you know, people at XAI hate this and like a lot of them and and many of them have some of them have left. 或者，"嘿，你有机会购买她看起来像这个动漫中这个动漫女孩的服装。"或者，"嘿，你有机会购买这件她穿修女服的服装。"我的意思是，显然，你知道，XAI的人讨厌这个，其中很多人，很多人已经离开了。 **1:50:53** But I think like he has to figure out like some business model beyond just this, although I think this could be a big business, right? like he he should partner with Only Fans and make like make manifestations of the Only Fans creator with that are Annie and then he subsumes the Only Fans platform into X 但我认为他必须弄清楚除此之外的一些商业模式，尽管我认为这可能是一个大生意，对吧？就像他应该与OnlyFans合作，制作OnlyFans创作者的Grok化身，然后他将OnlyFans平台纳入X **1:51:11** the everything app and be like XXX like you know it's like you could you could just like Trojan horse like Only Fans away cuz like the discovery mechanism for only fans is like Instagram and Twitter as far as I understand and like you own one of them and you could partner with the biggest Only Fans creators to like you know get them over right like you know 万能应用程序，就像XXX，你知道，就像你可以特洛伊木马一样，就像OnlyFans，因为据我所知OnlyFans的发现机制是Instagram和Twitter，你拥有其中之一，你可以与最大的OnlyFans创作者合作，你知道，让他们过来，对吧，你知道 **1:51:29** and then they they don't have to respond to all the losers like they can also just like train a model that like acts and looks like them and talks to them. Anyways, like there's all these different monetization methods and I don't think that's what he should only focus on, right? To be clear, XAI can get to the next stage of compute. They won't have more compute than opening. 然后他们不必回应所有失败者，他们也可以训练一个像他们一样行动和看起来像他们并与他们交谈的模型。无论如何，就像有所有这些不同的赚钱方法，我不认为这是他应该唯一关注的，对吧？明确地说，XAI可以到达下一阶段的计算。他们不会拥有比OpenAI更多的计算。 **1:51:48** I won't have more compute than any individual company at Google, Meta, etc., But they will have the biggest individual data center and what he does with that and he and they'll be they'll have a very focused team and what they do with that they have to do something like really big otherwise they will fall behind in the race 我不会拥有比谷歌、Meta等任何单独公司更多的计算，但他们将拥有最大的单个数据中心，他用它做什么，他和他们将有一个非常专注的团队，他们用它做什么，他们必须做一些真正大的事情，否则他们会在竞赛中落后 **1:52:05** and and Elon will not let that happen like he doesn't want that happen but he can't he can he can subsidize and fund this round but like he can't go to a 3 gawatt data center unless he gets capital which he can't do unless he gets revenue and fundraising. 马斯克不会让那发生，他不想让那发生，但他不能，他可以补贴和资助这一轮，但他不能去3千兆瓦数据中心，除非他获得资本，除非他获得收入和筹款，他不能这样做。 **1:52:17** Oracle Oracle is going to make so much [ __ ] money if you believe if you believe OpenAI is successful. But if you think Open AI is going to be successful enough to pay $300 billion dollars to them, how many users do they have and what's that IP worth? Like maybe and also like you know there's reasons you shouldn't own OpenAI, like the Microsoft Oracle，如果你相信OpenAI成功，Oracle将赚这么多该死的钱。但如果你认为OpenAI将成功到足以向他们支付3000亿美元，他们有多少用户，那个IP值多少？就像也许，还有你知道你不应该拥有OpenAI的原因，就像微软 **1:52:40** stuff and like the risks around Enthropic and all these things, but like you know in most worlds where open where Oracle gets paid $300 billion by OpenAI, OpenAI is like a$10 trillion or $5 trillion company or something crazy. We'll end with the the OGs, the the old last generation best two business models. First being Meta. 东西和围绕Anthropic的风险以及所有这些事情，但就像你知道在大多数世界中，Oracle从OpenAI获得3000亿美元，OpenAI就像一个10万亿美元或5万亿美元的公司或一些疯狂的东西。我们将以OG结束，旧的上一代最好的两个商业模式。首先是Meta。 **1:52:56** I think Meta's got the cards to potentially like own it all. I don't know if you've seen these new glasses that they came out with the screen. Yeah. As we go through the history of computing, you have, you know, initially it was like ter it was like punch cards programming. Then it was like DOSS terminals, right? 我认为Meta有可能拥有所有牌。我不知道你是否看过他们推出的这些带屏幕的新眼镜。是的。当我们回顾计算历史时，你有，你知道，最初就像打孔卡编程。然后就像DOS终端，对吧？ **1:53:08** And then it was like then it was like, oh, you have gooies and mouses and keyboards. Then you had touch. And the next paradigm, a human computer interface is we don't actually have to touch it at all. We tell the AI what we want and the AI will translate that into reality, right? 然后就像，哦，你有图形用户界面、鼠标和键盘。然后你有触摸。下一个范式，人机界面是我们实际上根本不必触摸它。我们告诉AI我们想要什么，AI会将其转化为现实，对吧？ **1:53:21** Whether it's, hey, send an email to this person, send a text to this person. That's basic stuff that you can already do that with Siri or whatever, right? But like, oh, go buy this. We're so close to all of these things. The input method into a computer changing entirely. And the only company in the world who has the full stack from good hardware that is a you know what what 无论是，嘿，给这个人发邮件，给这个人发短信。这是你已经可以用Siri或其他什么做的基本东西，对吧？但就像，哦，去买这个。我们离所有这些东西如此接近。计算机的输入方法完全改变。世界上唯一拥有从好硬件到完整堆栈的公司 **1:53:40** Meta just showed with their glasses with the screen plus the good models um plus the capacity to serve them plus the uh knowledge and knowhow around recommendation systems to know what content to put in front of the user. Um because it is ch it's not just generating the content. It's not just interpreting the user's word and taking actions. Meta刚刚用他们带屏幕的眼镜展示了加上好模型，加上服务它们的容量，加上围绕推荐系统的知识和专业知识，知道在用户面前放什么内容。因为它不仅仅是生成内容。它不仅仅是解释用户的话并采取行动。 **1:53:58** It's also putting the right content in front of the user. It's all four of these that you need to put in front of the user plus the capital. Plus the capital. Um and I think I think Meta is so close to being the only company that can do that. U there's a lot of risks there too, right? 它还在用户面前放置正确的内容。你需要在用户面前放置所有这四个加上资本。加上资本。我认为Meta如此接近成为唯一能做到的公司。那里也有很多风险，对吧？ **1:54:09** So I like Meta a lot. Google to finish it off. I it was pretty bearish Google like two years ago, but I'm I'm like super bullish Google. Why would change? They're waking up on every front front. You know, they're taking the TPUs, they're selling them externally. They're taking the um their models and they're actually like competitive on them and they're training much better and better and better. 所以我非常喜欢Meta。谷歌来结束它。我两年前对谷歌相当看空，但我现在对谷歌超级看好。为什么会改变？他们在每个前线都在觉醒。你知道，他们正在采用TPU，他们正在外部销售它们。他们正在采用他们的模型，他们实际上在它们上具有竞争力，他们的训练越来越好。 **1:54:27** Um they're being aggressive on infrastructure investments. Um there's still a lot of dysfunction throughout the company, you know, but you know, they do have the hardware business that they can pivot into this. They won't be as head as Meta is. Um they won't be as good as Apple is, but like they they they do have Android. 他们在基础设施投资方面很激进。公司内部仍然有很多功能障碍，你知道，但你知道，他们确实有硬件业务，他们可以转向这个。他们不会像Meta那样领先。他们不会像苹果那样好，但他们确实有Android。 **1:54:39** Um they do have YouTube. They do have like all these IPs. They have search that can come together when we turn to that next interface of consumer. But also they can also dominate the professional sense too potentially. Whereas Meta I don't think can dominate that professional sense um only the consumer sense and I think Google's well positioned to go capture both markets or a meaningful share of both. 他们有YouTube。他们有所有这些IP。当我们转向下一个消费界面时，他们有搜索可以结合在一起。但他们也可能在专业意义上占主导地位。而Meta我认为不能在专业意义上占主导地位，只能在消费意义上，我认为谷歌处于有利位置，可以占领两个市场或两者的有意义份额。 ## The Death of Traditional SaaS Business Models ## 传统SaaS商业模式的消亡 **1:55:03** I feel like we've covered like an incredible amount of ground. Is there anything that we haven't talked about that you feel is like really critical to what happens in the future that we didn't cover? I think the question, you know, sort of of everyone that I constantly get asked is like, okay, Dylan, you know, you're lucky your 我觉得我们已经涵盖了难以置信的大量内容。有什么我们没有谈到的你觉得对未来发生的事情真的很关键的吗？我认为问题，你知道，我经常被问到的每个人的问题是，好的，Dylan，你知道，你很幸运，你的 **1:55:22** obsession is that you loved hardware and you like followed it and you followed the supply chain and and you built this business on it, but like you really like you don't follow the software side nearly as much and all the value is going to get created there, right? When is that going to when is that flip coin going to flip over? 痴迷是你喜欢硬件，你喜欢跟随它，你跟随供应链，你在它上面建立了这个业务，但就像你真的喜欢，你不那么关注软件方面，所有价值将在那里被创造，对吧？那什么时候会翻转？ **1:55:39** Um, but I think the thing that most people don't realize, software is not the same as it was 5 10 years ago. you've had dramatic changes in software and the business model is going to change as well. Right? If we go back like 5 years, three years, whatever when SAS was the darling November 21, I remember SAS started tanking and um at the time it was like it was like mostly like they were over earning and all these other things doesn't matter. 但我认为大多数人没有意识到的是，软件不像5-10年前那样。软件发生了巨大变化，商业模式也将改变。对吧？如果我们回到5年前、3年前，无论什么时候SaaS是宠儿，2021年11月，我记得SaaS开始暴跌，当时就像主要是他们过度盈利和所有这些其他事情不重要。 **1:56:04** The interesting thing about the business model is that it was it is such a good business model when your R&D is sort of this it stays flat, right? And you grow a little bit but really R&D doesn't flex that much. Your cogs are super low. Um you're you know the flip side is in a SAS business your customer acquisition cost is quite high. Yeah. 商业模式有趣的是，当你的研发保持平稳时，它是如此好的商业模式，对吧？你增长一点，但研发实际上不那么灵活。你的销售成本超级低。你知道，另一方面，在SaaS业务中，你的客户获取成本相当高。是的。 **1:56:20** And so when you look at what like what like certain companies have done when they've acquired a business is they've just crushed the customer cost acquisition cost or crust sa they made the business amazing whether it's like Broadcom with VMware and stuff. It's not really customer acquisition. They just had a bunch of wasted SGA 所以当你看某些公司在收购业务时所做的，他们只是压低了客户获取成本或外壳SGA，他们使业务惊人，无论是像Broadcom与VMware的事情。这不是真正的客户获取。他们只是有一堆浪费的SGA **1:56:37** but like this SGA this customer acquisition that was most of your cost. R&D was small but not like crazy. And then once you hit critical mass, you just you just cash cash money money. But software changes a lot when the cost to build that software that you have tanks like crazy. 但就像这个SGA，这个客户获取是你成本的大部分。研发很小，但不像疯狂。然后一旦你达到临界质量，你只是现金现金钱钱。但当建造你拥有的软件的成本疯狂下降时，软件会发生很大变化。 **1:56:53** You look at non US markets and the prevalence of SAS, it's very different. I will bring up China as an example and a counterpoint. China doesn't have that much of a SAS business. Actually, their cloud business is pretty small too, right? relatively to the US not despite them like importing tons of CPUs and storage historically right there most people just did stuff on prem and design their own software 你看非美国市场和SaaS的普及，它非常不同。我将以中国为例和反例。中国没有那么多SaaS业务。实际上，他们的云业务也相当小，对吧？相对于美国，不是尽管他们历史上进口了大量CPU和存储，大多数人只是在本地做事情并设计自己的软件 **1:57:18** because the cost of developing software in China was so much less than America that the SAS business model didn't work as well people could just build rather than rent it out and buy and that creates inefficiency in the market I'm sure those weren't the best of breed solutions always anyways that's what the software development cost may be like 因为在中国开发软件的成本比美国低得多，SaaS商业模式效果不佳，人们可以构建而不是租用和购买，这在市场中造成了低效率，我确信那些并不总是最好的品种解决方案，无论如何，软件开发成本可能就像 **1:57:35** you know it was like software developers in 2015 in China were getting paid maybe fifth of the US and they were maybe twice as good or something like that. So 10x lower cost of software. I'm I'm making up numbers, right? Um you know they had 10x lower cost of software and so SAS never happened. 你知道，2015年中国的软件开发人员的薪水可能是美国的五分之一，他们可能好两倍或类似的东西。所以软件成本低10倍。我在编造数字，对吧？你知道他们的软件成本低10倍，所以SaaS从未发生。 **1:57:50** Cloud never happened and and at least as big of a way as it did in the US and around the world for all the companies that use that sort of that have that same economic reality and that's despite the outsourcing right to India and and and Eastern Europe and South South America, etc. 云从未发生，至少不像在美国和世界各地对所有具有相同经济现实的公司那样大，尽管有对印度、东欧和南美洲等的外包。 **1:58:04** Um you you you changed all of this with AI software development, right? um and AI SAS products generally, right? Not just AI software development. So there's two sort of coins here. So AI software development tanks the cost of building a competing software stack. 你用AI软件开发改变了所有这些，对吧？一般来说AI SaaS产品，对吧？不仅仅是AI软件开发。所以这里有两种硬币。所以AI软件开发使建造竞争软件堆栈的成本暴跌。 **1:58:17** Do you now move to a world where X can just build I can just build instead of buying renting? Two is if you are a SAS business and your customer acquisition cost remains the same and most businesses in AI and in SAS are going to remain having a high customer acquisition cost. Sales is hard. Uh breaking into a competency is hard. 你现在转向一个世界，X可以构建，我可以构建而不是购买租用？第二是，如果你是一个SaaS业务，你的客户获取成本保持不变，AI和SaaS中的大多数业务将继续拥有高客户获取成本。销售很难。打入能力很难。 **1:58:36** But now you add this AI part of it, you've now added a humongous cogs, right? Your cost of goods sold in any AI software is really hard and really big. And this is partially why I think Google also has an advantage. They have the lowest cost of goods sold for any token of any company because they have their own vertical stack on TPUs. 但现在你添加了这个AI部分，你现在添加了一个巨大的销售成本，对吧？任何AI软件的销售成本真的很难，真的很大。这部分是为什么我认为谷歌也有优势。他们拥有任何公司任何token的最低销售成本，因为他们在TPU上拥有自己的垂直堆栈。 **1:58:52** Anyways, coming back to this because you have this high customer acquisition cost and you have this high uh cogs and then the cost of anyone developing it themselves or competitors in the market means you're going to have a very fragmented SAS market or they're just going to build it themselves and therefore you never hit the escape 无论如何，回到这个，因为你有这个高客户获取成本，你有这个高销售成本，然后任何人自己开发它或市场中的竞争对手的成本意味着你将拥有一个非常分散的SaaS市场，或者他们只是自己构建它，因此你永远不会达到逃逸 **1:59:11** velocity where your customer acquisition cost and your R&D get and get amortized and because you have such a high COGS your amortization point means your gross your net profitability is actually much worse and so I think like the era of like software only businesses is really really tough in the age of AI now already scaled businesses can do great 速度，你的客户获取成本和研发成本被摊销，因为你有这么高的销售成本，你的摊销点意味着你的总净利润实际上要糟糕得多，所以我认为在AI时代，仅软件业务的时代真的很艰难，现在已经规模化的业务可以做得很好 **1:59:28** right I think YouTube is going to have its glory days and I'm sure it'll it'll always be amazing but with with with the cost of generation of content falling and falling creating content he who controls the platform is going to win and win and win 对吧，我认为YouTube将拥有它的辉煌日子，我确信它总是很惊人，但随着生成内容的成本下降和下降，创建内容，控制平台的人将赢得胜利 **1:59:41** but like you know there's like the functionality you build within Salesforce is actually going to be like way less like what you can build on your own like or like you know there's there's or like whatever it is. I'm not saying it's a take on Salesforce itself specifically, but I think many software businesses will have a reckoning with the fact that their COGS is going to sore, their customer acquisition cost isn't going to fall and they have a lot more competitors 但就像你知道你在Salesforce中构建的功能实际上会比你自己可以构建的少得多，就像或者你知道有或者无论它是什么。我不是说这特别针对Salesforce本身，但我认为许多软件业务将面临他们的销售成本将飙升，他们的客户获取成本不会下降，他们有更多竞争对手的事实 **2:00:04** and so then they don't hit that escape velocity. And I think that's the the thing that uh maybe software um it's something I've I've like sort of like thought about. There's a couple people in my company Doug Douglas Olaflin and he's the one whose idea this actually is. 所以他们没有达到那个逃逸速度。我认为这是，也许软件，这是我一直在思考的事情。我公司里有几个人，Doug Douglas Olaflin，这实际上是他的想法。 ## The Kindest Thing ## 最善良的事 **2:00:16** This has been incredibly fun. I I love love learning from you and listening to you and reading what you put out. I think I think you're just one of the most um energetic and awesome thinkers in this whole space right now. So, thank you for all the work you've done. When I do these, I ask the same traditional closing question. What's the kindest thing that anyone's ever done for you? 这太有趣了。我喜欢从你那里学习，听你说话，阅读你发布的东西。我认为你是现在整个领域中最有活力和最棒的思想家之一。所以，感谢你所做的所有工作。当我做这些时，我问同样的传统结束问题。任何人为你做过的最善良的事是什么？ **2:00:34** Done for me? H I mean, it have to be my brother. Everything he's done in my life. Um I've been a [ __ ] my whole life and I still am a [ __ ] Um, and so like every time he like pulls me back on path, he corrects me. He loves me unconditionally. 为我做？嗯，我的意思是，必须是我的兄弟。他在我生命中所做的一切。我一生都是个混蛋，我仍然是个混蛋。所以就像每次他把我拉回正轨，他纠正我。他无条件地爱我。 **2:00:52** I think my brother is probably the most he's done the kindest things for me, right? And I've been an [ __ ] like so much of my life, right? Like unconsiderate and like everything, right? He's just always been there for me and always been Why were you an [ __ ] Why? Yeah. If you're aware of it, it makes it into No. No. It's terrible. 我认为我的兄弟可能是为我做了最善良的事情的人，对吧？我一生中的很多时候都是个混蛋，对吧？就像不体贴和一切，对吧？他总是在那里支持我，为什么你是个混蛋？为什么？是的。如果你意识到这一点，它会变成不。不。这很糟糕。 **2:01:05** Yeah. And maybe this is like the MMO of like who I am and maybe that's why I'm like a good thinker, but like I I like vibe really hard and I'm in the moment really hard and I digest tons of information, but I'm very like bad at like um like how would I say like task orientation, remembering to do specific things. 是的。也许这就像我是谁的MMO，也许这就是为什么我是一个好的思想家，但就像我真的很有共鸣，我真的活在当下，我消化大量信息，但我非常像不擅长像，我怎么说，任务导向，记得做具体的事情。 **2:01:24** Like I'm very bad at those things. And thankfully I've like been able to surround myself in my life whether it's through birth or not. um with people who help me with the things I'm bad at because I'm very bad at a lot of things like I think like you know as far as like you know radar plot of like how good I'm at things 就像我非常不擅长那些事情。谢天谢地，我能够在我的生活中围绕自己，无论是通过出生还是不。与帮助我解决我不擅长的事情的人，因为我在很多事情上非常不擅长，就像我认为，你知道，就像你知道我擅长事情的雷达图 **2:01:37** and so when I don't like call people or like think be considered of what they're thinking because I'm just vibing and I'm doing whatever you know I'm like like kind of like focused in on like this path and like that path ends up hurting someone else right whether it's like hey I didn't I didn't call someone or I didn't like think about their feelings when I did an action or when I said something that makes be an [ __ ] right? 所以当我不喜欢打电话给人或不考虑他们在想什么时，因为我只是在共鸣，我在做任何你知道的事情，我就像有点专注于这条路，那条路最终伤害了别人，对吧，无论是像嘿我没有打电话给某人，或者当我做一个行动或说一些话时我没有考虑他们的感受，这让我成为一个混蛋，对吧？ **2:02:02** And yes, I should be more conscious of this and I try to be, but it's like it's just one of the things I'm going to wrestle with in my life forever. And a lot of times I don't even realize I'm being a freaking idiot until my brother's like, "You're a freaking idiot." God for your brother. And so like I I you know, that's the kindest thing anyone's ever uh done for me is like my brother through my whole life. 是的，我应该更加意识到这一点，我试图这样做，但就像这只是我将在生活中永远挣扎的事情之一。很多时候我甚至没有意识到我是个该死的白痴，直到我兄弟说，"你是个该死的白痴。"感谢你的兄弟。所以就像我，你知道，任何人为我做过的最善良的事情就像我兄弟在我的整个生活中。 **2:02:19** I love it. I love it. Wonderful place to close. Thanks so much for your time. Thank you so much. Yeah. 我喜欢它。我喜欢它。很好的结束地方。非常感谢你的时间。非常感谢。是的。 --- **翻译完成。这是一次关于AI、计算、硬件、未来以及商业模式的深度对话，涵盖了从OpenAI与Nvidia的交易机制，到中美AI竞争，再到个人对未来的思考等广泛话题。**