Short answer: Ivan Lee says Meta's new 100 billion parameter model is built and tested to run on two M5 Mac Studios, low five figures for a company, while some clients have bought millions of dollars of hardware.
Watch this answer from Episode 56 (starts at 7:18) · Episode 56 notes and transcript
From Episode 56: He Built Siri at Apple. Now His Private AI Runs Inside the FBI, Google, and Netflix., starting at 7:18.
Jordan Metzner [7:18]: Okay. I think that's like a super interesting point because that basically means we're reaching the asymptote of like log... Rig myth curve. Right? Essentially, where they're kind of all starting to meet at the same place and the marginal gains are diminutive. And what I think is proof of that is if... I think Claude announced Duet yesterday or the day before, and they announced that they built it on top of Opus five and a half. And I think, as you mentioned, now we've got, a few other models that are, I would say, pretty close to comparable to OPUS 5.5 on the market, especially in the open source world. And so that line just keeps getting closer and closer together, right, and flatter to your point as well. So maybe you can talk to a little bit about kind of how that has an impact on hardware and demands and kind of... I mean, I know I think they're talking about AWS being out of hardware, and everybody wants kind of the same hardware to run these machines. So tell us a little bit about kind of, like, how hardware has had an impact on your ability to create these kind of private clouds, for lack of a better term.
Ivan Lee [8:23]: Yeah. Absolutely. Well, I think I would point to Meta's most recent announcement. They have this incredibly powerful 100,000,000,000 parameter model that they just released. And alongside that announcement, they mentioned that it can... It's built and tested to run on two M5 Mac Studios. Now for maybe consumer you and me, that's a little expensive for us to bring into our homes. But for the average company, that's, low 5 figures, to have your own 100,000,000,000 parameter model. That is incredible. So I think, yes, there is a factor of, like, the hardware.
