Short answer: Jordan Metzner says Karpathy released an auto research tool on GitHub that trains in five-minute increments. Iliya Valchanov says it is a blueprint for running experiments at a scale and cost never seen before.
Watch this answer from Episode 35 (starts at 22:06) · Episode 35 notes and transcript
From Episode 35: AI Agents Are Replacing Entire Marketing Teams, starting at 22:06.
Sam Nadler [22:06]: Well, speaking of crazy, let's cover our last story, which Jordan, I think may be helpful for you to give us an overview of this new open source auto research that lets you do hundreds of AI experiments. Ilya, you may know a bit more than me about this, but I know Jordan has some context and, just give us the download.
Jordan Metzner [22:25]: Yeah. So Andrej Karpathy, one of the founders of kind of AI overall, and he's the originator of the word vibe coding. He worked at OpenAI and Tesla and has been building AI tools for the public for the last few years. And this week, he launched kind of two new pieces of technology both on his GitHub. This auto agent tool, and then this other tool that we're mentioning here about this kind of build your own model. And I've just seen some incredible feedback of what's going on in the market of people saying that like this has been incredibly helpful. I think this morning, Tobi from Shopify said that this auto research tool works really well on even code bases and other non kind of research based products. So it is a really exciting time of this, like, kind of mini model of training the model and kind of then having the model get better and you know, one thing that's pretty interesting, I don't know if you read about this Sam, but the model training works in five minute increments and it's specifically timed so that it limits the amount of training it does to a maximum of five minutes. And I think what makes that interesting is that, almost all training of these models takes hours and days and weeks and months and all of that. And I think that it was a really interesting function like selection of like code choice that he selected to have this model trained on only five minutes at a time.
Jordan Metzner [23:41]: And it gives this ability of, back to what Ilya said, I can only do so many agents that are only so busy, etcetera. It's like, you can have your agent trained for five minutes, go work on another task or so, and then five minutes later you're gonna get results. And then, seeing those results and then iterate again and again. And so I really like this idea of kind of like these small mini experiments, but, you can do that obviously so many times an hour and so many times a day, etcetera. Wow.
Sam Nadler [24:04]: Yeah. Ilya, did we did you get a chance to read the article?
Iliya Valchanov [24:09]: Yeah. Yeah. I actually read the GitHub repo when we came out, and it was very interesting because it's it's very small and stripped to the essence, and only someone like Karpathy can do it so, to the point. So it was very, very cool to see. In my background, we were teaching people how to train these models, and, one of the first things that that we did so so imagine, we've created more than a 100 courses, AI and how to train models. And then Gypsy comes out, and everything that we've been teaching people is, like, in the tip of your hands, and you get the scripts. They're ready to use, and then the question is, what do you do with them? And then as the hardware becomes faster and faster, and you can do all these experiments, one thing that stuck in my brain was a friend of mine came and he said, I saw your website, and it said that you're doing, let's say, AI for marketing, And then I refreshed the page, and it said the same thing. Why is this the case? You should be AB testing every single moment of every single minute, and you should be, constantly optimizing on every page, every word.
Iliya Valchanov [25:22]: And I'm like, well, this sounds like an overkill. And he's like, but now it's cheaper than ever to do it, and, I can't imagine why people are not doing it yet. And this article was basically saying, okay. Here's kind of a blueprint to start running these experiments in a scale never seen before and at a cost never seen before. So I was very excited about this spin of it with the experiments. So I don't know. This makes every website a living being. Right? So websites are they're no longer static.
