Short answer: In a GroqCloud demo, Jordan Metzner has the OSS 120b model write 10 marketing emails in 5.3 seconds, then a rewrite with no dashes in 4.2 seconds, and says the cost is going to zero.
Watch this answer from Episode 7 (starts at 17:47) · Episode 7 notes and transcript
From Episode 7: How Daily Check-Ins Fix Remote Work, starting at 17:47.
Jordan Metzner [17:47]: And you could have another one that's just working on documentation, another one that's just working on something else, and, literally imagine that the cost of running these LLMs is going to zero. And I just wanted to give a quick demo of it. I don't know if we'll have time to chop it up or put it in, but this is GroqCloud. This is a open source cloud solution provider where, they have hosted models that are open source, including Facebook models, and here we have the OSS one twenty b model. And what's cool about GroqCloud is it's incredibly fast. So I wrote here you're a marketing expert as my system prompt, but you talked a little bit about clay and customization. But, let's just say I wanted to do something like, write me, let's just say, 10 emails to tell people about our new advertising for our podcast built this week, the AI podcast with Sam and Jordan. Cool? So, it may not know about built this week since it's like a pretty new podcast, and it might not be in its repertoire already. But if you just watch the speed, and we're gonna go one, two, three, let's go.
Jordan Metzner [19:02]: And boom. I mean, email's done seven, eight, nine, ten. And there you go. There's 10 emails. And so you could just see that, maybe they're not perfect. Maybe we need to go back, maybe I even wanna say, like, no dashes. And then, let it run again. The last one took five point three seconds. So now it's writing it again with no dashes, and that one took four point two seconds. So you can see just see the power and speed of leveraging these models, and the cost is, going to zero. And if the cost goes to zero or, close to zero, it means that you can do a lot more things. So yeah. Any questions about that one specifically?
