Short answer: Jordan Metzner says servers left on led to a $35,000 bill and one developer using a new Cursor feature cost $2,000. He expects efficiency scores comparing token use with each worker's output.
Watch this answer from Episode 36 (starts at 13:09) · Episode 36 notes and transcript
From Episode 36: How We Built an AI Video Editor for Recruiters (Remotion + Claude + Codex), starting at 13:09.
Sam Nadler [13:08]: Awesome. Well, that queues us up for the news. So let me this is kinda top of mind for us. So we're on internally here on a big push to pretty much ramp everyone in the organization on AI regardless of job title, regardless technical or nontechnical. We've got multiple people pursuing, I wouldn't even say side projects, but just for the sake of this conversation, like AI driven side projects, if their role isn't inherently technical, just on a really big push to ramp our entire organization on being very AI savvy. And this article stood out to us. You finally figured out AI at work. Now here comes the bill. AI output, which has turbocharged productivity and promises to change the nature of work, might seem like it materializes out of thin air, but it's really the work of data centers churning through prompts and interpreting them in elaborate and expensive, if unseen, process.
Jordan Metzner [14:04]: Yeah. So, basically, the real question, now we're giving everyone access to all of these tools. I think, we've given everyone access to Claude Code, Claude Cowork. We have people using Codex. We've had ChatGPT for a while now and other models. I think everyone has Gemini access, but, not all the models are the same and not the cost of use of all these models are the same. And, currently, like, we're just kind of telling everyone just like use whatever you want, whatever model you want, but, optimizing your model used for output will be important if you kind of see that your spend gets out of hand and like, we've seen kind of leaving servers on accidentally, how that left us with like a $35,000 bill. We saw recently how one developer used a new feature in Cursor and left us with like a $2,000 bill. So, this has hit us like in the bottom line for sure. And, I think both, these companies as well as ourselves are putting in like kind of more guardrails.
Jordan Metzner [15:01]: But, at the end of the day, I think like you're gonna look at token input and output and then output of the worker and, kind of get an efficiency score of kind of like is this worker like, using these additional software pieces in a way that provides, good value for the organization? Or, are they just wasting money kind of thing? And that's that's an interesting discussion and a new format and a new way to manage, I think.
Sam Nadler [15:25]: Yeah. Totally. I mean, I don't think we I think you mentioned that we've had two little surprise two well, a big surprise and a little surprise. But, there is going to be it's going to be managed both, I think, how an employee uses AI, both on, like, are they adopting? Are they maximizing their time on the job? Are they, like, driving things forward that, could be faster with AI? And also, how much they're spending doing it. Are they being reckless? And, I think, like, our internal organization, we haven't gotten to the, throttling or managing what tools certain teammates get based on, their role or what models they have access to yet, but I couldn't you know, we're one big bill away from it being an important part of how we provide access.
Jordan Metzner [16:14]: Yeah. I've heard of large organizations giving users access to using LLMs and coding, but only a certain amount of tokens. And so they only use it kind of, like, selectively as an example rather than just using it going all out. And so, yeah, I think, like, kind of cost consciousness, obviously, open source is exciting in this area where if you can host your own models and you can get those kind of on, old servers or servers you already have, then you maybe can save money in that way and kind of drive up efficiency and allow everybody to become more kind of token consuming. But, and then at the same time, obviously, these models are getting cheaper. Right? So, using what previous model gets you today for the same model, you get just way more output, right, and way higher quality as well.
