Did weak customer checks by AI labs lead to model restrictions?

Short answer: Jordan Metzner argues labs grew fast without the KYC controls clouds like AWS use, citing reports of 25,000 Chinese accounts hacking onto the model, and says it is too early to tell how AI should be regulated.

Watch this answer from Episode 49

Watch this answer from Episode 49 (starts at 13:39) · Episode 49 notes and transcript

What was said on Built This Week

From Episode 49: A Ten Minute Phone Video Replaces Hours of Home Inspections. We Demo the AI Doing It., starting at 13:39.

Jordan Metzner [13:38]: Let me just play a little devil's advocate there because I don't think I do totally disagree with you, but, I think, what part of this is, just these companies trying to move fast to grow revenue versus kind of really doing a lot of deep KYC and understanding who the customers are and using their tools. Because, if you think of, I think the equivalency of looking at, like, know, the top tier clouds. Right? You can't easily go on AWS to launch a server or scale of servers as a no name user that just created an account in the first day, you know. And even just to get a bigger server, you need the permissions and you need acceptance. This is a whole series of controls put in place in order for you to do something that, like, could be somewhat dangerous. And, obviously, these are startups that are VC backed and are, like, growth at all costs. And I think the news said something like they found, like, 25,000 Chinese accounts, like, hacking you know, basically, like, hacking onto the model. That doesn't sound like they put, like, great KYC in place and, like, IP checks and really, like, know who their customer was. So, how much of this was just, like, maybe these laboratories being a little flagrant with their behavior, and then this happened with a powerful model.

Jordan Metzner [14:47]: And, have they put, more controls in place earlier on, maybe wouldn't have seen this type of behavior. But, I don't know. I mean, like, self regulation might be the best. I don't know if regulated industry, to your point, like, is AI like pharmaceuticals, a car, or is it more, like a coding language or a database, technology or something like that that, shouldn't be regulated. I don't know. For me, it seems too early to tell, but I hear your point.

Sam Nadler [15:16]: I think you both had really interesting points. It feels like we're getting into the adolescence era. It feels like these models are becoming more and more powerful, and, having a rollout framework makes a lot of sense. It doesn't seem like it's really there yet. But, yeah, I think, as these models become more and more powerful, just something urgent or something defined needs to be created.

More questions from this episode

Go deeper

New episode every week

Watch us build with AI, live.

Real builds, founder interviews and the AI news that matters. Join 43K+ builders on YouTube.

▶ Subscribe on YouTubeBrowse all episodes →