Short answer: Jim Olsen says that at one manufacturer, a model deployed to customer sites was trained only on one employee's desktop, with no tracking or versioning, and he saw that pattern again and again.
Watch this answer from Episode 55 (starts at 12:41) · Episode 55 notes and transcript
From Episode 55: Every Client Is Fortune 500. What Do They Know About AI That You Don’t?, starting at 12:41.
Jordan Metzner [12:41]: And Jim, like, how did you guys get into this? Kinda where did this thing start, and how did you get it to where it's at today? Obviously, we've seen so much adoption and change in AI in and of itself. So how did it kinda... Where did you guys start with the product and get it to where it's at now with kind of what we've seen in the market and whatnot.
Jim Olsen [12:56]: Yeah. Well, as I mentioned, we were working... I was doing consulting at the time with some very large manufacturers, banks, etcetera, on that, and we were... It is in the day of the big data, the data lakes, all that stuff, and that kind of things. And we were helping them to establish these kinds of huge data lakes for use specifically for training AI models. They wanted to basically, one was a manufacturer, and there's hundreds of models they had that ran on their individual machines that were critical to their business. When working with one of those people, I actually ran into a situation where the only place where that model was being trained, that was being deployed into customer sites, was on a guy's desktop. He's just right there. If this person got hit by a bus, god forbid, tomorrow, they would have no idea how to reproduce it. They had no tracking on it. They didn't know what went in it. They had no versioning. And I saw this again and again where, there was just nothing behind it. Nobody had any insight into what was being deployed. So, we realized there needs to be a different process for that to actually track this information, validate the information, make sure you're compliant with any regulations, all this stuff.
