Short answer: Sam Nadler says GPT-5's routing gave quick answers when he wanted thinking. Jake Trefethen says users mourned GPT-4o's personality, and Jordan Metzner suggests the router may be a cost-saving mechanism.
Watch this answer from Episode 8 (starts at 22:57) · Episode 8 notes and transcript
From Episode 8: Can Public Make You a Better Investor?, starting at 22:57.
Jordan Metzner [22:57]: I think that was great. We recorded last week the same day ChatGPT five came out, so it's been almost a week now since it's came out, and it's been quite controversial for many different reasons. But, yeah, Altman's obviously been pretty loud about trying to make changes quickly to satisfy the public, but just curious, maybe to Sam, Jake, and Emily, kind of what have your been experiences on using ChatGPT five, and do you like it better or worse? I've heard mixed opinions from pretty much everyone I know.
Sam Nadler [23:27]: For me, it's been a little frustrating. There's been times, and I know this, like, auto fast thinking feature has come out, but there's been times when I've gotten the quick answer when I want the thinking answer, and it's just I've had to, re prompt to think deeply and just kinda it's there's been, I feel, more back and forth, whereas previously, I could just pick the model that I thought best suited my needs right away. What about you all, Jake and Emily? Have you had much experience with GPT five in the past six days?
Jake Trefethen [23:58]: I think I definitely gotta echo some of your struggles with it. I don't think I've sent it a single prompt where it didn't try to just split test thinking mode and quick mode. So I don't know. It sounded like from the article too that I don't know if the router was even really working when they launched, but it sounds like they've been dialing it in. I thought another interesting part from the article was that some customers kind of were mourning the loss of the GPT four o's personality, and that, Altman acknowledged basically that, we're gonna ultimately have to get to a point where managing the personality of your model is something that's on like the customer side rather than on, like, a model selector kind of front, which is is pretty funny. And I think it said something also about, like, some funeral in San Francisco for Sonnet three five.
Emily Kurtz [24:50]: I think it's the same. I think they really wanted, to try to solve everyone's needs and sort of take the decisions out of your hands. And I think it's realizing that that was probably done a little prematurely for where the product was, and people are wanting that flexibility and that control back.
Jordan Metzner [25:06]: What about that it could be a cost savings mechanism where, basically, customers are using expensive models to do low quality tasks, and that's costing them a lot of money. And so, in fact, they wanna make the decision on which you know, how much resources to put towards the query, which will allow them to save a lot of money considering the scale.
Emily Kurtz [25:24]: Yeah. I mean, I think it's a good point. I'm supportive of that, especially when you think through the impact on the environment of all of everything that's being run. But I think this ecosystem has been evolving so rapidly, and so customer, all of us just expect so much from them. And so, unfortunately, like, if you wanna make that decision, great, but it needs to be done in a product that's that's pretty robust and still doesn't feel like a step down to all of our users, and I think, unfortunately, that's not what happened here.
Jake Trefethen [25:52]: I was gonna say it's definitely hard for them to avoid the buildup of hype. I don't you know, it's been a while, people waiting on GPT five, and I think the allure of the name, definitely kind of maybe got out of hand for them as far as, what they were able to actually deliver. I know that I heard it conflated with, that's AGI when we hit GPT five and stuff like that for at least, whatever, six months. So, for it to be kind of a router adjustment, cost saving, slightly different model is is always gonna be a disappointment regardless of what broke on day one.
Jordan Metzner [26:26]: I think I mean, it's disappointing. It doesn't it's not a better coding algorithm, and it didn't beat, the Anthropic model, so seems like everyone's gotta get back to work.
