Should product manager interviews require vibe coding like Shopify's?

Short answer: Jordan Metzner says asking PMs to vibe code is obvious. Emily Kurtz says it suits prototypes but is risky for brokerage production code, and Jake Trefethen says he would question the curiosity of a PM who never tried it.

Watch this answer from Episode 8

Watch this answer from Episode 8 (starts at 26:35) · Episode 8 notes and transcript

What was said on Built This Week

From Episode 8: Can Public Make You a Better Investor?, starting at 26:35.

Sam Nadler [26:35]: I do wanna transition to the next. I do have three PMs on this call. Jordan, you are excited about this post. I don't know what platform it was from, that Shopify is requiring vibe coding in all PM interview processes. Do you think that's fair? Do you think that's necessary? Do you think that's ridiculous? What are your thoughts here?

Jordan Metzner [26:55]: No. I mean, it seems like so obvious. How could you not ask, a PM to vibe code in an interview? Just like how could you not ask a developer to use AI to code an interview now? And it almost be, the equivalent of asking someone to do, complicated math without a calculator just to see if they could, do it by hand. And so, yeah, it seems like so obvious. So this seems like another opportunity space, but also just seems like, the way of hiring and the way of building is changing so quickly, at least in, what is our vertical, which is, product development. And I think, like, everybody becomes a product manager, and then, like, product manager also becomes everybody. So don't know. There's a huge, like, opportunity to open up here. Curious what Emily's thoughts are, especially as, like, leading product teams.

Emily Kurtz [27:42]: Totally. I mean, I think every day, right, our CEOs are saying how can everyone at the company use AI more efficiently to job and make it better. And so I think this is just, to your point, like a no brainer example. How can you do some of the work? How can you prototype? How can you put something in and do the low hanging fruit to put something in front and then have the team and the folks that you hired to really code, do the more complex architectures side of the house.

Jordan Metzner [28:09]: Yeah. What about you, Jake?

Jake Trefethen [28:11]: Yeah. I'm interested to see how it evolves. I mean, obviously so, I mean, I have a technical background and a product manager, and so it's it's always, I guess, been helpful for me, especially, like, scoping AI features to understand, like, feasibility. So I think having a technical angle to any PM is definitely helpful as we move into, having to manage a lot more AI style kind of development. But I think it'll be interesting to see how it evolves, the specific use of vibe coding from a PM. I think today, PM's vibe coded prototype, the code's not gonna go so far as to maybe reach production. I think there may be some, developer pushback, and I don't know how reusable it's all gonna be. I think, you know, Replit, Cursor, very good at using standard, like, UI component libraries, shadcn, whatever. And so a lot of it ends up being throwaway code, but that can be very helpful in iterating through design cycles really quickly, not having to involve a bunch of different parties communication overhead and getting from idea to something that's executable super quickly with one person. I think that's that's where, like, the real time value leverage is right now. But I wonder how it'll evolve, into actually, involving PMs and production code.

Emily Kurtz [29:26]: Yeah. And I think to Jake's point, right, like, it also depends on what you're building, because we are a brokerage, and we're responsible for investing in folks' money. Obviously, like, making me vibe code something that we're gonna put into production that directly impacts, their withdrawals, funds, whatever it may be, is it's pretty risky. But, obviously, if it was if I was by putting a Slack integration around some product updates, that's a lot less risky. And so I think there is we still have to figure out the balance of when to use it, when it's just a prototype, and when it can really go live in deep production.

Sam Nadler [29:58]: Yeah. I'm curious how much of a signal it really would be because, I'm not a PM. I'm an operator. But, in a thirty minute period, an hour period, I can vibe code, at least in my opinion, a decent prototype for almost anything. So, and I probably wouldn't hire myself as a PM. So I'm just not sure how strong of a signal it would really be. But I would agree that PMs should be comfortable in creating quick prototypes for whatever they're working on.

Emily Kurtz [30:30]: Totally. And to your point, it's it's definitely not the only thing that's important. Right? How you work cross functionally, how you manage bunch of different stakeholders, how you prioritize, how you bring in the voice of the customer and communicate with them, how you make a team more efficient and sort of push them to be the best product velocity they can, all of those things you're not gonna get via coding.

Jake Trefethen [30:50]: Exactly. To your point of a signal, hiring somebody who's supposed to be inspiring others to ship software products, if they haven't played around with a vibe coding tool and tried to build something themselves, I'd definitely question their curiosity a little bit.

Jordan Metzner [31:06]: Well, on the flip side too, vibe coding my public trading bot was really fun, and I learned a lot about trading, to be honest. So I think that is a cool way to build new features and some of the things you guys mentioned and how AI is gonna have an impact on all the data flow of financial information.

Emily Kurtz [31:22]: Totally. And obviously not the topic we're talking about, but it's all it's also related to the hot topic of MCP right now.

Jordan Metzner [31:28]: Yeah. That's for sure. Yeah. Being able to connect to the datasets and being able to make decisions that way.

Emily Kurtz [31:32]: That and communicate with LLMs in, a clear way.

Jordan Metzner [31:36]: Great episode. Just wanna say thanks to Emily and Jake and the entire team at Public for helping us put this together and getting us access to the public APIs so that we could build our prototype this week. It was super fun and learned a lot about stock trading. Don't forget to subscribe and like and click the notification bell. And thanks, everyone. Great episode.

Sam Nadler [31:59]: Thanks, everyone.

Jake Trefethen [32:00]: Awesome. Yeah. Thank you, guys.

Jordan Metzner [32:02]: Bye, guys. Thanks. Built this week,

Tools mentioned

More questions from this episode

Go deeper

New episode every week

Watch us build with AI, live.

Real builds, founder interviews and the AI news that matters. Join 43K+ builders on YouTube.

▶ Subscribe on YouTubeBrowse all episodes →