Short answer: Amar Goel says PRs merge about 10x faster, from fifty hours to five to seven, with feedback in five minutes. He says Bito provides 75 to 80% of PR feedback and customers' reverted PRs fell about 55%.
Watch this answer from Episode 23 (starts at 11:36) · Episode 23 notes and transcript
From Episode 23: OpenAI Code Red? The New AI Reviewer Changing Everything, starting at 11:36.
Sam Nadler [11:35]: How do you think and that, obviously, I'm not an expert in this domain, but how do you think about, like, the quality versus speed trade off? Like, in terms of you know, I think Bito is working to help teams have higher quality code pushed to production, but does it significantly hurt, like, the speed to which they're moving?
Amar Goel [11:56]: It's actually speeding teams up. So we're seeing on average PRs merge in, 10 x the speed. So a PR that used to take fifty hours to merge is now merging in five to seven hours because most enterprise teams still have one or two people who manually go and review a PR. But Bito's agent runs and we provide feedback in, five minutes. So team like Rakuten or Gainsight or Whatfix or whatever. I mean, teams are like, hey, I got my PR feedback in five minutes. I used to have to wait maybe a day or twelve hours to get feedback. So the developer can quickly address those. Then the senior reviewer who's comes in you know, the developer who submits that code and the reviewer are still responsible for the quality of that code. You know? And so but then that senior reviewer who comes in is like, okay. I don't have to deal with all the basic blocking and tackling. I can focus more now on kinda architectural level thinking, high level design. So they also know that the PR review isn't this mind numbing task and going through the code line by line and being responsible. So they come in a couple hours.
Amar Goel [13:00]: They don't have as much feedback to give. So the amount of typically, we're providing, like, 80 75, 80% of the total feedback that a PR receives. And, where customers are seeing bugs go down by, like, 53, 55%. Like, the number of kind of reverted PRs, in the month following is down by 55%. Right? So it is helping speed the team up and do a better job because it doesn't get tired. So, it kind of feels to me like a no brainer, know.
