This is such a fun and fantastic breakdown. Love an end of year listicle, but even better because you added all the ways a good product person should have caught these things. Great read!
Many companies are more interested in quickly capitalizing on AI than in implementing this technology robustly, reliably, and safely. I agree that a lot of teams are perfectly capable of addressing these issues, but there's probably just too much pressure to deliver.
Yes, I think so too. I really hope I get to meet the PMs and engineers from those teams someday and learn what it looked like from behind closed doors.
I only knew about #3 issue, the others are new to me. It sounds like cases like those are keep happening cause we need to "race to AI faster so that China doesn't win" (or something like that).
Also, "pesky humans" that don't fit the *ideal user* profile. It makes things hard for us builders. 😆
Yes, I think so too. Companies like Anthropic set a better example, they name the risks and delay launches when needed. Hopefully that becomes the norm.
I'd bet my money that Anthropic is the company that will last longer in this "race" cause they're actually thinking about the security & privacy as main concerns and not an afterthought.
Thanks, Karo. This is a really interesting post. It definitely made me raise a couple of wry smiles as I was reading through it. And there was definitely an element of schadenfreude as well.
As you say, it's really interesting that people forget that there's no such thing as an ideal user and that actually that's what we need to use beta testers for as well. Thanks so much for demonstrating this with your own practise and for reminding us that you're never too big to get things wrong.
The Grok case was absurd, but Taco Bell’s speaker system ordering chaos had me simultaneously laughing and cringing! What’s clear is this: product velocity without sanity checks is just fast failure.
What’s your take on how teams can actually bake this kind of red-teaming into weekly sprints without slowing to a halt?
Great question! It’s a mindset and ways-of-working shift, where compliance, legal, security, engineering, design and PM work together as one cross-functional team. Same sprints, same retros, same objectives.
In my experience, this makes the work faster, not slower. Fewer silo walls to break through, and far more shared wins.
I chuckled at a few of these gotchas that made your list, Karo. Some of them common sense. I look forward to reposting on LinkedIn where more than a few PM and PPM spend time—I’m sure they’ll appreciate your list ✅!
Thank you fir mentioning my Holiday Scavenger Hunt (by way of StackShelf and Pink Slip Pivot)! 😁
Awesome list Karo. As a risk taker & creative leader I'm famous for celebrating mistakes. When we chastise mistake making the team stops taking risks so that's my rationale.
Totally with you! Bold experiments and the occasional misstep are part of building great products. We celebrate those too. I recommend that they're done before launch, not after. When real people are involved, their wellbeing outranks any learning goals 🤗.
Loved this one, so many lessons unpacked. My takeaway after reading this is how important it is to include edge cases and not design systems only for "ideal" users / situations.
I didn't hear of most of these, but rolling out without testing is what causes the failures. It is very similar to the number of times that MS Windows created patches and told every organization to roll them out without first testing. Complete chaos every single time, that easily could have been avoided.
Based on these examples, maybe we should be more worried about privacy breaches and AI run amok from big enterprises than from small vibe-coded apps? 🤣
I loved this retro. I am in violent agreement with all of it. The self-inflicted chaos of it all just to push another release where the public is used as QA.
There is too much focus on the launch and not enough on stability and UX. At least not from a prioritization scale. The sad part is it still happens, even with sufficient examples.
Consumers aren't clueless. Your product either works or it doesn't. It either provides a frictionless experience or it spotlights the disorganization and gaps. The result doesn't care how far you push items down the backlog. The customer, who didn’t sign up to be a tester vy the way, has already seen the delivery.
A thousand pardons for the thesis. This one hit a little to close to home. 😉
This is such a fun and fantastic breakdown. Love an end of year listicle, but even better because you added all the ways a good product person should have caught these things. Great read!
Thank you so much Katie!
Many companies are more interested in quickly capitalizing on AI than in implementing this technology robustly, reliably, and safely. I agree that a lot of teams are perfectly capable of addressing these issues, but there's probably just too much pressure to deliver.
Yes, I think so too. I really hope I get to meet the PMs and engineers from those teams someday and learn what it looked like from behind closed doors.
I only knew about #3 issue, the others are new to me. It sounds like cases like those are keep happening cause we need to "race to AI faster so that China doesn't win" (or something like that).
Also, "pesky humans" that don't fit the *ideal user* profile. It makes things hard for us builders. 😆
Yes, I think so too. Companies like Anthropic set a better example, they name the risks and delay launches when needed. Hopefully that becomes the norm.
I'd bet my money that Anthropic is the company that will last longer in this "race" cause they're actually thinking about the security & privacy as main concerns and not an afterthought.
Totally, trust is the only real moat left, and they’re actually investing in it.
Thanks, Karo. This is a really interesting post. It definitely made me raise a couple of wry smiles as I was reading through it. And there was definitely an element of schadenfreude as well.
As you say, it's really interesting that people forget that there's no such thing as an ideal user and that actually that's what we need to use beta testers for as well. Thanks so much for demonstrating this with your own practise and for reminding us that you're never too big to get things wrong.
Thank you for reading and for your thoughtful comment Sam! I had to check what schadenfreude means 😂
Omg, the Grok story was shocking for me. You’re right, so many open questions on governance and ethics. 🩷🦩
Yes, it was. I’d love to know how it looked from their side.
The Grok case was absurd, but Taco Bell’s speaker system ordering chaos had me simultaneously laughing and cringing! What’s clear is this: product velocity without sanity checks is just fast failure.
What’s your take on how teams can actually bake this kind of red-teaming into weekly sprints without slowing to a halt?
Great question! It’s a mindset and ways-of-working shift, where compliance, legal, security, engineering, design and PM work together as one cross-functional team. Same sprints, same retros, same objectives.
In my experience, this makes the work faster, not slower. Fewer silo walls to break through, and far more shared wins.
Amazing! Great examples 👌 ❤️
The kid dying to push "launch" haha
heheheh 😂
The disregard to privacy is criminal in some of these cases. This is why you need anticipatory governance to help deal with some of these issues.
100%. Thank you for reading Ousmane!
I chuckled at a few of these gotchas that made your list, Karo. Some of them common sense. I look forward to reposting on LinkedIn where more than a few PM and PPM spend time—I’m sure they’ll appreciate your list ✅!
Thank you fir mentioning my Holiday Scavenger Hunt (by way of StackShelf and Pink Slip Pivot)! 😁
Thank you so much Dee, that would mean a lot!
My pleasure, Karo!
These case studies are insane. i had no idea! Thx for sharing Karo.
My pleasure, glad you enjoyed it! Thank you for reading 🤗
Awesome list Karo. As a risk taker & creative leader I'm famous for celebrating mistakes. When we chastise mistake making the team stops taking risks so that's my rationale.
Totally with you! Bold experiments and the occasional misstep are part of building great products. We celebrate those too. I recommend that they're done before launch, not after. When real people are involved, their wellbeing outranks any learning goals 🤗.
You're the star player! Have a brilliant end to 2025 !
You too!!!
Loved this one, so many lessons unpacked. My takeaway after reading this is how important it is to include edge cases and not design systems only for "ideal" users / situations.
That's correct. Thank you so much for reading Daria!
I didn't hear of most of these, but rolling out without testing is what causes the failures. It is very similar to the number of times that MS Windows created patches and told every organization to roll them out without first testing. Complete chaos every single time, that easily could have been avoided.
That's a great example too!
Based on these examples, maybe we should be more worried about privacy breaches and AI run amok from big enterprises than from small vibe-coded apps? 🤣
That's a very good point 😂
I loved this retro. I am in violent agreement with all of it. The self-inflicted chaos of it all just to push another release where the public is used as QA.
There is too much focus on the launch and not enough on stability and UX. At least not from a prioritization scale. The sad part is it still happens, even with sufficient examples.
Consumers aren't clueless. Your product either works or it doesn't. It either provides a frictionless experience or it spotlights the disorganization and gaps. The result doesn't care how far you push items down the backlog. The customer, who didn’t sign up to be a tester vy the way, has already seen the delivery.
A thousand pardons for the thesis. This one hit a little to close to home. 😉