There's all kinds of hype every single day about AI taking the place of programmers. That reasoning infers that typing is the hard part of product development.
Nothing could be further from the truth.
Here are the hard parts of product development:
- Understanding what the customers want. Often, the customers don't know until they see something. That means we need a low cycle time to create and offer our deliverables, especially if those deliverables are prototypes.
- Understanding the essential and accidental complexity of the product. Developers and testers spend time reading and understanding what already exists: code, tests, or other documentation. Often, they spend much more time reading and understanding rather than typing. Managers and team members consistently underestimate the amount of time people require to understand what already exists.
- Validating the product with the customer to create some impact. Sometimes, that's a continuous flow of feature deliveries. Sometimes, that's providing the customer a way to integrate this new feature into existing workflows, systems, and other products. That means the team needs to complete all the testing and packaging for this release. That's the total cycle time or lead time for the entire product.
What do all of these have in common?
People collaborating with other people to learn. That learning allows people to deliver. The faster the learning, the shorter the cycle time.
Because we need low cycle times for the hard parts of product development, we need everyone focused up on that overarching goal. Not down to what a single person does.
The LLMs Encourage Optimizing Down
Right now (July 2025), people use LLMs to focus on what one person does. Some people claim that they're pairing with an LLM. That's not how I think of pairing.
Every time I've paired, my human partner helped me gain more insights into the total effort, a form of optimizing up for the overarching goal. Because LLMs are predictive models, yes, they might offer you more exposure based on someone else's already-existing work. Not brand new ideas.
I realize that learning about how other people think can offer you innovative ideas. Not every problem needs to be invention, as in the right side of the problem determinism continuum.
However, given the current degradation I see in too many products, I wish people would consider more new ideas.
Instead of pairing with an LLM, I see people using an LLM as a rubber duck partner or as a way to spike work. However, these people work alone, not collaboratively. That shortcuts all the possible team learning. (See Need to Learn More About the Work You’re Doing? Spike It!)
When we don't optimize for team-based collaboration, the cycle time increases. I see way too many individuals working alone, in their area of expertise along with an LLM. That's the definition of a component team. (See Component Teams Create Coupling in Products and Organizations for why and how to reduce that cycle time.)
The result: LLMs might make each person more “productive,” whatever that is. But unless the team collaborates on limited WIP, the LLM does not reduce overall cycle time. I am sure some teams have figured this out. But not enough.
The Scree: Promoting Resource Efficiency Instead of Flow Efficiency
The hard parts of product development all deal with verifying and validating a product hypothesis. With any luck, if we deliver incremental features on a regular basis, we can get some feedback that tells us if that feature (and then the feature set) offers value to the customer. But too often, we need to release enough features to offer the customer sufficient outcomes.
That requires a focus up, on a team's or the project's overarching goal. For larger efforts, the program's overarching goal.
That focus on the individual and “their” work is an example of resource efficiency thinking instead of flow efficiency thinking. Resource efficiency focuses “down” on the individual, regardless of the overarching goal. (The one hope I have is that working with an LLM reduces the individual person's WIP.)
Instead, the hard parts of product development require us to focus up on an overarching goal. We can do that if we use flow efficiency thinking because we all reduce our cycle time and maintain a reasonable WIP.
The Hype Cycle Is Alive and Well
Why is everyone so hyped up about these LLMs?
They do show promise. I rarely use Google Search anymore, because it's so terrible. Instead, I use ChatGPT. Sure, I have to check every single “fact” it offers, but that's far superior to the garbage I get from a search.
I also use Chat and Grammarly to improve marketing copy, such as my bio, book blurbs, etc. (Not to improve regular writing, because the LLMs want to homogenize my author voice. No thank you.)
Have you noticed how the hype cycle, the image at the top of this post looks just like the Satir Change Model? I am not opposed to change—far from it. But I sure would like to know I can depend on the tools that offer such hope for change.
Right now, the LLMs offer a promise for innovation. They are not there yet. And the more we work alone, the more likely we increase both WIP and item aging. Then we kick off reinforcing feedback loop dynamics in the flow metrics.
If you want to use an LLM, consider collaborating with at least one other person as you do. Or, consider the LLM as a rubber duck. But don't expect the LLM to think for you. We're calling these things “AI.” They are not. They are statistically predictive of what other people have done. Not innovative work for your overarching goal.
That's the hard part of product development and why we need teams of humans (possibly augmented with fast predictors) to create products customers will buy.
Yes, yes, and yes.
I laughed out loud. Thank you.
Love the focus on teams rather than individuals, but I have a couple of comments on things that I believe should be different in the spirit of continual learning and improvement. Fist, LLMs are generative models, not predictive models. Second, Flow efficiency is not a good measure of product development effectiveness according to Don Reinertsen “Usain Bolt & I can achieve same flow efficiency in 100m dash, but not same cycle time.” Unfortunately the original tweet has been deleted, but Nick Brown has what I think is a very good blog post on the subject that includes a reference to the original thread: https://medium.com/asos-techblog/the-many-flaws-in-flow-efficiency-bc7845ba4c9
Javier, thanks for your comment.
I never measure flow efficiency. It is a terrible measure. Instead, I advocate using value stream maps to actually measure the cycle time and the various wait times. I also find that when we understand the various wait states, we can choose what to do about them. But the first thing is to see those wait states.
As for what LLMs are: I will continue to disagree with you on that. The current LLMs predict the next bit of text. Or they scraped someone else’s ideas from all the writing they ingested. As far as I can tell, the LLMs do not create new ideas. I realize that is at odds with the current claims that these models create ideas.
I did a quick search and found this IBM article that claims the neural networks in current Gen AI models mean they create new ideas. Yes, they can create new images—based on ingested data. But are those new ideas? At this point, I say no.
We can agree to disagree on that part. At least, for now. I am ready to change my mind based on new information in the future. My experience with expert systems and neural networks is now almost 40 years old. The world has changed since then.
I came back because I realized that my comment about LLMs was not clear. LLMs are considered generative AI because that is their purpose, but both generative and predictive AIs are based on statistical predictions. However, LLMs add deliberate randomness to be able to generate interesting outputs, and are thus classified as generative. It’s not just a matter of semantics, however. There is a reasonable argument to be made that randomness is foundational to creativity, even in humans (https://www.quantamagazine.org/researchers-uncover-hidden-ingredients-behind-ai-creativity-20250630/?mc_cid=16b1140aed).
Many artists say that what they intended when creating art does not matter, what matters is how the person experiences their work. Perhaps creativity really is in the eye of the beholder.
Thank you for that link. That makes more sense to me for image creation now.
As for creativity? There is a big difference between how people experience their work, how they express their work, and how the “recipient” (for lack of a better word) experiences that work. I’ll use writing because I understand and practice that. I do not practice image or other kinds of creation.
I write fiction and nonfiction. I am much more experienced at writing nonfiction, since I’ve practiced a ton since 1997. (I’ve only practiced fiction, and not nearly enough, since 2018.)
My experience of writing useful nonfiction ranges from total delight to total frustration. When the words flow, I feel delight. When it feels like pulling teeth to get any of the words down, it’s total frustration. I can feel both when I write anything. However, my experience writing has almost no implication on how the readers might feel when they read it.
The reader’s experience is divorced from my experience(s).
Since I mine my experiences to write, I do feel creative. And is there ever randomness? Absolutely. Every so often, I hear something at just the right time. That thing offers me a new way to look at a problem and then write about it. (That happened when I heard about Clay Shirky’s book about small-world networks, and I realized that was precisely the frame I experienced but never had a word for. So I wrote about it in Agile and Lean Program Management and in my other books. I now realize that’s a form of working in informal flow efficiency. Hehehe.)
However, that is not deliberate randomness on my part.
Instead, I suspect that randomness is what we have called serendipity in the past for people. We all get exposed to so much every day—–sometimes, one of those exposures does introduce randomness, or more likely, a new way of thinking and therefore, expression, of our outputs.
Can a statistical model introduce randomness? Sure. However, I’m not sure all that randomness is random. We have at least one example where the LLM (Grok) takes the positions of its creator (or the owner of the company that created it).
The cynical part of me wonders if the smart people tried to introduce randomness in output as a way of saying, “We didn’t just steal all that stuff. Look, we generate new work because we introduced randomness.” (I have worked for managers who absolutely would have tried to forestall the copyright issues in that way.)
My real problem is this: I do not trust the people who include AI in all these products. I am still angry they stole all my intellectual property. Not only did they not pay me, they don’t even acknowledge I wrote that thing in that way. That’s why I won’t use image generators. No one can copyright an AI-generated image, and I want copyright on all my writing.
I do agree with you that the reader’s or the viewer’s experience matters. I’m not sure that means the reader or the viewer gets to say what’s creative. But the reader or the viewer can certainly have an opinion about what they read and see. We often call that opinion “taste.”
The first thing I though of when I saw the hype cycle image was the striking resemblance to the Dunning-Kruger effect.
Thank you so much for sharing this.
Thank you, Frank!