I started this series discussing where AI use might create bottlenecks. The more people work solo on anything, the more likely they become bottlenecks for the team. When each person works solo with AI, they reinforce resource efficiency, where everyone is busy, but the team can't seem to finish anything when anyone wants it. Instead, we need flow efficiency to finish work.
I then suggested a team with a lot of WIP consider a value stream map for each person, mostly to show each person how their multitasking creates the team's bottlenecks. However, that value stream map does not discuss the team's issues. It only shows the individual's issues.
That's why almost all teams need a kanban board with as many columns as necessary. Teams need to see where their work is stuck. In addition, any team that's working in resource efficiency needs to somehow find common ground so they can work as a collaborative team. A standup does not do that. A retrospective might.
Now, why did I not suggest AI for diagnosing these agility problems? Because the LLMs do not know your team, unless you somehow use your team's data.
How to Create Your Team's Data for AI Diagnosis
LLMs excel at uncovering patterns in our data. They only offer generic information about how a team's agility problems. (Although, I know they scraped Create Your Successful Agile Project, so they should know more than they do. Sigh.)
If your team already uses an electronic tool, you can ask the LLM to use your data to turn any three-column (Ready, In Progress, Done) board into a kanban board. (That board might have to be outside the tool because all tracking tools vary in how they convert a three-column board to a multi-column board. Here are three minimum questions you can consider for an LLM to ask the tool:
- For the last two weeks, how many separate in-progress and wait states do you see in our work? This is the minimum number of columns on your kanban board. You might need more columns because the LLM does not know everything. Or, you might need more time, depending on how long your cycle time is. If your team hasn't finished anything in two weeks, extend that time period to at least four weeks and try again.
- Create a time-series graph of the team's cycle time for all finished work. (Read How to Move from Story Points and Magical Thinking to Cycle Time for Decisions for more information.)
- Create a table that compares the number of unfinished items and their current cycle time (where they are so far) with the finished cycle time data. You might have to ask this in different ways, but I would expect a long table of unfinished items with very high current cycle time and a comparison against the small number of finished cycle times.
Now you have some data that works for when people work primarily as individuals.
While you can start here, make sure your LLM retains the context. That will allow you to continue to add and refine your questions. You might need other information from the four flow metrics. I would start with aging, because that is the leading indicator of high WIP and low throughput.
But what about when you have a collaborative team, and the team has interrupting work? You can use the LLMs, and you have to move up and out several levels to find the WIP.
How to Use AI When the Bottlenecks Are Outside the Team
That's the problem with the collaborative team in Part 1 of this series. While they collaborate on their work, they get pulled out of their work to other meetings.
In my experience, managers cause many of these meetings. The managers have decisions in progress, and either want the team to explain the team's data, or they need the team to work with the managers. That's Management WIP.
If you have access to all the management decisions in progress, you can use an LLM to ask questions about which decisions are stuck in Management WIP. (See How Agile Managers Use Uncertainty to Create Better Decisions Faster for a little information on that. I clearly need to write more! Also, see Why Minimize Management Decision Time.)
Your LLM might be able to interrogate this kind of data:
- Portfolio WIP: Which portfolio decisions are still in progress? What is the time duration for this version of the project portfolio? The longer the time duration, the longer it takes managers to decide. Have they limited the number of projects to—at most—the number of cross-functional teams?
- Non-team WIP: How often does the team get dragged into which kinds of meetings? How many of those meetings have completed outcomes? Is the team trying to workshop future stories and finish this round of work? Is that reasonable? Are there production support issues the team needs to address? What is the cycle time for all of that work?
- Other Management Decision WIP: What stage-gate decisions do managers feel they need to make? How does that impact the team's cycle time? I keep seeing more “agile” organizations trying to write PRDs. That requires a PRD review, which is a stage-gate approach to a project.
It's too easy to outsource judgment to an LLM, which is exactly the wrong thing to do. That creates a stage-gate approach to product development.
LLM Outputs, Especially Documents, Need Validation
When a product leader “outsources” their thinking to an LLM, the LLM uses all the product data in the organization to create that PRD. Yes, everything from 10 years ago to 2 weeks ago. Then, the LLM can spit out a somewhat coherent PRD in five minutes.
How good is that document? Does that document have the judgment a human would have? Can you trust the data in the document?
The only way to know is to validate the PRD. What is that validation? A stage-gate. Yes. When product leaders relinquish their judgment to an LLM and create a big, honking PRD, multiple someones have to review that document. That's a perfect example of a serial lifecycle.
Serial lifecycles increase the aging of everything (bad). They also increase WIP (bad), cycle time (bad), and reduce throughput (bad). So a serial approach is anti-agility.
Why would you do that to yourself, the team, or the organization? (This is why I love product value teams and dislike solo product leaders.)
This PRD document is a product leadership example of using an LLM in resource-efficiency mode. To use an LLM in flow efficiency mode, you can ask the “How little” question as a team to start the planning.
No Need for an LLM to Diagnose Common Agility Problems
If you want to diagnose your agility problems, start with the flow metrics. What does that data tell you? Now, move backward to what causes those problems in your organization. I recommend a retrospective to find the root causes. I can almost guarantee you will not find just one root cause. Instead, it will be a constellation of causes. (See the systems thinking tag to see everything I've written about systems thinking on this site.)
Here's the summary for these three posts:
- If you want to use AI to help you develop products faster, always use an LLM as part of a team, not an individual. Maybe even especially if you are a product leader.
- Use an LLM to integrate your data into its dataset. Then, use an LLM to visualize that data in ways that make sense to you.
- Make sure you can trust what the LLM tells you. You will have to use your best judgment to verify that data.
- Be human wherever possible when it comes to the “why” questions, such as you would use in a retrospective.
The LLMs offer real promise, especially for insights into our (relatively) clean data. But don't think they can cure all of your problems. Since humans created these problems, we can solve them. With whatever tools can work for us.