This is Johanna Rothman's February 2026 Pragmatic Manager newsletter. The unsubscribe link is at the bottom of this email.
Jim, one of my trusted advisor clients, was all gung-ho on AI last year. He told me that his AI work allowed their senior management team to define their strategy, “once and for all.” That strategy was supposedly going to make the work better for everyone.
I was suspicious of all of that. In my experience, we need an iterative approach to strategy. And, because the portfolio implements the strategy, there is never a “once and for all.”
Was it possible for AI to make the work better for everyone? Jim's organization has decades worth of trustworthy customer and sales data. They can use that relatively clean data with an LLM to gain insights into more customer segments and markets. LLMs can be good at accessing information because they can see patterns us humans might miss.
However, Jim's product development data is not trustworthy. For decades, they ignored the overtime people spent on projects, so the data is not useful for prediction. Since they started measuring cycle time six months ago, they might gain some insights from their time-series data. If they fully fund projects, they can probably use their most recent data to do a little prediction.
At this point, I do not see how they have enough full-project data to use an LLM to make the work better.
But the real issue is this: What questions do you ask of people, the LLM, or even advisors like me? Our questions reflect how we think. We can ask better questions when we use critical thinking skills.
Ask Better Questions with Critical Thinking Skills
Managers deliver decisions to the organization. That's why late or insufficient management decisions reflect a form of management debt. See Why Minimize Management Decision Time for a value stream map of how this works.
Most of the time, managers need to ask questions to understand what's going on. That was the case in Jim's organization.
Two years ago, the flagship product began experiencing slowly declining sales. No one was worried for the first two quarters. But by the third quarter, it was clear that the flagship product was no longer competitive.
The delay in recognizing the change is a failure of critical thinking.
Now, Jim's organization no longer had “plenty” of time to consider new products and an updated strategy. But they still spent another quarter debating their new strategy instead of trying some experiments.
Here are their possible experiments:
- Start a new product line for the same customer segments.
- Listen to focus groups to see what the current product was missing.
- Consider which new customer segments they want to attract, choose a new candidate strategy, and start a new product in that strategy.
They could not decide. After a week-long strategy offsite, they decided to continue everything as is. Worse, “continue everything” is another critical thinking failure. And was dependent on not realizing that things had changed. They needed to choose something different. But they were unable to think about what that was. That's another failure of critical thinking.
Jim's organization is in a hole. Worse, they continue to dig that hole.
Critical Thinking Can Help Stop Digging the Same Hole
Jerry Weinberg used to say, “If you're in a hole, stop digging.” Then there's this quote, generally attributed to Einstein: “We cannot solve our problems with the same thinking we used when we created them.”
Jim's organization had these three failures because they did not use critical thinking:
- They failed to recognize that their world had changed. Once they did, they took too long to respond.
- That response was ineffective. “Continue everything” is not a sufficient response to “our world has changed.”
- They had to choose something. Earlier, they had more options. But time has removed too many of those options. Now, they need to consider what will generate revenue fast or open up new customer segments.
Can an LLM help the management team think? That depends on the questions the management team asks. Right now, LLMs do not offer new insights. They can access information we might miss. That means the LLMs cannot yet make the work better. Nor can they offer anything except the same thinking.
That's why I recommend Deming's Plan-Do-Study-Act loop any time we need to run experiments and iterate on problem-solving. That's the image at the top of this post. (You might prefer the Toyota Kata. That's also an iterative approach to exploration and problem-solving.)
Critical Thinking Requires Iterative Problem-Solving
By definition, managers deal with ambiguity. They never have enough information. Nor do they have the ability to predict the future. Instead, they must make the best decision they can at the time. Then, managers need to be able to make another decision once they realize things have changed.
That's why I like to consider management decisions as experiments. The shorter the experiment, the more frequent the decisions. Yes, managers need the fastest feedback loops they can create.
Some experiments, such as the portfolio, might have a useful life of several months. And, when the world changes, it's even better to re-evaluate the portfolio every few weeks. That allows the management team to rethink their experiments more often. Then, they can use their critical thinking skills to evaluate the experiments.
How to Use Critical Thinking Skills
Here are some questions to consider so you can use your critical thinking skills:
- How fast can you recognize when changes mean it's time for your business to change? (I've framed this as “business” but it works just as well for every level of the organization: backlog, roadmap, and portfolio.)
- What will you stop doing to make room for an experiment?
- How small can you make your experiments so you can learn fast about where to invest more?
Managers make decisions all the time. The trick is knowing which decisions need to be experiments with fast feedback loops, and which decisions will last for longer. Strategy and portfolio decisions cannot be long-lived, not in this environment. They must be experiments with short feedback loops.
While I remain an LLM skeptic, it's possible that you, as Jim, can use an LLM to learn more about your existing business. That requires good questions and relatively clean data. That's why critical thinking about what's working and what's not working might make the work better.
Read More…
This newsletter touches on topics in these books:
- The Modern Management Made Easy books, especially Practical Ways to Lead an Innovative Organization.
- Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects
- Project Lifecycles: How to Reduce Risks, Release Successful Products, and Increase Agility
- Successful Independent Consulting: Relationships That Focus on Mutual Benefit
This is a part of the intermittent series of leadership tips.
Announcements
If you are part of the agile community, consider checking out The Enterprise Agility Network. Also, don’t miss out on discounted membership options. Use Discount Code: ROTHMANPMC33 to get 33% OFF all memberships.
Links of Interest
New to the newsletter? See previous issues. (I post these newsletters to my YouTube Channel a few days after I send them.)
Here are other links you might find useful:
- Managing Product Development Blog. (Yes, I offer an RSS feed so you can read everything in a newsreader.)
- Create an Adaptable Life Blog to see the weekly question of the week.
- My Books. You can also buy my self-published ebooks and audiobooks on my store.
- Johanna’s Fiction
- Tip Jar. (My newsletters will remain free. And I do want to offer you an option for you to show how much you value my work.)
Johanna
© 2026 Johanna Rothman
Pragmatic Manager: Vol 23, #2. ISSN: 2164-1196