It's a good thing I said my post about musings was just that–musings! I didn't bring all of you along. Sorry about that. Let me be more clear.
A program is a collection of projects, where the value is in the overall deliverable. Yes, each project may have a deliverable that's valuable, but the value to the organization is when all the sub-projects get together and deliver their product. Here's an example. Think of a cell phone. One sub-project might be the team(s) who create the features that allow the phone to make a call. There may be another sub-project for the features to answer calls. Another sub-project could be the features to manage/interface with voicemail. Another sub-project is the minute counting for administration/billing purposes. The texting features would be another sub-project.
Each of these sub-projects have teams that are as large as they need to be. So, to answer Veretax and André's questions: yes, working in smaller teams so each team can be agile does work. Yes, each sub-project works in parallel. The teams for sub-projects work in parallel. When I think of these sub-projects for a program, each sub-project has its own rhythm and staff and backlog. That's why the program needs an overall backlog, because the sub-projects may have interdependencies and may uncover program risks as the program proceeds. In the example of a cell phone, you might not want to start the voicemail until after you know the phone can receive calls. Maybe you can, because maybe there's a platform that supplies the services your project needs. What happens if you realize you need to change the minute-counting after the call-placing features are done, but before the billing features are complete? You work it out.
But the idea is that you have a program: deliver a cell phone. Each of the sub-projects has its own list of features and each of the sub-projects delivers a valuable result. But, you don't have a cell phone unless you can make and receive calls and get voice mail. You may need more than that, but you don't have a product that can succeed in the market without those minimum requirements.
Now, for the why we need program management. I've seen (and I bet you have too) programs where the software was all done except for one small piece. Or the software was done but the marketing was not. Or the hardware was done, the marketing was done, and the software was going nowhere. If you have agile approaches to programs, you get to see visible progress (or lack thereof) at the end of each iteration. You don't have to wait until the predicted or desired end of the program to see the risk.
That's one of the ways agile reduces technical and schedule risk. The iterations help you get to done across the entire program each iteration and help you see how things fit together. The demo at the end of an iteration shows you where you have technical risk, which reduces schedule risk. In general, incremental approaches reduce schedule risk and iterative approaches reduce technical risk. Because agile combines both, you reduce both kinds of risk. (For more detail on lifecycles, see Manage It!)
Stephan also asked how agile program management differs from “plain” program management. In non-agile program management, managers of sub-projects speak for their functional area, commit people, manage risks, and commit other resources. Notice that there is no program-specific view of the product or transparent coordination across the functional teams. There is not necessarily a ranked product backlog. You could have those things in a “plain” program. I haven't seen them, but maybe that's how your program management works.
In agile program management, you have coordination across the cross-functional teams. That's a huge difference. You don't have functional managers, you need cross-functional project managers (of some stripe) for each sub-project, and one for the program. Managers don't commit; teams commit. The program team is all about managing the backlog and risks, the business value of the architecture, and the obstacles for the program.
In case you're not sure, I'm starting to write the agile program management book, so you nice folks are hearing about this as I try to articulate it. Thank you for your comments.
The biggest problem I’ve seen in large programs, and I’ve been involved in several in the last few years, is cross project dependencies. If you define your projects to minimize these, you push the risk, and the high likelihood of considerable rework to the integration phase of the program. If you don’t minimize the cross project dependencies, problems in one project can snowball across the entire program.
One question about terminology. If a program is made up of a collection of projects, why use the term sub-project? This would imply, although you don’t state it, that the projects that make up the program must be further decomposed so that projects are in turn made up of sub-projects. Maybe that was your intention.
I thought your comments about incremental and iterative approaches reducing risk were very insightful. However, risk remains in programs unless integration occurs after every iteration, and this is almost assuredly unworkable on any program that I’ve ever seen.
Another thought, on your definition of a program, below:
“A program is a collection of projects, where the value is in the overall deliverable. Yes, each project may have a deliverable that’s valuable, but the value to the organization is when all the sub-projects get together and deliver their product.”
That is certainly one definition, but there are others as well. A program may contain several related projects, whereby managing them collectively provides benefits to the organization. An example might be the building of several houses in a development. Each house is a single product, but by coordinating purchasing and construction across many products within the program, substantial efficiencies are realized. Another example is a program that manages a single product throughout the product lifecycle, such as software over multiple releases or a model of aircraft. There may be many projects in the program, but each project delivers a valuable product or enhances an already existing product.