Being always on the lookout for new ways of reaching the objectives fixed for projects, I was affected by this article: Informatisation à la justice: une dépense inutile de 75M$ (Computerization of Justice: unnecessary expenses of $75M). Not so much because it is a public project, but rather because I believe the market of software development still lacks expertise to ensure the success of its projects. Michelle Courchesne’s quote summarizes this symptom really well: “We cannot wait to have spent the authorized 105 million dollars to realize it is not working. Otherwise, after, we will have to give even more money. That’s over, we can’t do that anymore.”
What is missing?
Our colleague Martin Dupont asked in front of the Agile Quebec group: “How can we define that a project is a success?” Philippe Tremblay answered something interesting: “One of the factors that is clearly neglected is measuring the impacts of this project on our clientele. It does not matter if we respect the budget, the quality or the schedule; if the clients do not use what has been developed or if they use only part of it, what value have we created?”
Agile approaches offer frequent inspection points. During each iteration, it is possible to verify the quality of the deliverable good, the consumed budget and the time available to complete the project. In my experience, these indicators are being measured for most projects. However, they are insufficient to measure the true success of a project.
I remember a project that was aiming at recoding software being used to quickly calculate charge rates for clients of the Société québécoise des infrastructures (SQI). The system in place was not efficient because:
- It required consolidating data coming from different sources.
- It required ensuring data integrity before calculating anything.
- It did not allow any pricing increases other than by manual analysis of calculations.
So the client asked us to create a new system to help him calculate new pricing rapidly. We can very easily imagine a system answering the following needs:
- An automated process to consolidate multiple sources.
- An automated process to control data integrity.
- A feature to generate the details of clients’ pricing on demand.
With this solution, it is already possible to build a product backlog aligned on the improvement objectives. However, in my opinion, it is not enough to measure if the project is a TRUE success. It lacks the means to measure goal attainment for each iteration. I believe that measurable objectives would be even more useful. For example:
- Calculating semester pricing in less than a month.
- Importing data from ten sources in one click.
- Answering requests on pricing details per client on demand.
Why are measurable objectives useful?
Measurable objectives allow attribution of business value to product backlog items. In the case of the pricing program, it allows choosing solutions that save maximum processing time, either by consolidating data in a single operation or by identifying on a daily basis the data necessary to create billing detail quickly and ad hoc.
With a business value amount for each product backlog item, it is possible to compare gains to cost, and therefore to calculate return on investment and acquired value. In an ideal world, the sum of delivered items’ values should be equal to the sum of the gains expected from the product’s delivery.
If, in addition, an Agile approach is being used, it is possible to measure at the end of each iteration if the last software increment met the objectives targeted by the project.
Furthermore, since each increment is a deliverable good, it is possible to verify that measure with the real users of the system.
How to improve?
Here is what I suggest to ensure a better control and to guarantee the success of an Agile project:
- Realize that the product backlog estimation is not sufficient to measure the success. The estimates (like story points) are indicators of the cost, not the business value.
- Define measurable objectives at the beginning of the project.
- Measure achievement of the objectives for each iteration. A good way would be to include them in the definition of done or in the acceptance tests.
- With a functional deliverable good in hand, measure achievement of the objectives in a context closer to production, with real users of the system.