Let’s take a moment to think about the process of building a house. The first step consists of the owner communicating his needs. Based on these guidelines, plans can then be drawn to define how the construction will be conducted to obtain the desired result within the timeframe.
Traditional software development follows the same procedure. Before Agility, analysts and architects had the responsibility to define how the product was going to be built from A to Z. Today, we can still frequently see projects invest 10% to 20% of production time in gathering needs and analysis, and then spend another 10% to 20% of the project’s total duration for the system’s design and architecture. So we speak of as much as 40% of the allotted time possibly invested without a single line of code being written.
This is a very costly step if compared to its equivalent when building a house. Nowadays, you can even find free plans on the Internet!
The implementation step that follows the development is also much riskier than in the case of a house. Indeed, it is rare to find the plan of a building for which we ignore if the walls will stay upright or bear the roof’s weight. While in software development, we frequently need to change direction because we learn that a component is not compatible with the needs.
Have I said change direction? So why not build the product in an Agile way? Good idea! OK, let’s take a closer look at the Agile Manifesto to soak up its philosophy.
The Agile Manifesto
Three of the manifesto’s four values influence the vision we can have of a project’s architecture and analysis:
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
These elements favour a tangible and testable solution rather than a plan detailing the global result we need to achieve in the end. Why? Because no matter how much effort we invest in the planning, everything can change from one day to another.
So we need to be well equipped for this change. Otherwise, we end up using Agile practices without actually benefiting from them. To succeed with a transition and to have a good opinion from all the levels of a business, it is important to maximize the benefits. Then, by providing a simple comparison between “before” and “after” the transition, everyone can easily see why this decision was made and nobody wants to go back.
If we want an operational version of the software from the first sprint, it is logical to make sure that the work of all team members is done at the same pace as the development. In the case of analysts and architects, it raises a specific challenge, because their contribution to the project is traditionally necessary even before the programmers write a single line of code. We now tell them there is no possible preparation, they need to master the matter from the beginning.
Patrick and SpongeBob SquarePants
How can we do that? Does it mean the architects need to work round the clock during the first days of the iteration to make sure the team has what it needs to accomplish its work in the following days?
If we follow Agile principles, architects are part of a team responsible for delivering results at the end of the iteration. So, why divide the teams in silos when we can all collaborate to get to the same result?
Opposite to having a complete design before the programming phase, emergent design encourages the implementation of the software infrastructure pieces at the same time as the construction.
It is obviously unrealistic to think that we can start developing without making a few choices beforehand. For example, choosing the language, certain libraries and the basic design layers, is required to avoid team members engaging on a path that we will not be able to consolidate afterwards.
That is why it is a priority to find only what is “just enough” to launch the development work and to focus on these requirements first. In collaboration with the Product Owner, we can use different tools like: a project chart, a user story mapping, a sequential or layers diagram, to define the business criteria and technical considerations that will impact how the software will be built.
If you have never done any analysis or emergent design, be aware that with any transition, your habits can be changed. So to succeed, you will have to be disciplined and follow principles that may appear simplistic at first glance, but that will make a difference in the long run.
Emergent Design Principles
Think big, take small steps. Fail quickly and learn immediately…
We see it in group contests, like hackathons for example. A team has no other choice than to aim for exceptional results to be able to compete with others. On the other hand, to succeed, everyone must do it to find the small steps that will realistically mark the way toward this common goal that will allow them to reach first place.
The same vision should be adopted for a project. Partners will want to have software that makes a big difference, but the team members’ confidence should not be destabilized by a bar that might seem too high. We need to keep a positive attitude in front of each challenge because the worst thing that can happen is to have only one failure: the final delivery. That is why it is necessary to divide into smaller steps. This way, failures will also be small and the learning they bring will considerably improve the team’s evolution and everyone will realize it with the final result.
The last responsible moment
The sooner a decision is made, the riskier it is because it can change at any time and cause a string of consequences on other features, other modules or even other teams. If a review of these changes is conducted, we find that the cost of this choice was enormous, where if it had been made later, when the team really needed it, the decision would have been made with better vision.
The concept that represents this reality is called the Cone of Uncertainty. To illustrate it, we can think of mapping the trajectory of a cyclone or hurricane.
Irene, tropical cyclone
The further we look in time to predict the trajectory, the larger the cone gets. We can explain it by the difficulty to know if there will be a change in temperature or air front that will affect the path taken by the storm. The same concept applies to software development: the more we try to rely on what will happen in the future to make our decisions, the more these may change.
It is difficult to find a precise moment when we need to make a decision. That is why the team, including the Product Owner, needs to use its judgment and discuss it openly and transparently. We need to find the right balance between the present and what could happen, while being careful not to look too far.
Delivering the simplest possible operational increment
The goal is to have the smallest possible increment, which we can experiment with and on which we can develop iteratively to meet all the requirements.
So the needs should be minimized in order to validate the hypotheses without embarking on days of unnecessary efforts. The Product Owner will bring big features, programmers will do more than planned, but everyone needs to be vigilant not to lose sight of the small steps that should be taken one at the time.
The Lean Startup spirit follows this practice by bringing the concept of MVP (Minimum Viable Product). The goal stays to always keep in mind the business objective in priority and to make sure that the user stays satisfied with what he has in hand, even if it is not complete.
This approach enables us to invest less effort in a solution that may be different from what was required initially. Furthermore, we can make a demo of the product very early, maximizing its visibility and enabling us to reach more clientele.
Once the first version is delivered, others coming after give the opportunity to advance the product iteratively (making it better) and incrementally (adding features).
Inevitably, time will come for the team to face problems preventing it from making architectural or design decisions. These are often caused by uncertainty, or a risk that is too high in regard to little-known technology.
To better understand how to deal with these problems, I suggest you read my article: Spikes and Emergent Design.