Agile Software Development

Key Practices of Agile Development

“I am breaking my two-year projects into two major phases: requirements and coding. Does that make my project agile?” The answer to this question is clearly no. Beyond that simple answer, we need to deliver an explanation of what will give a project an agile spin. The various agile processes have some successful patterns in common, which are isolated as key practices. These key practices are so essential to any agile project that they will affect the energy, spirit, and eventually the success of the agile project. These practices are, in no particular order:

• Iterative-incremental development.

• Test-driven development.

• Continuous integration.

• Face-to-face communication.

Let’s explore each of the key practices in more detail in the following subsections.

Iterative-Incremental Development

This practice is, from a management perspective, the most noticeable change toward an agile approach. Instead of executing every software development cycle (requirements, design, programming, and so on) only once in the entire project (which is the traditional or waterfall way of doing it), iterative development enforces repetition and semi-parallelism of the development activities. That means the activities are extremely narrow and close to one another. They seem to blend, and the order of activities can change. That might sound strange, but you will see later in this chapter why this is a very good idea.

As a rule of thumb, the shorter the iterations, the better. That’s why agile processes require a time frame for each iteration that is typically between 2 and 6 weeks.

The second aspect of this practice is the actual increment. Whereas the iteration provides the rhythm for the project, the increment demonstrates the actual progress of the project, as illustrated in Figure 2-2. Viewed in that way, it looks as if the project progresses by stepping up a staircase.

Figure 2-2. Incremental development and progress

In agile development, progress is measured in working software. For example, a project that focuses on a throwaway prototype during the first iteration and defining requirements in the next iteration has not demonstrated progress, based on the definition of agile. Similar to a pizza, a project must be cut in slices. The difference is, however, that we won’t know in the beginning how many slices we will need and what kind of pizza it will be. We will figure it out as the project continues, slice by slice.

To state it another way, a project will have a few very high-level requirements, and the project team will take one of these requirements (for example, the highest-priority item first) and drill down into more detail. Then the team approaches the more detailed requirements, writes unit tests (as described in the “Test-Driven Development” section), designs and programs the code, and executes test cases periodically. At the end of the iteration, one high-level requirement, a slice of the project, is completed. Now, this is where iterative-incremental development really shines. One requirement is converted into working software and can be demonstrated to the customer just two weeks after the project was kicked off. Think about where your waterfall project would be two weeks into the project. Even better, by taking this approach, we opened an early feedback loop to the customer, who can now make changes and help navigate the project in the right direction (as illustrated in Figure 2-3) according to their expectations.

Figure 2-3. Feedback and direction for an iterative-incremental project

In my experience, I’ve seen many projects implement an iterative-incremental approach, but the majority of the customers have not taken advantage of this feedback loop. We have told so many users (wrongly) in the past that software requirements have to be complete and clearly defined up front that we will have to bring customers back to the natural way of building applications—that is, back to an agile way. The feedback loop has huge potential for many projects and shows that transitioning to agile development is a process that includes the customer as well.

Within each iteration, stakeholders will shape the direction of the increment for the scope of the iteration. In between iterations, more significant (and less disruptive) adjustments to the scope of the project are possible and often welcome. Stakeholders who are not involved in the progress of the project on a day-to-day basis are required to take part between iterations. That way, the development team also gets important feedback from peripheral stakeholders and can meet their expectations as well. According to authors Gause and Weinberg (mentioned in Chapter 1), this is also called the “Yes, but...” syndrome. Whatever follows the “but...” are changes in the direction of refining the project. The shorter the cycles between the input and the feedback from stakeholders the better. That is one of the motivations to keep the iterations very short.

We will make heavy use of this feedback loop when agile projects are part of the portfolio.

Besides being able to better manage expectations, the customers and the project team also benefit from iterative-incremental development in many different ways. Nothing is more rewarding than hearing, iteration by iteration, that the team did a good job. Even if they did not, negative feedback allows the team to rectify things in the next iteration or even within the current one. In contrast, in the traditional approach, the verdict comes in so late that the project team cannot take corrective actions. Iterative-incremental development also overcomes requirements paralysis and the issue of ending up with too few requirements. It might sound surprising, but agile, in some respects, introduces more discipline into the process than other project methodologies—such as the discipline to deliver working software in two-week increments. What customer does not like that?

Other advantages of iterative-incremental development include the following:

• The highest risks can be reduced or eliminated in early iterations.

• Confidence in planning and estimation increases iteration by iteration.

• Based on past iterations, trends for a completion date can be determined.

• Completed means completed, not 90 percent done.

• Morale is increased through constant feedback.

Iterative-incremental development is a tremendous change to the traditional development processes currently in place in most organizations. However, this degree of change is worth undertaking, as it immediately provides invaluable benefits.

Test-Driven Development

I know this might make a lot of executives nervous, but many organizations have a lot of untested software code in production. To be fair, though, not even the software engineers are aware of that. One of the reasons is that developers write their code, debug it, make corrections to it, and so forth. What emerges is a network of classes, objects, and methods with a variety of conditions and loops. Nested within these clauses are statements that might never be executed because the conditions that guard (and protect) these statements from execution will never be fulfilled. But who says the condition will go unfulfilled forever?

Agile development promotes a practice of test-driven development (TDD). The idea behind this practice is that the developer writes the unit test prior to the actual code. If you don’t know what to test, how do you know what to code? The result is a unit test that tests the actual object but does not interfere with the object itself. The test object contains messages for all the various guards and conditions and makes sure the object acts as planned.

In agile projects, these unit tests are commonly automated, including the code coverage. That way, the project team can monitor the number of classes and the number of unit tests. If the unit tests are falling behind the number of classes, somebody on the team is not practicing test-driven development and unverified code might have entered the code base.

Test-driven development will have a significant positive impact on the quality of a project. The test cases evolve alongside the iterations; therefore, the code base that exists after each iteration is tested. Remember the waterfall approach, where the writing and execution of test cases are commonly performed at the end of the project? And that is exactly when budgets get tight and resources begin transitioning out to other projects.

The idea behind TDD is to capture the unit test code in its own component, separated from the component that implements the functionality. The activity of writing the test code and the functionality are almost parallel, with the test code slightly ahead of the game. Often the unit test as well as the functionality is developed by the same developer (pair). Then both components are compiled and the unit test executes behavior on the component. The results either pass or fail and are recorded. Through the separation of the test and the functionality, a build and, eventually, the release can be easily assembled by simply leaving the test object behind. That also means that the component does not need to be recompiled, which assures that the component with the last time-stamp is the component that passed the test.

With the test cases stored parallel to the actual code using an agile approach, it takes only a small step to automate the execution of these unit tests. This is exactly what agile developers do by using tools to execute the tests when certain triggers occur.

But even if teams are more casual about the timing of writing unit tests, tremendous organizational value can still be achieved. Once the behavior of the team has changed to make writing unit tests a common practice, the team can make a relatively easy transition to the final step—writing them prior to the actual code.

Without any unit tests, regression testing would be extremely challenging and tedious, if not impossible. The automation of the unit tests, and therefore regression testing, must be the basis for test-driven development, which will bring us to the next topic—continuous integration.

Continuous Integration

The integration of components and the testing that goes along with it is nothing new for software engineers. Where agile development substantially differs is in its continuous approach. One of the major issues with the traditional approach is that the testing of the integration is deferred to late phases in the project, and the architecture is expected to just fall into place. In reality, projects can blow up right at the worst possible moment, when little project budget and time are left. Patching problems with workarounds and mediocre approaches might help get the system out the door, but new issues are just waiting to pop up again. There has to be a better way.

One approach with iterative-incremental development is to integrate items at the end of each iteration, but that means that you have this peak and stress at the end of each iteration. So why not, instead, implement ongoing integration? That is exactly what agile teams do. Figure 2-4 shows steps in a continuous integration approach. All of these steps are commonly automated.

Figure 2-4. Continuous integration

Even the trigger for performing the integration can be linked to regular software engineering activities—for example, checking code in through a configuration management system (subversion or CVS [Concurrent Versioning System]). With a continuous integration infrastructure in place, the team, management, and executives in the organization know what the most recent successful version of the system is. When a developer on a waterfall project reports 100 percent done for a certain requirement, that means something totally different than when an agile developer reports something as being 100 percent done. In agile projects, 100 percent means developed, tested, and integrated.

Continuous integration is relatively easy to achieve from a technical perspective. Many open-source tools have been developed to support the agile development team with its continuous integration efforts. Once the environment is in place, source and version control, execution of tests, creation of software builds, and the continuous deployment of builds are delegated to reliable tools.

Face-to-Face Communication

When I began working with IT projects, there was no such thing as electronic communication. Yes, there was e-mail, which was used to distribute information to the masses, but project team members were usually located in close proximity to one another, if not in the same room. Although we did not have the alternatives we have today, there is something about verbal and nonverbal communication that electronic communication cannot replace. Think about foreign accents or body language, for example. Agile development factors in this loss in communication and forces team members to collaborate directly with one another, person to person, without electronic firewalls. Especially when business analysts and software developers work together, face-to-face communication is essential. Anonymously sharing requirements documents just opens the door to misinterpretation and misunderstandings—not to mention that written information travels so much slower than verbal communication. Just to be clear, I’m not saying that no requirements are documented—they are co-developed, clarified, or negotiated between team members. Last but not least, we are all humans, and a project is a social network. We need these networks to increase morale and fun. I have never come across a project team that worked in an exclusively distributed way and actually had fun.