Scenario-Focused Engineering: Take an Experimental Approach

The Fast Feedback Cycle

The Fast Feedback Cycle is the heart of Scenario-Focused Engineering, and there’s a clear science and rigor for how to do it well.

As the mouse example showed, you want to take a more experimental approach to building solutions. The key term here is experimental. This means that your job as an engineer is less about deciding what the product will do and more about discovering what is actually going to work in real time, in real usage, with real people, and with real technology constraints.

It’s important to see how the different parts of the Fast Feedback Cycle fit together to achieve this goal. Together, the parts of this cycle form a general approach to problem solving; they aren’t a specific prescribed tool set. In fact, you could (and should) apply lots of different tools at each stage in the cycle, depending on your situation and how far along you are in your project. We’ll talk about the most common ones used for software projects throughout the book, and we’ll also mention alternatives for more unusual situations or different domains. However, while you need to pick the most appropriate set of tools for your individual situation, the underlying rhythm and general approach shouldn’t change. The science (and the power) of the Fast Feedback Cycle is in this rhythm, illustrated in Figure 3-1.

Let’s walk through the Fast Feedback Cycle at a high level so that you can see the overall pattern. The chapters in Part II, “The Fast Feedback Cycle,” go into each stage in detail.

Target customer

First, before you can start iterating, you need to know who your target customer is. That is, who do you intend to build this solution for? The choice of target customer is primarily a business-strategy decision—which customers are the most lucrative or most leveraged or most likely to drive long-term success?

Defining a target customer is essential because it acts as a lens through which all the other stages are focused. The target customer is the fuel that powers the cycle, and knowing your customer is the most important prerequisite before starting the iterative process.

Chapter 4, “Identifying your target customer,” discusses this topic in more depth and describes why this focus is so essential.

Observe

After your target customers are identified, you spend time researching them—looking for their unarticulated (or latent) needs: that is, what customers can’t quite tell you themselves but which are the deep desires and underlying needs that drive their behaviors. Identifying an unarticulated need that your competition has not yet noticed, and then building an excellent solution for it, is a great way to achieve differentiation in a crowded market.

Site visits and other observational research approaches help ground you in customers’ real-world situations—not in abstractions of what they should need, but in the reality of what they run into in their actual usage. By watching customers in their native habitats (on their couch, in their office, or walking in a crowded shopping mall), you learn things you wouldn’t have thought to ask about. For example, “Why are you doing that eight-step workaround when this beautiful feature over here would do it for you? . . . Oh, you knew it was there, but it doesn’t work with your company’s procurement system? Hmm, you’re right; oops, we didn’t plan for that situation.”

Your goal in observing customers is to gather data that will help you ferret out root causes—not the symptoms or the Band-Aids or the referred pain, but the original source of the problem or the deep human need that is driving the customers’ behavior. You may uncover a critical detail—a product interaction you never noticed before. Or you may discover a surprising insight into what really matters for a knowledge worker. Perhaps for that worker deep delight comes not from getting the job done but from unobtrusively staying in touch with his family throughout the workday, which counterintuitively improves productivity because he isn’t worried about whether his kids got to school safely.

Collecting and analyzing more numerical, or quantitative, data adds another piece to the puzzle. Whether you are looking at statistics about your competitors or crunching massive amounts of usage data from your existing systems, quantitative data can help alert you to usage patterns, anomalies, or pain points that may help you find a new, unique opportunity to delight your customer.

After doing even a moderate amount of research, it’s easy to collect a veritable mountain of data points, customer requests, ideas, and notes. These may come in a wide variety of formats: quotes, survey data, photos, video footage, competitive information, usage statistics, and so on. Unfortunately, the breakthrough opportunities you are looking for may not always be immediately obvious. As you get ready to exit this stage, it is vital to look across your research to identify patterns and themes that point to the most pressing customer needs and opportunities. Affinity diagramming is a straightforward and extremely effective technique for making sense of a large amount of unstructured customer data of this sort and will help you identify patterns and insights about your customers that might elude you otherwise.

Chapter 5, “Observing customers: Building empathy,” discusses how to use a mix of subjective, objective, qualitative, and quantitative methods to uncover unarticulated customer needs and develop empathy for your customer. It also shows how to use affinity diagrams and other analysis techniques to distill a large set of data points into a small set of key insights.

Frame

While observing customers, you will undoubtedly discover lots and lots of needs and potential opportunities. The next step is to make a judgment about which of those needs are the most important to address, and to precisely articulate what those needs are. This is called framing the problem.

Think of it like holding up an empty picture frame in front of an imaginary tapestry that depicts every aspect of your customers’ needs, usage patterns, and desires. Depending on where you hold that picture frame, you determine which aspect of the customers’ experience you are going to focus on. Will you hold it close and focus in on a specific, narrow situation? Or will you hold it farther away and zoom out to include a complete cradle-to-grave experience? Will you focus on this problem or on that one? On this situation or that other one?

Once you decide on a frame, this allows you to focus on how to fill out the inside of that frame—and not become distracted or continually debate why you are working on this particular need. In addition, framing encourages a bit of rigor to ensure that you articulate exactly which problem or situation you are trying to solve before launching into building solutions, which is vital for alignment when you have multiple people (or multiple teams) who need to contribute to a project.

One very helpful and broadly applicable framing technique is to write stories such as scenarios, epics, or user journeys that tell the story of what a customer is trying to accomplish that your product could help with. We have found the scenario technique to be a particularly effective tool for describing the kind of end-to-end experiences that we aspire to build with software. A scenario introduces the target customer, describes a real situation in which the customer has a need, and articulates what qualities a good solution would have—without saying anything about exactly how that solution would work.

Additionally, it is critical at this stage to establish a few metrics to track as key criteria for success and to embed these metrics in each scenario. Putting thought into specific, measurable goals for each scenario helps zero in on exactly what kind of experience you are aiming for. For instance, when looking for a photo of my daughter in my vast photo archive, is it more important to achieve a sense of magic (that the software somehow knows how to distinguish my daughter’s face from my son’s), or simply to get the job done quickly (where a simple text search for her name might be close enough—and much easier to build)? What aspects of a solution are going to drive customer delight in this situation? How might you measure them? Later in the project, these metrics can be listed on a scorecard to provide a view of how well you are achieving your goals across all of your scenarios and to help you know when you are “done” and ready to ship.

Scenarios are a popular and effective method for framing software projects, and they have some strong benefits. However, it’s important to note that they are not the only possible approach, or even always the best method for every team. Other options include goals and non-goals, requirements, user journeys, outcomes, Agile user stories, and epics, some of which share characteristics with scenarios. Each tool has its pros and cons, but the essential job at this stage is to clearly frame the problem and define what success looks like before you jump into building anything. For the purposes of this book, we often refer to scenarios as the most typical framing tool, but you can easily substitute a different approach if that is appropriate for your situation.

Chapter 6, “Framing the problem,” discusses framing and measuring against that frame in more detail.

Brainstorm

Now that you have the problem clearly framed, it’s time to start considering possible solutions. You want to take the time to explore—to brainstorm and play around with lots of ideas before you make any decisions about which of those ideas you will pursue. At this point, you aren’t concerned with finding the single perfect idea; rather, you’re generating lots of ideas to give yourself plenty of choices that you can mix and match throughout the iterative process. There are many techniques to help you generate creative, innovative ideas—and it truly is possible to get better with practice.

When you’re brainstorming, it is important to think through “complete” solution ideas—that is, ones that solve the entire end-to-end scenario and are not just snapshots of one piece of functionality. It is also very helpful to visualize your ideas, whether in a sketch, a flow chart, or a comic strip. Storyboarding is a very popular technique at this stage because it allows you to visualize a sequence of events that map out a potential end-to-end experience, yet it is lightweight enough that you can sketch dozens of alternatives in an hour or two.

We’ll discuss storyboarding as well as other brainstorming and visualization techniques in Chapter 7.

Build

After you explore the range of possible end-to-end solutions, it’s time to flesh out a few of those ideas in more detail—not all of them, but the ones that seem to have the most promise from a business, technical, and customer-experience perspective. It’s essential to find very cheap, fast, lightweight ways to create prototypes for your first few iterations through the Fast Feedback Cycle—to be as efficient as possible and minimize wasted time on dead ends. You can often do this without writing any code, even when you’re working on deeply technical platform components or APIs. The goal of these early prototypes is to get customer feedback on some specific approaches as quickly as possible so that you can make quick course corrections with a minimum of sunk cost.

Your prototypes can be paper drawings, wireframe mockups in PowerPoint, SketchFlow drawings in Expression Blend, an API interface written out on paper, flow charts or block diagrams in Visio, or a skit designed to work out a process workflow or customer-service scenario. The format of the prototype is much less important than how well it facilitates quick feedback from customers about whether you are on the right track. It’s important to choose the prototyping technique that works best for you, depending on the type of solution you are working on and the feedback you are looking for. When you finish prototyping, write code in slices to enable continual customer feedback.

We’ll discuss prototypes and coding techniques for many different kinds of situations in Chapter 8, “Building prototypes and coding.”

Repeat: Observe customers again to get feedback

Now that you have a few prototypes, you start the cycle over again by observing customers using your prototypes. After all, the reason you built those prototypes is to get feedback from customers as quickly as possible so that you can learn and make course corrections while it is still cheap and easy to do so.

Ideally, your first prototypes are little more than simple sketches on paper. You want to learn as early as possible if you are on the right track or where your ideas are falling short. In the extreme, your first prototype for customer feedback could be the scenario you wrote earlier, to be sure that it rings true before you spend any time on solutions for an irrelevant problem.

When you show rough, early prototypes to customers, you learn an amazing number of things. You may find out that your approaches resonate with the customer and that you’re on the right track, so you can accelerate your team’s work with confidence. You may identify a preference among the alternatives you’re considering. Or you may discover that your solution doesn’t make sense to customers at all, that you forgot to consider a key constraint, that you addressed only part of their end-to-end problem, or that the overall approach is flawed. It is not uncommon to discover in this first customer touch point that the problem you thought you were solving isn’t really a problem—or that you misunderstood the essence of what needed to be solved. Or perhaps what you thought would be a key source of delight for your customers just doesn’t resonate with them.

All of these possible outcomes have dramatic impact on the future of the project—and might even cause you to scrap it and focus on a different scenario altogether. Yet, in a typical software project, you wouldn’t get this kind of feedback until the code is well on its way, and possibly already working. We’ve all been in situations where we discover way too late in a product cycle that we made a bad assumption somewhere along the line, that there was a crucial misunderstanding or a critical missing component. The idea here is to ferret out those big oversights much earlier, while it’s still cheap and easy to recover. For highest efficiency, optimize your approach to verify that you’ve got the big picture correct first, before you spend too much time on the details, and certainly before you invest in writing production code. That way, if you find that you need to change direction, you haven’t wasted much time or effort. Showing your early, incomplete thinking to customers is a powerful and effective way to catch these big bugs early.

We’ll discuss ways to get customer feedback on your prototypes in more detail in Chapter 9, “Observing customers: Getting feedback.”

Keep iterating

You’ve surely noticed that this approach isn’t a linear progression of steps but rather a cycle. In fact, the most important aspect of the Fast Feedback Cycle isn’t any single step, but rather the fact that you repeat them. The faster you loop through the cycle, the better the results you are likely to get and the more efficient your team will be. We call each trip around the cycle an iteration.

Here is a description of how your thinking might progress as you continue iterating after your first trip around the Fast Feedback Cycle:

• Observe So, what happens after you show your first prototypes to customers? Well, it all depends on what you learn from the customer feedback you receive. Perhaps customers loved what you showed them. More likely, they liked parts of it, but other parts didn’t seem relevant to their needs or just plain didn’t make sense.
• Frame Now compare the customer feedback against your scenario. This may be a very quick check to confirm that the feedback you received is consistent with the scenario you originally wrote. However, perhaps the feedback suggests that you need to adjust the scenario. Maybe your customers’ actual needs or motivations are different from what you initially thought. Maybe their judgment of what a good solution looks like is different, which would suggest that you change the success metrics. Maybe there is a constraint that you didn’t realize—for instance, you didn’t consider that your customer might not have reliable access to the Internet when using your service on the road. Any of these things might cause you to edit your scenario to include these new needs and insights.
• Brainstorm Next, you move on to exploring some revised alternatives, this time with new information to inspire your idea generation in slightly different directions. Perhaps the solution alternatives you originally prototyped didn’t quite hit the mark, so you need to try a different approach. Or, perhaps your high-level solutions were working fine with customers, so now your brainstorming is focused primarily on getting to the next level of detail, to bring your rough solutions to a higher level of fidelity. But still, the key behavior is to generate more ideas than you need so that you have done the due diligence to really explore the problem space before you decide which ideas are worth pursuing.
• Build Then, as before, you move on to building or prototyping, but this time you create fewer alternatives and develop them in somewhat more detail than in the previous iteration. You still use the fastest prototyping techniques that are sufficient to get customer feedback. Perhaps after seeing what worked for customers, you can combine ideas from multiple approaches to create an even better prototype. At this stage you should also start exploring which alternatives not only delight customers but are also possible and reasonable from an engineering perspective and support your business plan. You also transition into writing production code.
• Observe, again Then you get that second round of prototypes right back in front of customers, just as you did before, and so it continues. But be careful that you don’t ask for feedback from just anyone—be sure that you prioritize getting feedback from your target customer.
• Keep going The Fast Feedback Cycle can be repeated over and over, and, indeed, you should continue repeating it throughout your project, all the way until the endgame, when you have a final, shippable solution that customers find delightful and that works flawlessly.

As your project progresses, your iterations through the Fast Feedback Cycle will naturally shift their core activities, even though the essence of each stage remains the same. For instance, in early iterations, the framing stage is mostly about deciding which problem to solve and making sure you have described it correctly. As you start iterating, you might realize that your understanding of the problem is incomplete, so you normally update your framing once or twice. In later iterations, this stage becomes more about assessing your solution against that frame to be sure that you are still solving the problem you initially set out to address and assessing how close you are to achieving your metrics.

Similarly, in later iterations, the build stage becomes less about prototyping multiple alternatives and more about writing production code, fixing bugs, and fine-tuning the details. As you shift from showing mocked-up prototypes, to having customers attempt to use working code, to finalizing production code, the techniques you use to gather customer feedback will change as well.

After you go through the Fast Feedback Cycle a few times, you should have a concept for an end-to-end solution that you like—one that works from a feasibility perspective and is testing well with customers. Only then is it time to start breaking that solution down into the bits and parts of features and work items that will populate your product backlog, Team Foundation Server, or whatever system you use for tracking engineering work items. But even after you start writing production code, continue iterating: test your latest builds with customers, check against your scenario to be sure you are still solving the customers’ core problem, and whenever you find that you have to make a design decision (addressing customer feedback, building the next feature, or fixing a bug), take a couple of minutes to brainstorm some alternatives and perhaps even quickly test them with customers before you pick an approach to move forward with.

It’s important to note that the rhythm of constant and rapid iteration around the Fast Feedback Cycle should continue throughout the project, even during milestones that are primarily about implementation or stabilization. Iteration is not just for the early stages of a project. Obtaining continual customer feedback on your work in progress is essential for maximizing efficiency and minimizing wasted effort, and it may cause you to rethink the need for a separate implementation or stabilization milestone in the first place.