How to create Measurable Outcomes

MobiusCanvas-2014-Aug23

Although there is a lot of discussion on “outcomes” and “value,” exactly what this means and how you define and measure it are unclear.

This is a very quick snapshot of the process to arrive at an outcome and how to measure the value delivered progressively using the Mobius loop.

Mobius is a simple strategy and product management framework. The model creates a common language and understanding of the natural flow to think through the problems and objectives, and generate outcomes. You can plug in the tools and methods you prefer to use, so it isn’t trying to reinvent the wheel, rather pull them together into a holistic structure.

You will recognize many components familiar to lean and agile (or spin-offs such as lean start-up, Kanban etc) making it easier to implement into your existing approaches.

Step 1

arrows and question mark

Start with the problem or objective

Create a high-level objective that captures what the customer wants to achieve, or the problem they need to solve.

When you first ask what the objective or problems are, you will get a range of answers, for example; Our customers are abandoning our website during the checkout process or We want a new Customer Relationship Management system. The first example is a problem that the customer needs to solve. The second is what the customer thinks their objective is, but it is not clear why they need it.

This is the initial starting point and a springboard to then delve deeper.

Step 2
submarine
Deep dive

The deep dive is to understand what is really going on and why problems exist. It also sets the stage for the baseline metrics we can use to create the target outcomes. Some research might be based on data captured by tools, other research might be in the field observing customers first hand. Although the information you seek might be scattered or non-existent, start with whatever you can find and improve the data capture iteratively.

A good place to start is to explore the original problems and objectives gathered in step 1.

For the problem Our customers are abandoning our website during the checkout process, the big question is why. Why are customers abandoning the checkout process?

This will trigger finding out some key information, for example:

1. How many people are abandoning the process? How does this compare to similar e-commerce websites?

2. Where are they abandoning the process? Key pages, key actions, what did they try to do immediately after?

3. What data do we already have? E.g. Customer service reports and complaints, web analytics etc.

Existing data might tell you a lot about the problems, or you might need to set up the means to capture the information.  An example of this might be to put in place an exit survey that is triggered every time a customer abandons the process to find out why they are doing this.

In the second example We want a new Customer Relationship Management (CRM) system, the first question we ask is why? What problems are they currently having, or what will change if they have a new CRM? Common reasons might include:

1. A need to improve the speed that people can access reports to be more effective at their work.

2. A need to reduce costs. The current system requires a lot of manual labour that is costly and time intensive.

3. A need to share information across different geographical locations that the current system is not able to do.

Without fully understanding the issues, the development group could build a new CRM that has many bells and whistles but never meets the real outcomes for the customer.

In the deep dive, the research will provide some data on the baseline (starting point) data uncovered. For example; to measure an improvement for the e-commerce website, the percentage of successful conversions in the checkout process will be measured. This will give the teams a number to compare their efforts against to see if they are making a difference, rather than simply delivering more features that have no impact.

Step 3
dart

Create measurable outcomes (MOs)

Outcomes are the desired results. There needs to be a clear definition and measurement of success . To get started it is best to keep it simple and create a plain English statement then move to being able to quantify and provide the means in which you will measure it.

For example, customers will often state they want a “user-friendly product.” This is fine to start with but quickly ask what do you mean by user-friendly? and What changes occur to help you know that this has succeeded? How can we measure this has been achieved? This might be translated into Improve learnability of product and be measured by the ability for a first time user to get up and running with the product without resorting to a help manual or getting frustrated and stopping altogether.

The act of measuring can be quite sophisticated and many people have written extensively on the topic. Tom Gilb has been a major influencer of Mobius yet even Tom will confess that it is not always easy to get people articulating outcomes effectively. To reiterate; your initial outcomes don’t have to be perfect, but start simply and improve them as you go.

There are 6 useful attributes for clarifying outcomes:

1. Name – keep it simple and an action. For example Increase the number of conversions, Decrease time to create a report.

2. Scale – Think of this as the unit of measurement like inches on a ruler e.g. % of successful sales,  or Seconds to create a report

3. How – How to measure and when to measure. E.g. Data gathered from the web analytics tool, measurements taken every 24 hours.

4. Baseline – Current level. The starting point to measure against e.g. 14% of current successful checkouts.

5.Target – Success level desired. E.g. 30%.. Based on similar e-commerce sites and their checkout conversions achieved.

6. Value – What value does this achieve? For example; each 1% checkout conversion is worth 3 million dollars per year. So a 16% improvement equals 48 million dollars annually.

An example of a MO for an e-commerce site , is shown as an arrow. Visualizing the target outcome helps the teams understand where they are currently and be able to show improvements progressively by filling in the white space of the arrow with the amount of improvement made.

outcome arrow

 

For the CRM example; to determine the measurements and value, the team will need to measure how long it takes people to create a report manually, and how much time they believe the new automated system will save.

For example; if creating a report currently takes five minutes and based on some initial tests or existing data the new CRM may reduce this to one minute, how much time and money does this save the organization? This could be calculated by taking a sample set of users (if there is no existing data on this) and multiplying the number of reports generated daily by the number of people generating them then using this to determine how much the time costs based on their daily salary rate.

There may also be other important consequences from delays in generating reports. Financial traders need speed to be able to buy stock at optimal prices where milliseconds count. Or slow speeds to generate reports mean longer waits for customers to get their information resulting in dissatisfaction with the system and going to the competition. Leading to a loss of revenue and a fall in the net promoter score which helps generate new business.

Understanding these impacts will help generate the potential value.

Step 4

signpost_rd st ave

Create options

Once the MOs are created and you know where you want to get to, the next question is how? If the outcome is clear then there are many potential solutions. Rather than calling these requirements, it is better to call them options. They are really hypotheses or guesses, that by delivering the features or activities you will achieve your goals.

The Options step is where you will come up with potential solutions, but also decide which order you may want to execute them based on their value, risk, cost and time constraints. They will also be broken down into smaller chunks and this is where people may create their MVPs (Minimum Viable Products), or groupings of options that make sense to deliver together, or will test out the option in a low risk, low cost way.

Chris Matts and Olav Maassen have written extensively on options and the conditions needed to decide when and how you should execute them. Often knowing when you have to start work to be able to get features to market will help determine its priority, for example there is no point building the most fantastic holiday card features if you release them in December.

Options tend to take one of three forms.

1. Research; activities gather some initial information and investigate

2. Experiment; testing the idea out

3. Probe; try something. Might be a gut-feel test. The aim is to create some feedback when you have little knowledge.

4. Implement; delivering the option

The three stages are not sequential so where knowledge is high and/or risk is low options might be implemented immediately. Where knowledge and risk is low then some research and experimentation will give the teams enough information to then decide if the option is worth implementing or not.

ResearchExperiment

One thing to note; this is not necessarily sequential. Often you might go directly to implementing something that is low risk and worth doing without much research or experimentation. An experiment might also be a probe, to test a random idea that ‘feels right’. After a while, good product managers steer both with data and instinct.

Step 5

Rocket

Deliver

The Deliver step is where the action takes place. The teams will be researching, experimenting and delivering the options. A delivery process that provides rapid feedback such as Agile and Lean are popular for fast learning and being able to adapt their path continuously. Work will be broken down into very small chunks, often less than three days work and smaller. Techniques such as Goal Mapping are useful for aligning the next layer down details to the outcomes.

Screen Shot 2014-11-26 at 9.32.40 AM

Step 6

dart

Measure

The strength of this process to define outcomes and measure value, is to review the results progressively and let the information gained determine the next steps.

Screen Shot 2014-11-23 at 11.45.23 AM

In the example above, the team has made an 8% improvement.
The value is 8 * 3 million = 24 million dollars annually.

The team can also track the budget spent thus far. To work this out simplistically, they could take the cost per week of the development team and resources, then multiply it by the time spent. This will give a running total of the cost to value delivered. There are other factors to be taken into account but for the sake of a blog post, you will get the essential idea.

 

STEP 7: ADAPT

monkeys adapt

Adapt

After reviewing the results of the research, experiments and implementation options, the team can decide where to focus next. Adapt is deciding whether to continue investing in more of the options already created, or create new options if the cost to value curve is dropping.

Screen Shot 2014-11-26 at 9.42.43 AM

For example; if the results of the experiments show a clear improvement against the current state, then it might be worth implementing in the next cycle. If other options are time sensitive and need to be delivered within the near future, then this will be considered against the other options.

The cycle is continuous. New options are researched and created on the left side of the Mobius loop while others are being experimented and implemented on the right side of the loop.

To increase the throughput and speed around the loop, the organizations capabilities need to be improved. Much like a racing car driving around the track getting faster and faster by tuning and tweaking it as you go.

mobius simple race car

 

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *