Evidence-Based Portfolio Management
How Evidence-Based Management Can Help Achieve Better Business Outcomes
Introduction
Organisations view agile approaches as the silver bullet to improve their market competitiveness by how quickly they respond to market changes. If you ask these organisations how that is possible, a common response would be “Our agile teams deliver business value faster.” If this sounds familiar to you, you are not alone.
While many organisations reaped benefits from their agile teams, the Agile practices must extend beyond the agile teams for these organisations to realise full improvement in their business outcomes.
Organisations often have the misconception that they only have to deliver faster to stay competitive, without much consideration over what they are delivering. The true value of agility lies not in the pace to deliver, but the ability to experiment ideas to learn what works and what does not, and to gather valuable feedback from end users. It is often the ability to react to these feedback that separates success from failure.
To help organisations invest wisely in innovation, we leverage on the learnings from Evidence-Based Management (EBM). EBM advocates delivering small increments to validate a hypothesis, collect feedback and make decisions based on evidence from the market.
Problem With How Organisations Invest
The traditional way organisations invest in innovation to build up their portfolio may impair the organisations from realising the full benefits from their investment. The amount of risks involved also reduces the organisations’ ability to pivot towards a new strategy in response to market changes.
So, what are the common reasons for such observations?
Vague business outcome
Many innovation journeys and products are built without a clear idea of the business outcome they are trying to achieve. Investments are made based on “insights” coming from HiPPO and ZEbRA in the organisation that do not yield the perceived values in the end.
Guesstimates under pressure
Based on those vague “insights”, the project team is often under tremendous pressure to provide estimates without a clear understanding of what is needed. In most cases, as you’d expect, the estimates are way off from the reality as the actual work unfolds.
Excessive planning and unwilling to change
Project teams spend weeks or months to create ‘perfect’ project plans when there are many unknowns at the beginning. Beautiful Gantt charts are created, but they often become obsoleted the moment the tyre hits the road. Why? Is it because the teams lack experience? Definitely not. Project teams are made up of smart people with vast experience in their respective fields. Unfortunately what they are unable to do, is to predict the future.
With so much effort being put into coming up with the plan, any changes to the plan is deemed unfavourable. With such high inertia to change, organisations restrict their ability to learn and agilely to respond to market changes.
What Is Evidence-Based Management?
EBM is an approach that helps organisations continuously improve business outcomes through intentional experimentation and feedback.
By knowing the as-is and to-be states, EBM forms small experiments that can be run quickly to provide evidence that helps organisations to make strategic business decisions on the next steps get to their goals.
EBM focuses on two main principles — experiments over excessive planning and less is more.
Experiments over excessive planning
In a world full of uncertainties, exhaustive planning is futile since no one has the crystal ball to look into what holds in the future. This makes building what the market wants in the future a challenge. Things get even more challenging when end users do not know what they really need until they see it.
“People don’t know what they want until you show it to them.” — Steve Jobs
Exactly how can we read things that are not yet on the page? This is how working empirically to run small, but focused experiments will help organisations achieve the right business outcome.
Not only does running these small experiments enable organisations to find out what the market needs, it also reduces potential investment loss. With small experiments, new ideas can be validated quickly and prevent the organisations from investing heavily in the wrong initiatives from the start.
Less is more
A common behaviour observed in organisations is they have many good ideas that they would like to experiment. Often these ideas are perceived to be time-sensitive. “We need to go to market faster” says the executive. This results in teams multitasking and working long hours to meet tight deadlines, which inevitably leads to burnout and high turn-over rates. For those who remain, they often context switch, thus losing focus and effectiveness. As a result, initiatives take longer to deliver.
Having teams to work on fewer things and focus on what matters help organisations to focus and deliver faster.
Apply EBM To Your Portfolio Management
EBM leverages on continuous feedback to help an organisation assess and realise its business goals more effectively, allowing it to focus on initiatives that move the needle and abandon those that don’t.
I have broken down the cycle into the following stages.
Setting and resetting goals
Let’s start by looking at two of the four key measures in EBM.
- Current Value (CV) refers to the value a user gets from using the product or service, the value employees get from working in the organisation, and the value investors get from investing in the organisation.
- Unrealised Value (UV) is the remaining opportunity in the market that is untapped by the organisation.
Goals set by an organisation should aim to close the gap between the CV and the UV, thereby improving the user experience.
It is important to highlight that as with all goals setting, these goals set by the organisation should be measurable. This allows the organisation to measure the actual outcome against what is expected, thus enabling decision-making based on the results. The organisation must also be prepared to continuously evaluate and evolve its goals based on the market feedback. Some organisations use a backlog to capture the goals which they constantly refine and re-prioritise.
Assess and remove unrelated work items
Focus is a crucial element for an organisation to be successful. As an organisation embarks on its journey to achieve business goals, there will bound to be distractions popping up every now and then. If the organisation lacks discipline and starts to work on these distractions, it will take time and attention from things that matter.
As such, it is important to periodically examine the work that the organisation is doing and ensure they are contributing to its goals. If the work does not contribute towards goals, be brave to stop it and shift focus to work that advances the intended outcome.
Start experimenting
To get an organisation to meet its goals, iterative experiments are used to validate the different hypotheses. The target audience for the experiment, what needs to be developed (could be a feature in the product or a process change), and the measurements for the outcome are all required for the experiment.
As with all Agile practices, we look to the team that is closest to the ground to propose what needs to be done in order to achieve the intended goal(s). I will emphasise that no one should dictate the work required other than the team itself.
Once the proposal from the team is ready, the executives and teams that make the proposal collectively evaluate and decide how should the organisation invest to meet goals. These discussions should be relatively short since the teams only need to determine what they need to focus on in the next two to four weeks. It is important to note that the discussions should not go into nitty gritty details around execution but to focus on how the proposal would contribute to the overall goals.
Next up is running the selected experiments. The proposal is broken down into Product Backlog Items (PBIs) and the teams will decide how what are the measurements of success. They then include these measurements into the feature of the product so user feedback can be collected. The feedback will then be used to validate the hypotheses and the results will be used to guide future investments.
Evaluate progress towards business goals
Like all empirical approaches, the teams need to evaluate at regular intervals if they are marching towards the intended goals.
During this stage, executives and teams will assess the results or feedback received and determine the logical next steps. It could be a decision to push ahead or to pivot towards other focus areas. Based on the evaluation outcome, the product roadmap will be updated and the next iterative cycle starts.
Conclusion
In today’s market environment, nothing is a constant except for changes. To reap the most benefits out of investments, an organisation needs to be agile and responsive. There is no better way to be agile and responsive than to be clear of what the organisation goals are and to run short bursts of experiments to test the market. Based on feedback received, to inspect and adapt the goals and investments.
The investment cycle can be summarised as:
- Link investments to business goals
- Have the teams on the ground propose short and focused experiments
- Evaluate the evidence from experiments to decide next steps
Since the duration of each experimentation is relatively short, there is no better time than to give this method a try immediately. Simply pick a strategic initiative and get going. For those who have given this a go, I am keen to know how it went.
