Date: Tue 10 Aug 2021
Title: How to Facilitate High Quality Risk Assessment
High Quality Risk Assessment implies comprehensive risk identification using certain guidelines to elicit and record notions of risk, and a sensible assessment using four key criteria. I share a generic methodology developed and refined over years with clients.
Let’s review what we accomplished by using a round table of experts (described last time) for risk identification, then go ahead with further detail on the process: how to facilitate High Quality Risk Assessment. There are four key criteria essential to risk assessment.
Instead of the disparate and vague risk information often gathered through interviews and informal, ad hoc approaches, we used an ordered method. We want to benefit from many person-years of experience and professional memory, mapped against a common context, in the most efficient way possible, within the constraints of the limited resources.
Risk identification, done properly, is:
– understood among participants using clear definitions;
– directly related to goals and core values;
– projection of future imagined possibilities by virtue of careful consideration of context;
– comprehensive by virtue of 3 elements;
– efficiently completed, within a focused context.
And referring to earlier podcast episodes, we see the quality of our risk ID rests on a firm foundation of informed planning and proper goal formulation.
Conducting the risk identification and assessment session
– the best use of meeting time;
– balance between free-flowing discussion and close analysis (risk formulation);
– practical tips in facilitating the session;
– my method is what I call LIFT: Listen; Interpret; Formulate; Test;
– your personal facilitation style;
– demonstration of method: skills transfer.
What are the four aspects of risk assessment, to be captured in the risk register?
– particular design of the risk register: see recommendations in Tools and Templates
– Likelihood (probability); Consequence (severity);
– Existing controls, not considered as just financial controls;
– Risk tolerance – use short high-medium-low statement in risk register;
– Making sense of “risk tolerance” (see article on risk tolerance and risk appetite)
– Order of operations at the risk identification session
1. The definition of High Quality Risk Assessment was given in Ep 04; I repeat it here for convenience.
2. The most advantageous method for risk identification is the round table of experts approach.
3. The whole method is grounded in consistent definitions and rigorous planning practice.
4. Facilitating the session is a matter of practice, with several nuances and finer points, ideally first explored in trial runs on smaller projects.
5. My method can be summarized as LIFT (Listen; Interpret; Formulate; Test).
6. Risk assessment per se is a matter of specifying four criteria (L; C; controls; tolerance).
Definition of High Quality Risk Assessment
“The comprehensive identification and analysis of phenomena that could prevent the achievement of objectives, or compromise associated values, of a researched and planned program, followed by a principled response.” (Solving the ERM Puzzle, p.11)
(Robertson 2016) Enterprise Risk Management Tools and Templates
(Robertson 2016) Solving the Enterprise Risk Management Puzzle: Secrets to Successful Implementation
RIMS document, pdf download Exploring Risk Appetite and Risk Tolerance
[edited for clarity]
This is Episode 11: How to Facilitate High Quality Risk Assessment.
In this episode. I’d like to review first of all what we accomplished last time, in doing risk identification according to what I call High Quality Risk Assessment, and the reasons for using the roundtable of experts approach to that exercise. So we’ll do that review fairly quickly and then we’ll move on to further detail on how to facilitate High Quality Risk Assessment, and how to carry out that assessment using the generic methodology that we recommend.
Instead of an informal or an ad hoc approach to risk identification, or just having a discussion around the table and taking notes and that sort of thing, we decided that it’s much better to have an ordered, structured approach to risk ID — in fact, to make sure that the quality [of risk information] is “built in”. For that reason we issued some caution around the idea of using either surveys or interviews, which can entail all kinds of problems of data quality. We recommend instead to use the roundtable of experts approach, so instead of disparate and vague risk information, which is often gathered in an uninformed approach, we’re going to use an ordered method.
Now, the great benefit of using a roundtable of experts approach is that you’re able to “map” [to use a mathematical term] many person-years of experience and professional memory against a common context. That context is carefully defined; everyone signed off on it; you sort out all kinds of foundational problems at the front end. Moreover, you’re making best use of the time that you’ve got. You’re working within limited constraints [of] getting people to cooperate to contribute risk information, and perhaps to attend meetings, so this is really the best way to make the highest, most efficient use of that time.
Consider this list of benefits:
a) You’re going to be conducting risk identification, which is, first of all, understood among participants to be the same thing — in other words, everyone’s using the same set of definitions with regard to what a risk is, and what the bounds of the discussion are.
b) The discussion of risk is going to be focused very closely on goals and objectives that are already clearly stated. That means that you’re going to be limiting and actually excluding — precluding — the discussion of risk that is irrelevant, that goes all over the map, that is out of context, out of scope, and that is just a time waster.
c) Now, in so doing, you won’t be closing your eyes to risk; you won’t be burying your head in the sand, because all of the large influences — let’s say, the conditions and trends that are extant in the environment that are affecting our business. Well, these things are not ignored. They’re taken account of in an earlier stage, in the planning.
d) This risk ID method, further, is not simply a look into the past to review the risks that already occurred or the accidents that happened, or even instances of non-compliance. No, beyond those conceptions of risk, we are actually asking participants to project their thinking, their imagination, into the future and to surmise what the uncertainties will be with regard to their abilities to execute on the stated goals and objectives.
Now of course to do that, they do rely on their professional memories — that’s understood — but it’s not simply a review of risks that have already matured. There are many ways in which problems experienced in the past can be understood and synthesized in one’s mind, to apply as a general principle in a new context. And that’s exactly what we’re counting on the participants in a round table to do.
e) Another characteristic of this risk ID method is that it’s comprehensive, that is, reasonably comprehensive within the constraints of the limits of our resources. It’s comprehensive by virtue of three things: 1. first of all we are tracing through a carefully formulated set of goals and objectives, and a context paper that keeps us on track; 2. Secondly, we’re using risk categories to make sure that people are considering all various sources of risk; 3. Third, we’re using the roundtable itself which is already a multiplicity of views of a common context.
Referring to [point (c) above and] even earlier podcast episodes in this series, let’s not forget that people will not be able to sit at the table and impugn the goals and objectives that have been formulated. Why? Because we already vetted the planning practice. We know that our goals and objectives are really the result of a serious exercise on analyzing the strategic context of the organization; doing an environmental scan; establishing our priorities properly; and formulating those goals and objectives in such a way that they’re actionable.
Well, we addressed the idea of starting the risk ID and how to conduct the session in our last podcast episode. But let’s elaborate on that a bit further now.
The idea is to somehow facilitate a discussion, and if you’re not practiced at this and you haven’t thought about it, very quickly [you find] that the discussion can go off the rails. And you’ll find that you wasted the meeting time that you had worked so hard to put together and get people to agree to.
Now a way to practice this is to do some trial projects. You don’t want to start off with the senior executive group on your first run trying to do risk ID. As you begin to discuss risk, you [start to] trace through each element of the context — that is, each goal and objective that comes under consideration — and ask people to contribute their ideas of what the uncertainties are with regard to trying to execute on these goals and objectives.
Getting started can be difficult, because it’s sort of like having a blank slate or a blank canvas in front of you. What I recommend is that you simply ask your program lead to begin the discussion, and state what that person considers to be one of the main risks in the project.
Now, once a person suggests an idea as to what the risk is, with regard to any particular goal, objective, activity or even a corporate value (to be preserved and safeguarded), the idea is to let the discussion ensue. Let people bat the idea back and forth, so to speak, exchange views and begin to understand one another’s opinion — one another’s stance on various issues.
But at the same time what the facilitator is doing is listening very carefully. At a certain point the facilitator wants to interrupt, and say, all right, I think I’ve got an idea of what the risk is, based on the discussion. And what you’ve done is, you’ve typed a trial version of the risk statement in the risk register. Now, this is all displayed on a screen so that everyone can see what you’re doing. And you check back with the participants, especially with the person who suggested the idea of the risk, and you say: is this the correct formulation? Then people will chime in; they’ll say “change this; change that word… okay, that’s right!” By sort of a process of consensus, people have usually an agreed view on what the proper expression of the risk is, for that particular issue, that particular goal or objective that is under consideration.
As you continue down the list of goals and objectives in the context, you’ll find that this [process]:
a) consideration of the risk
b) formulation of the risk
c) typing it into the risk register
d) getting that corrected and getting everyone to agree…
e) and then moving on to the to the next item
— you’ll find that develops into a cadence or rhythm that people get used to, and start to appreciate, because they perceive that they’re actually getting through the agenda quite quickly. The discussion has some has some sort of order to it.
Make sure you don’t let any particular person or group of persons dominate the discussion, by asking everyone around the table to contribute ideas — elicit their [everyone’s] ideas of risk. Make sure each one of those risks statements adheres rigorously to the rules that we established (that was in the last episode #10 — you can check that, and there’s also a blog post on the same topic.)
Be mindful that you don’t let anyone sneak in these considerations of broad risk that should have been taken care of in the planning. In other words, you don’t want to let people start to identify risk that is really out of scope. Now it’s not illegitimate to say something like okay, well there could be fire; there could be a hazard risk, or something. And if that’s the case, then, as I recommend, you just take that sort of thing off line, and say, well, we have to conduct a dedicated session to Business Continuity and Emergency Planning. That’s where the identification of hazard risk would really properly belong!
As I mentioned last time, there’s nothing wrong with the facilitator suggesting what the risk might be. Realistically speaking, the facilitator will be bringing a whole different management experience and history to the table, which could be very useful. It’s only necessary that you make clear when you’re contributing, as opposed to simply facilitating.
Then again, it is perfectly feasible to facilitate a session when you don’t know very much about the content. In fact, that actually gives you a license to ask questions with regard to the logical continuity of people’s remarks…to ask questions (since you don’t know too much about a highly specialized subject matter). And that can actually be useful because people often assume certain things that are contradictory, and that really need to be examined for full disclosure of risk.
Now if the subject matter is particularly contentious, if it’s really controversial or potentially charged with emotion, and so on, then first of all, it’s a great idea to get formal training in facilitation before trying to conduct such a session.
Another way to help handle that kind of situation is to facilitate in tandem with someone else, so that people perceive that there’s a balanced approach to the facilitation — it doesn’t bring any undue bias.
Well, there’s more discussion on how to actually run the session both in the last podcast episode and in the book. It’s very difficult to try to list everything in this format; it’s a rather lengthy discussion. The thing is that it’s going to involve some experimentation and development of your own personal facilitation style.
I try to encapsulate my method with an acronym LIFT which stands for: Listen; Interpret; Formulate and Test.
Listen very carefully to the participants to see what their view of risk is, and how they might express it. Keep in mind that people won’t always express perfectly well even what their thoughts are with regard to the risk. It’s a common difficulty that all of us have because when we’re too close to the business, it’s sometimes hard to articulate it. And that’s why I have the next step: Interpret. You try to interpret what they say… and then Formulate what they say, by typing in a trial risk statement, so that everyone can see. Then you check back. That’s the Test.
Now, I found that in the vast majority of cases, people will actually agree as to the formulation of the risk when you use that method. If there’s some serious disagreement to how the risk should be formulated, it often points to a root issue, where there’s two issues that need to be expressed as two different risks — something like that!
Finally, with regard to process, I want to point out that one of the main purposes of this session is to demonstrate the method to the program lead, and the participants themselves. Because what we’re doing, eventually in enterprise risk management, is using the skills transfer method. We want to demonstrate the risk ID and assessment methodology, so that other people can pick it up, and lead the session themselves when they get back to the office and continue in their normal business. For that reason, what you want to do is trade off with the program lead, from time to time, to give that person practice [in leading the session].
So that’s pretty much all I want to say about conducting the session, that is, leading the process, for the moment. As I say, check back with the previous episode because I had other comments with regard to how to facilitate the session. But for now, let’s continue on with the idea of assessing risk.
How do we do risk assessment? Well, if we looked at the recommended risk registers, or the process that has been suggested in the standards, of course we come across the idea of establishing for each risk: a) the likelihood (that is the probability of its occurrence) and b) the consequence (or the degree of severity) of the risk should it actually mature [occur].
I’ve got some detailed recommendations on how to set up the schema for Likelihood and Consequence, and how to interpret each one. I think there’s going to be some differences from one organization to another in terms of these schemas, and exactly how the risk register is designed. You can see examples in my Tools and Templates. (You can get Enterprise Risk Management Tools and Templates, or even check the excerpts from it that I publish with the show notes that go out to subscribers. To received that, you subscribe at the website RiskCommentary.com. )
But regardless of how the schema for Likelihood and Consequence is designed, the important thing to consider is that, in order to do this quickly, and at the same time, to not be deficient, there’s a way to assess risk by using four lenses — looking at the risk through four different lenses, so to speak. That is, applying four different criteria.
Of course, the first two are the Likelihood and Consequence. Now we know that we don’t have [most of the time] actuarial data or statistical data that is going to give us exact probabilities with regard to forecasting the probability of a given a risk event. So for this reason, we just want to use our professional judgment to estimate the possibility, based on our past experience and the present circumstances.
When we assign a ranking for the Consequence or degree of severity of the risk, it’s always in relation to the goals and objectives that are under consideration.
Now, if you test this with certain risks in your context, you’ll probably find that just using Likelihood and Consequence — and coming up with a resultant ranking — is really insufficient. You need to consider two more things. One is the Existing Controls, that is, the controls that are in place already to mitigate or attenuate the risk that you’re considering. And this is not just financial controls. I’m talking about controls in a broad sense, applied to any sort of risk.
Even those three are insufficient (that is, Likelihood, Consequence and characterizing the Existing Controls). The fourth thing that is really necessary is the degree of Tolerance for the risk that you have. That needn’t be a complicated calculation. It can simply be a statement of “high”, “medium” or “low”. The reason for that is you might have a risk that has a certain criticality, or resultant ranking, with a certain measure of control that’s already applied against it. And yet, because of your low tolerance for that particular risk, you decide that you must take action.
So I found that, on balance, you’re able to record Likelihood, Consequence; characterize the Existing Controls (just very briefly, in a few words in a statement in the risk register); and at the same time put an indication of your Tolerance for that risk, simply by expressing high, medium, or low.
As a result of those four lenses, you’re able to make a really well informed decision on how you want to respond to that risk.
Just a reminder that if you want to look more deeply into this question of risk tolerance and risk appetite, I put a link in the show notes to an article that I recommended previously in Episode 3.
One final point on High Quality Risk Assessment facilitation. I recommend that you first list all of the risks in the risk register before you start on the assessment of each one. The reason is that, often, you’ll want to adjust the formulation, the statement of risks, as you proceed down the risk register and cover all the issues. Also, you’re likely to change your opinion of the source of the risk, or the risk category, for different items. And the last reason is that if you try to assess each risk as you formulate it, it interrupts the flow of the discussion.
Let’s summarize what we covered today!
1. The definition of High Quality Risk Assessment was given back in Episode 4. But I repeat it here in the show notes.
2. The most advantageous method for risk identification is the roundtable of experts approach, for various reasons.
3. The whole method is grounded in consistent definitions and a rigorous planning practice.
4. Facilitating the session is a matter of practice, with several nuances and finer points — ideally, first explored in trial runs on smaller projects.
5. My method can be summarized as L I F T:
Listen; Interpret; Formulate the risk; Test, or check back with participants.
6. Risk assessment per se is a matter of specifying four criteria. In my experience it’s sufficient, and necessary, to do at least: Likelihood, Consequence, assessment of Existing Controls; and level of Tolerance.
In the next episode, we’ll discuss the significance of the risk register — that you’ve built so carefully! — and how to do breakthrough, dramatic risk mitigation.