MEASURING RISK MATURITY IN THE AUSTRALIAN GOVERNMENT
Age does not equate to maturity per se – simply existing and crossing some arbitrary timeline doesn’t guarantee a wiser outlook.
So it is with risk maturity. Building a risk management framework is a good start and a critical foundation, but how do you know whether the underlying activities are risk ignorant or risk aware?
Useful signposts are the range of criteria setting out differing characteristics along the maturity path. In the Commonwealth government environment, the Comcover[1] risk management capability maturity model for the 150+ entities in the Australian Public Service identifies six capability descriptors: Fundamental, Developed, Systematic; Integrated; Advanced and Optimal. [2]
Other descriptors used in the government and private sectors include: Basic, Mature & Advanced (Canadian Government); Functional, Co-ordinated, Standardised, Integrated & Optimised (United States Government); Basic, Organisation wide rules/processes in place, Processes part of daily life and strategy, Fully integrated and internally assured; and Values & behaviours fully aligned and externally assured (private sector model); and Critical control approaches which measure the extent to which risk controls are known, clearly identify who is accountable for control implementation and monitoring and how routinely the ‘health’ of controls is checked (private sector).
What do these descriptors mean? What does good look like? Again using some Commonwealth descriptors (levels 2-5) as an example:
Developed (level 2): A risk management policy & framework endorsed by the accountable authority (entity Secretary/CEO); absence of common risk language; ad-hoc processes to discuss and understand shared risks; limited and/or shared resources allocated to manage risk.
Systematic (level 3) A risk management framework that is fully embedded; a high level and qualitative risk appetite statement; a common understanding of the importance of managing risk events; formal arrangements to discuss and understand shared risk; accountability and responsibility for managing risk is clearly defined within the governance framework; and dedicated staff to implement the risk management framework.
Integrated (level 4) A risk management framework is part of the overarching governance and management framework; risk appetite statement contains both quantitative and qualitative elements which are linked to strategy and communicated to all staff; the risk management program is reviewed regularly to identify improvement opportunities and assess the level of investment in risk management activities; and risk information and data is stored in a readily accessible central repository.
Advanced (level 5) A risk management policy integrated with strategic and business planning processes and reviewed and updated annually (or as changes arise); formal arrangements facilitate identification of current, future, emerging and shared risks; these are clearly articulated across the entity; a senior executive sponsor leads, promotes and drives risk management capability; and the risk management framework includes measures for the accountability and management of risk controls at business unit and program levels.
As can be seen, each level requires increasing commitment and resources. Just getting to level 3 requires a fully embedded risk management framework, organisational wide acceptance of the importance of risk management and clearly defined accountabilities.
Given these descriptors, what ways are used to test where an organisation sits on the risk continuum?
In relation to the Commonwealth criteria, each year Comcover members undertake a benchmarking survey to assess their risk maturity. Since 2014, the survey has assessed maturity against the nine elements of the Commonwealth Risk Management Policy. Survey questions relate to the content of an organisation’s risk management framework and policy, the extent and use of risk appetite, types of information gathered and how it is assessed, risk accountabilities and responsibilities, risk culture and ongoing system review.
The survey results are only provided to Comcover members and not made public. However, the State of the Service Report (2017-2018) published by the Australian Public Service Commission[3] (APSC Report) provides an overview of the 2017-18 Comcover survey. Page 43 of the APSC Report notes that the survey:
… has shown a consistent increase in risk management maturity in the four years since the Risk Policy was introduced. Data from 2018 found modest improvements against all of the policy’s nine measures. Entities scored best in establishing risk management policies, embedding systematic risk management and defining responsibilities for managing risk.
The lowest scoring measures were developing a positive risk culture, understanding and managing shared risk and maintaining risk management capability. These measures are considered the most challenging to improve because they rely on changes to organisational culture and capability..
While these are generalisations, the trend suggests improved risk maturity, particularly in relation to embedding systematic risk management and defining risk responsibilities.
The APSC Report (p.40) also considered risk maturity in the context of answers to eight questions in the 2018 APS employee census (Response options were: Agree; Neither Agree nor Disagree; or Disagree).
My agency supports employees to escalate risk related issues with managers.
Risk management concerns are discussed openly and honestly in my agency.
Employees in my agency are encouraged to consider opportunities when managing risk.
Employees in my agency have the right skills to manage risk effectively.
When things go wrong, my agency uses this as an opportunity to review, learn, and improve the management of similar risks.
Senior leaders in my agency demonstrate and discuss the importance of managing risk appropriately.
In my agency, the benefits of risk management match the time required to complete risk management activities.
Appropriate risk taking is rewarded in my agency.
The APSC Report’s findings in relation these questions were as follows:
Encouragingly, most respondents agreed that their agency supports escalating risk-related issues to managers. Almost two-thirds of respondents agreed that risk management concerns are discussed openly and honestly in their agency.
However, only 28 per cent agreed that appropriate risk taking is rewarded. A large proportion of respondents neither agreed nor disagreed with the questions posed…
… The results suggest that a significant cohort of employees may not understand their agency’s risk management framework, may not observe or experience risk management in action, or simply do not know how the statements apply in practice in their agency. This suggests there is some way to go in building an appropriate risk culture in the APS.
So within the same APSC Report are two different assessments of Commonwealth risk maturity, albeit based on very different methodologies – a 40-50 question survey compared to eight quite vague and general questions open to very different interpretations.
A further assessment can be found in the recent Thodey/Alexander Independent Review of the PGPA[4] which stated that:
… risk practice across the Commonwealth is still relatively immature. There is still significant work to be done to embed an active engagement with risk into policy development processes and program management practice, and to have officials at all levels appreciate their role to identify and manage risk.
My view, as a former manager of Comcover and now as a private consultant working with Commonwealth agencies on risk and governance issues, is closer to the Thodey/Alexander view. In particular, that most entities have made progress along the maturity continuum, with risk policies and processes now commonplace, but that embedding for most entities is theoretical, rather than practical, such that risk management does not consistently or materially inform day to day operational decisions.
In terms of the Comcover’s risk management capability model, I consider that the actual maturity of entities is likely to be at level 3 or below, which is lower than the general survey trend. Further, that entities are too often (over)optimistically moving themselves up the maturity ladder well before they are actually capable of operating at that level.
In my view, there are a number of reasons for this:
As the survey is a self-assessment, it is not independent, and therefore prone to self-interest and a range of unconscious biases, such as homophily (preference for those with similar views) and information bias (including confirmation bias) [5].
Too much emphasis is placed on the fact of having a risk management policy and framework, employing a CRO and/or a risk team, rather than the extent to which it is understood and applied. Related to this, the survey is often conducted/coordinated by the CRO or others responsible for risk management. It will be difficult for those staff to avoid construing improvements in framework and processes more broadly to also equate to improved maturity.
Even if the focus was on the right areas, the survey questions use language capable of multiple interpretations to try to determine the risk maturity of decision making behaviour across an entire organisation. The APSC survey also suffers from the same problem e.g. Appropriate risk taking is rewarded in my agency. What is appropriate ? In what circumstances?
Where entities have invested resources (CRO and support staff, consultants, developed new policies and tools) there is an expectation that this will automatically generate improved maturity over time. This sometimes militates against deeper questions about the effectiveness of current risk management measures, or interrogating line areas about what they are really doing. If deeper knowledge of an entity’s risk culture indicates that an entity is not at level 3 or 4 but actually at level 2, there is a real question as to how this would be reflected in the overall rating and/or whether senior management would be willing to accept (and tell their accountable authority/CEO) that the entity is not as mature as previous years would suggest (How did we go backward?).
In addition to the amorphous concept of ‘risk culture’ being difficult to define and measure, often risk-averse behavioural elements are ‘unspoken’ yet part of the entity’s values, such as a prevailing culture where staff are not allowed to discuss policy/program/risk details with external agencies.
Knowing more about the Unknowns..
There is clearly value in an entity undertaking a ‘snapshot’ of its risk management systems, whether it is the Comcover survey, independent review or other process. It reminds all staff of the need to engage with risk management processes, (hopefully) identifies areas for improvement, guides entity maturity, and supports better informed risk management reporting (e.g. annual report and corporate plan).
But for the reasons identified, maturity does not equal age or more refined processes. A deeper, more targeted and qualitative dive is necessary to understand how staff actually engage with risk in the murky waters far from senior management, where most policy/program/process transactions occur. This would preferably involve both a survey and qualitative discussions with staff at various levels of the organisation - senior managers, team leaders and those on the front line. Areas to be considered could include:
better understanding the complexity of interactions within the organisation: What are the linkages between the CRO, the legal area, the audit committee? How do policy, research and program areas interact?
behaviours with a risk focus: Is risk listed on meeting agendas? Is the risk policy read, understood and applied in day to day situations? How many staff undergo risk training? Is risk related data collected? Do managers undertake scenario planning or test systems?
responsibility and accountability: Are shared risks actually identified, discussed, agreed and allocated? Do job performance criteria include consequences for failure to manage risk? To what extent is poor performance or culture tolerated?Are incidents or near misses reported, to who and how? Is there a culture of sharing stories/risk events - good or bad?
non-risk specific behaviours (external cultural factors) such as the extent of decision making autonomy, whether diverse, dissenting or contrary views are tolerated/encouraged, performance management dis/incentives, and
greater analysis of controls, control owners and control effectiveness, particularly in relation to shared risks.
These steps require senior managment support and a more significant resource investment (people, time & money) than undertaken currently. However, doing so not only brings an organisation far closer to Knowing its actual risk maturity, but also self reinforces maturity through more meaningful interactions with staff at all levels.
[1] The general insurer of Commonwealth government assets
[2]https://www.finance.gov.au/sites/default/files/commonwealth-risk-management-maturity-model-one-pager.pdf
[3] https://www.apsc.gov.au/state-service-report-2017-18
[4] Alexander and Thodey, Independent Review into the Operation of the Public Governance, Performance and Accountability Act 2013 and Rule (2018) p.20. My previous articles discussing the PGPA review are here.
[5] I note that in past years, Comcover had survey results of several entities independently reviewed, however I am not aware of the review methodology, the review outcomes, or whether these reviews are still undertaken.