Amanda Khatri
Editorial Manager
Continuous monitoring vs. outcome testing: two sides of the same coin
Compliance expert and former Head of Compliance, Sylvia Yarbough, shares secrets and insights from the heart of the compliance team.
For the last few years as data has become more readily available, risk management teams have begun to find ways to use data in monitoring and testing. As with many things in compliance, this has been an uphill battle. Organizations with strong analytical and data management teams are making significant strides, but others are falling behind.
I had the opportunity to make progress in the compliance monitoring arena by building in-house and partnering with a third-party vendor. After a few years of slow progress, the monitoring program moved from monthly reporting to continuous monitoring of some products. I won’t try to sugar-coat it; the struggles were great. Internally, we didn’t have sufficient analytical support and getting funding for expanding third-party support was always a challenge – especially in a tight expense-management environment.
Nevertheless, my monitoring team made progress and demonstrated to the business, 1st line risk and audit and the value this program could bring to the entire organization. While my compliance team raced to make progress to counter-balance cuts in resources and increased regulatory scrutiny, some 1st line risk managers were discovering outcome testing. This focused on automating the control testing to support the Risk Control Self-Assessment (RCSA) program.
The article’s subtitle, “…two sides of the same coin” alludes to how continuous monitoring and outcome testing are valid attempts to automate processes that ultimately achieve the same outcome: understanding and managing risk.
For individuals new to these concepts, I want to take a few moments to describe each. So, for the experts among you, please bear with me and read on.
Continuous monitoring
In the compliance world, our focus is on regulatory requirements. Therefore, in developing a monitoring program, we focus on aligning these requirements to the business process which would prove if the requirements were being met. Often, we leverage data elements to prove the process.
This could be as simple as comparing two dates to see if adverse action notices are sent on time. Or get more complex with document scanning and text analytics to ensure the accuracy of a mortgage settlement statement. In all cases, the focus is on running the transaction through the analysis, identifying the exceptions, and – based on the risk associated with the requirement – setting acceptance thresholds and tolerance levels. This puts compliance, 1st line risk, and the business partners in the best possible place to understand where their highest risks are and focus on improving processes upstream to minimize, if not eliminate the exceptions.
The term ‘continuous’ can be defined by the frequency of monitoring that must occur. For some activities, monitoring may need to be done daily. For others, it could be monthly, e.g., FCRA reporting. The value of this type of monitoring will benefit the audit team, who can inspect the same data and/or look at the monitoring process to form opinions on the compliance program.
Outcome testing
Outcome testing’s final results are similar and uses data to test the steps in the business process. However, the analytics is built based on the control’s definition. The premise assumes that the controls are accurately defined, therefore, finding ways to automate testing them versus the manual efforts. Controls should already be aligned to regulatory requirements that are risk rated. The outcome is that exceptions can be identified, thresholds set, and 1st line risk, compliance, and business partners can focus their attention on the areas of highest risk that seem problematic. More effort can then be put into fixing the business process that caused the exception.
Two sides of the same coin
As you can see, the only difference between the two approaches is the angle that drives your analytics development. In both methods documenting business process, it is foundational that the department focuses on products and process horizontal rather than vertical. As a compliance professional, I am a firm believer in testing compliance with regulations. You create the analytics based on the regulatory requirements aligned to the process, eliminating the steps of creating controls, thereby achieving more precisely defined continuous monitoring. Nonetheless, some would disagree and believe leveraging the controls is a better process.
Perhaps I’m a cynic to that approach as I’ve seen numerous poorly written controls and believe we should spend time rewriting them rather than focusing on getting to the end. For those who believe in the quality of their controls, building your analytics to achieve outcome testing is doable. I’ve discovered that organizations that are firmly wedded to their control testing process are more focused on automating the controls testing. These are often the same organizations that are constantly trying to enhance, improve and revamp these same controls – remaining in a constant state of rebuilding.
My statement still holds true for testing operational risk. To endear myself to my 1st line risk and business partners, I extended my analytics development to include operational items that have no regulatory implications. It is easy enough if you are doing a business process walk-through to identify operational items that align with policies and procedures. Once my team can access the appropriate data, they can simply retrieve data elements to support testing operational points if that data is available. Please note that continuous monitoring does not have to be just for regulatory focus. I am a compliance professional but first and foremost I am a risk professional, so if I can kill two birds with one stone when developing a new program then why not?
In closing, the last thing I would encourage is that an organization should pick one method or another. Either choose continuous monitoring or outcome testing. I am a firm believer in using the continuous monitoring process and eliminating the entire control process as it becomes redundant and an unnecessary burden. Continuous monitoring will still support Risk Self-Assessment (RCSA). You are just replacing the control testing with a different C – continuous monitoring becomes the control. The 1st line and their business partners still can land on a residual rating with more accurate testing results. The 2nd line can better assess the overall risk environment, you’re more likely to land on the same residual risk. This is because you are leveraging a more accurate basis versus the 1st and 2nd lines performing their individual testing routines.
It’s difficult for organizations to tackle both simultaneously, so if you are just getting started on this path, always pick one. Otherwise, through the competitive nature of human beings and the race to get things done, there could be repercussions – something I have seen first-hand. Coming at this work from both sides only results in the misalignment of risk and technology resources. In addition, it can cause business overload. The business lines must support both efforts to include business process walkthrough, vetting sources of data, and reviewing preliminary outcomes. Plus, the man-hours spent in comparing results that were created using different processes.
By deciding on one path, everyone in the organization can align their goals, support the program, build, and leverage the same outcomes. If you have a stake in the game, vote for continuous monitoring, it truly is a game changer.
A few things to keep in mind regardless of the method your organization chooses:
(1) focus on the products/processes with the highest risk,
(2) focus on the highest risk regulations,
(3) build wide, not deep to start,
(4) bring the entire organization along in the journey – change is difficult,
(5) and yes, you can get the regulators behind eliminating controls and control testing if you can show them a process that is working much better and fits into the expectations of strong risk culture and the 3 lines of defense model.
