Anything less than full monitoring of controls leaves room for error - finally you can do away with manual dip sampling. Cable provides automated evidence of your compliance, risk management and effectiveness, allowing you to:

• save money by eliminating expensive remediation projects
• reduce the risk of regulatory fines
• save time by automating reporting
• improve stakeholder communication
• scale compliantly and with confidence

Why manually test 100 accounts when you can automatically monitor 100%?

‍

Introduction

Regulators, international standard setters and private sector groups have all started to talk about prioritising financial crime effectiveness over technical compliance. There is growing momentum as more people come to the realisation that financial institutions must be able to prove that what they’re doing is not just legally compliant, but is actually working to reduce financial crime.

‍

The overall message from these organisations is clear, and it is only a matter of time before firms around the world are obliged to demonstrate the effectiveness of their financial crime controls.

‍

Whilst there are clear financial benefits to measuring and evidencing financial crime effectiveness, how to do so remains unclear.

‍

This series provides a deep dive into the world of financial crime effectiveness, covering:

‍

1. The Heavy Cost of Ineffectiveness
2. What the Regulators and International Bodies are saying about Effectiveness
3. How to Measure Effectiveness

Download the full whitepaper

‍

Part 3: How to Measure Effectiveness

Summary

1. Measuring effectiveness should cover regulatory and internal control compliance, adherence to your risk-based approach, the performance of controls, unknown unknowns and pure financial crime
2. Manually measuring effectiveness based on dip samples and external reviews is the traditional approach. It is well understood, although it does have some limitations.
3. Automated assurance is the next evolution in financial crime and fundamentally changes and upgrades the nature of your assurance process.

If you’ve read the first two parts in this Series, you now understand the importance of assessing and understanding the effectiveness of your financial crime controls. It’s good for financial institutions, because they can eliminate ineffective controls, reduce costs and improve stakeholder communication. It’s good for regulators, because it gives them a better understanding of whether firms are just ticking boxes or actually managing their risks. And it’s good for society, because it helps us understand how much financial crime we are stopping, and tells us how to stop more.

‍

Regulators are increasingly promoting the concept of effectiveness. It’s only a matter of time before this is embedded in regulations themselves, with firms obliged to prove to the regulators that not only do they have controls in place, but that those controls actually work.

‍

Firms can frequently find themselves playing catch-up with regulatory developments and changing systems or processes which they’ve only recently implemented or reviewed. So when regulators signpost a change in direction, as they are now, there are clear advantages to getting out in front of those changes.

‍

But, how do you measure the effectiveness of your financial crime controls?

‍

What to Measure

It’s all very well to say that firms should measure effectiveness, but what does that actually mean? What specifically should you be measuring, and what metrics and data will tell you if your program is effective?

‍

Regulatory Compliance

The basic starting point is establishing whether your program is compliant with regulatory requirements. If not, you’re in trouble.

‍

You need to ensure that you’re collecting all the mandatory KYC information, all customers and payments are correctly screened, and all appropriate actions are taken when you identify suspicious activity. The question you’re asking here is essentially a binary one - are we meeting our regulatory obligations, yes or no? Provided you understand your obligations and have access to the relevant data and information about your program, you should be able to answer this question.

‍

Internal Control Compliance

As well as regulatory compliance, you need to ensure your financial crime program complies with your internal policies and procedures. If you commit to implementing certain controls, you must adhere to those controls. The basis of any good program is a clear risk appetite, so this is a good place to start. You should confirm your risk appetite is being met through both prohibitions such as “we won’t onboard customers in certain sectors” as well as limits such as “no more than 5% of our customers will be high-risk”. It’s relatively simple to tell what ‘good’ looks like; you established a defined risk appetite statement, so it is important to be sticking to it.

‍

Adherence to your Risk Based Approach

A cornerstone of all financial crime programs is the adoption of a risk-based approach, so you need to confirm this approach is being implemented in practice. Are you correctly applying different levels of KYC to different types of customers? Are you conducting periodic reviews at the correct intervals? Are higher-risk customers being escalated for senior approval?

‍

As well as confirming if you’re implementing a risk-based approach, you should consider whether it’s giving you the results you expect.

‍

Did the outcome that you expected to happen, actually happen?

Are you classifying the right customers as high-risk, or do you actually identify more suspicious activity amongst low or medium-risk customers? If you’re applying different transaction monitoring rules to different customers, are you missing suspicious activity by low-risk customers, or drowning in unhelpful noise from high-risk customers?

‍

Performance of Controls

Most assurance processes involve confirming if controls are being implemented as expected. But not all consider if the controls are actually performing. Are they doing what is expected, and are the right outcomes being generated, for the right reasons?

‍

Did the outcome that you expected to happen occur because of thething that you expected to trigger it?

For instance, did suspicious activity that you deem connected to money laundering get flagged by transaction monitoring rules looking for money laundering, rather than fraud or terrorist financing? If you amend your transaction monitoring rules to reduce unnecessary false positives, or conversely to capture more activity, you need to measure whether the number of alerts has gone up or down as expected. And at a deeper level, you need to measure whether the activity now triggering alerts is relevant - i.e. has an increase in the number of alerts highlighted your awareness of suspicious activity, resulting in more investigations and more SARs? Has a reduction in the number of alerts led to fewer investigations and SARs, meaning you’re not just cutting out noise but also suspicious activity?

‍

Unknown Unknowns

An additional challenge is measuring not just what you are doing, but also what you’re not doing. Your controls may be working well, but are there other controls which you lack which you can’t even begin to measure? Are there risks that you are experiencing that you don’t know about?

‍

Fortunately, there are ways to assess your program holistically to identify where there might be gaps. For instance, if transaction monitoring regularly flags fraudulent activity, consider how other elements of your program address fraud, to prevent such activity happening in the first place. Can you refine your customer risk assessment to better identify fraud indicators? Can you invest in new tools or integrate additional data feeds to calculate fraud scores? Gaps may appear over time, so you need to continuously monitor what your assurance process is showing you. For instance, if you start filing fewer fraud-related SARs, this could be explained by changes to KYC or monitoring controls, or by the fact you’ve hired a new team of analysts, who need more training on fraud typologies.

‍

Keeping on top of industry trends and typologies will also help you understand the unknown unknowns. Unfortunately financial criminals are always coming up with new methods, so your financial crime controls need to be updated just as frequently.

‍

Financial Crime

Finally, we come to a paradox. The metric which should be the most important and the most integral is also the one which is hardest to assess.

‍

The aim of fincrime programs, the reason we’re all here, is to stop financial crime. So are we? And if so, how much?

If we can’t tell, how can we possibly assess whether what we’re doing is working? The good news is, while it’s hard to accurately calculate comprehensive figures, there are ways we can assess progress by looking at whether controls are becoming more (or less) effective. SARs are a key indicator, e.g. the number of SARs filed over time, the number filed compared to your peers, the value of the transactions covered by SARs, whether your SARs cover all known typologies, and the number of SARs triggered by internal alerts vs. those triggered by intelligence from law enforcement or other financial institutions. Fraud losses are another good metric - e.g. the amount of money identified as the proceeds of fraud which you stop before it leaves an account, vs. the amounts reported which you do not stop.

‍

How to Measure

Having looked at what to measure, let’s think about how.

‍

Manual Assurance

Traditionally, this has been an intensive manual exercise. Regular testing generally comprises financial crime compliance analysts performing monthly or quarterly dip testing, supplemented by annual testing by the third line of defense or an external party.

‍

Dip testing involves reviewing a certain number of samples of a prescribed list of activities (e.g. KYC files, screening hits, transaction monitoring alerts, SAR filings, etc.). Sample sizes can be chosen arbitrarily or calculated using complex statistical measures. Samples may be entirely random, or stratified - e.g. 60 high-risk customers files, 30 medium-risk customer files, and 10 low-risk customer files. They are reviewed to confirm if the correct procedures were followed; for instance, whether all the correct pieces of information were collected at onboarding, or whether SARs contained the right information presented in the appropriate manner. This generates numerical outputs (“what percentage of KYC files contain all the required data?”), ideally supplemented by narrative on common issues or shortcomings.

‍

Manual testing is the most common way of evaluating fincrime controls, and it has a few advantages. It can be scaled up or down in response to available resources (although clearly scaling down decreases the quality and reliability of the output). It can focus on high-risk controls or customers, in line with a risk-based approach, and it is effective at identifying clear-cut issues. It ultimately asks straightforward binary questions about surface components of the program (“were all PEPs escalated per the policy - yes/no”), so the outputs are easy to understand and failings are easy to define.

‍

However, there are clearly serious problems with this approach:

‍

1. Sample-based testing means there’s a chance of missing things, as it doesn’t offer 100% coverage
2. It is backwards-looking, so can only identify issues which started in the past; it cannot identify issues in real time. This means once an issue is identified, a remediation project is often needed to go back through ALL the client files, transaction monitoring alerts, etc. to identify other instances when the process did not work or procedures were not followed. This can be a massive exercise.
3. Given its partial nature, dip testing doesn’t reveal the extent of a problem. If 5% of your sample fails the test, how confident are you of a 5% failure rate across all your records? Has the problem started recently, has it improved over the past year, or is it a continuous long-term issue? Based on this, what resources will you need to remediate and fix it?
4. This lack of information makes stakeholder engagement with senior management, partner institutions and the regulator very difficult.
5. As you grow, you need to either grow your headcount to maintain sampling levels, or reduce sample numbers and therefore assurance over your financial crime program.

‍Automated Assurance

So, if dip testing is not the answer, what is?

‍

Rather than a manual process, you can adopt an automated one, which fundamentally changes and upgrades the nature of your assurance process.

There are clear benefits of an automated approach:

‍

1. It can test and measure 100% of accounts and activity. This means you will never fail to identify an issue, and will always know the full extent of any failings that arise.
2. As automated testing runs continuously, you become aware of problems as soon as they emerge. Being able to identify issues in real time is a gamechanger. It enables you to fix problems before they have too much of an impact, limiting your exposure to financial crime threats. It also means you can avoid huge, expensive remediation exercises. Rather than a manual process, you can adopt an automated one, which fundamentally changes and upgrades the nature of your assurance process.
3. Automated testing can link different components of your program together, meaning if weaknesses emerge in one of your processes, or you introduce improvements, you can immediately see if there is a knock-on effect elsewhere.
4. The ability to react immediately and provide clarity around the scale of a problem can transform stakeholder management. If you can tell a regulator exactly how substantial an issue is, when it began, and how long you’ll need to fix it, you’ll be in a much stronger position and should be able to reduce the risk (and cost) of any regulatory fines.
5. There are savings benefits to introducing an automated process, as well. You no longer need a team of analysts performing time-intensive dip testing, and can instead redeploy your valuable second-line resources to higher value areas like in-depth investigations of suspicious accounts, or improving your controls.
6. You don’t need to grow the assurance team as the business expands; the tech will continue to offer 100% coverage, so you can scale with confidence.

Automation can also help ensure consistency, another topic regulators are super keen on. This applies both internally, across different pools of analysts and over time, and across programs for sponsor banks and Banking-as-a-Service providers. Using an automated solution means you can monitor the controls of every program in your portfolio, giving you complete oversight and total coverage even as the number of customer programs grows.

‍

Conclusion

There may not yet be a universal definition of financial crime controls effectiveness or specific regulatory instructions, but now you hopefully know that this needn’t stop you introducing a robust process to measure and understand your firm’s effectiveness.

‍

The benefits of automating this process should also be evident; ensuring you have 100% coverage, understand the breadth and depth of any issues, and catching problems in real-time. This is the next step in the evolution of financial crime. The first wave of automation targeted the operations of the first line of defense (KYC, screening, transaction monitoring etc.). The next leap forward will be harnessing the power of technology to revolutionize assurance and enable financial institutions to understand and evidence their financial crime controls effectiveness.

‍

Download the full whitepaper