Anything less than full monitoring of controls leaves room for error - finally you can do away with manual dip sampling. Cable provides automated evidence of your compliance, risk management and effectiveness, allowing you to:
Why manually test 100 accounts when you can automatically monitor 100%?
Regulators, international standard setters and private sector groups have all started to talk about prioritising financial crime effectiveness over technical compliance. There is growing momentum as more people come to the realisation that financial institutions must be able to prove that what they’re doing is not just legally compliant, but is actually working to reduce financial crime.
The overall message from these organisations is clear, and it is only a matter of time before firms around the world are obliged to demonstrate the effectiveness of their financial crime controls.
Whilst there are clear financial benefits to measuring and evidencing financial crime effectiveness, how to do so remains unclear.
This series provides a deep dive into the world of financial crime effectiveness, covering:
If you’ve read the first two parts in this Series, you now understand the importance of assessing and understanding the effectiveness of your financial crime controls. It’s good for financial institutions, because they can eliminate ineffective controls, reduce costs and improve stakeholder communication. It’s good for regulators, because it gives them a better understanding of whether firms are just ticking boxes or actually managing their risks. And it’s good for society, because it helps us understand how much financial crime we are stopping, and tells us how to stop more.
Regulators are increasingly promoting the concept of effectiveness. It’s only a matter of time before this is embedded in regulations themselves, with firms obliged to prove to the regulators that not only do they have controls in place, but that those controls actually work.
Firms can frequently find themselves playing catch-up with regulatory developments and changing systems or processes which they’ve only recently implemented or reviewed. So when regulators signpost a change in direction, as they are now, there are clear advantages to getting out in front of those changes.
But, how do you measure the effectiveness of your financial crime controls?
It’s all very well to say that firms should measure effectiveness, but what does that actually mean? What specifically should you be measuring, and what metrics and data will tell you if your program is effective?
Regulatory Compliance
The basic starting point is establishing whether your program is compliant with regulatory requirements. If not, you’re in trouble.
You need to ensure that you’re collecting all the mandatory KYC information, all customers and payments are correctly screened, and all appropriate actions are taken when you identify suspicious activity. The question you’re asking here is essentially a binary one - are we meeting our regulatory obligations, yes or no? Provided you understand your obligations and have access to the relevant data and information about your program, you should be able to answer this question.
Internal Control Compliance
As well as regulatory compliance, you need to ensure your financial crime program complies with your internal policies and procedures. If you commit to implementing certain controls, you must adhere to those controls. The basis of any good program is a clear risk appetite, so this is a good place to start. You should confirm your risk appetite is being met through both prohibitions such as “we won’t onboard customers in certain sectors” as well as limits such as “no more than 5% of our customers will be high-risk”. It’s relatively simple to tell what ‘good’ looks like; you established a defined risk appetite statement, so it is important to be sticking to it.
Adherence to your Risk Based Approach
A cornerstone of all financial crime programs is the adoption of a risk-based approach, so you need to confirm this approach is being implemented in practice. Are you correctly applying different levels of KYC to different types of customers? Are you conducting periodic reviews at the correct intervals? Are higher-risk customers being escalated for senior approval?
As well as confirming if you’re implementing a risk-based approach, you should consider whether it’s giving you the results you expect.
Are you classifying the right customers as high-risk, or do you actually identify more suspicious activity amongst low or medium-risk customers? If you’re applying different transaction monitoring rules to different customers, are you missing suspicious activity by low-risk customers, or drowning in unhelpful noise from high-risk customers?
Performance of Controls
Most assurance processes involve confirming if controls are being implemented as expected. But not all consider if the controls are actually performing. Are they doing what is expected, and are the right outcomes being generated, for the right reasons?
For instance, did suspicious activity that you deem connected to money laundering get flagged by transaction monitoring rules looking for money laundering, rather than fraud or terrorist financing? If you amend your transaction monitoring rules to reduce unnecessary false positives, or conversely to capture more activity, you need to measure whether the number of alerts has gone up or down as expected. And at a deeper level, you need to measure whether the activity now triggering alerts is relevant - i.e. has an increase in the number of alerts highlighted your awareness of suspicious activity, resulting in more investigations and more SARs? Has a reduction in the number of alerts led to fewer investigations and SARs, meaning you’re not just cutting out noise but also suspicious activity?
Unknown Unknowns
An additional challenge is measuring not just what you are doing, but also what you’re not doing. Your controls may be working well, but are there other controls which you lack which you can’t even begin to measure? Are there risks that you are experiencing that you don’t know about?
Fortunately, there are ways to assess your program holistically to identify where there might be gaps. For instance, if transaction monitoring regularly flags fraudulent activity, consider how other elements of your program address fraud, to prevent such activity happening in the first place. Can you refine your customer risk assessment to better identify fraud indicators? Can you invest in new tools or integrate additional data feeds to calculate fraud scores? Gaps may appear over time, so you need to continuously monitor what your assurance process is showing you. For instance, if you start filing fewer fraud-related SARs, this could be explained by changes to KYC or monitoring controls, or by the fact you’ve hired a new team of analysts, who need more training on fraud typologies.
Keeping on top of industry trends and typologies will also help you understand the unknown unknowns. Unfortunately financial criminals are always coming up with new methods, so your financial crime controls need to be updated just as frequently.
Financial Crime
Finally, we come to a paradox. The metric which should be the most important and the most integral is also the one which is hardest to assess.
If we can’t tell, how can we possibly assess whether what we’re doing is working? The good news is, while it’s hard to accurately calculate comprehensive figures, there are ways we can assess progress by looking at whether controls are becoming more (or less) effective. SARs are a key indicator, e.g. the number of SARs filed over time, the number filed compared to your peers, the value of the transactions covered by SARs, whether your SARs cover all known typologies, and the number of SARs triggered by internal alerts vs. those triggered by intelligence from law enforcement or other financial institutions. Fraud losses are another good metric - e.g. the amount of money identified as the proceeds of fraud which you stop before it leaves an account, vs. the amounts reported which you do not stop.
Having looked at what to measure, let’s think about how.
Manual Assurance
Traditionally, this has been an intensive manual exercise. Regular testing generally comprises financial crime compliance analysts performing monthly or quarterly dip testing, supplemented by annual testing by the third line of defense or an external party.
Dip testing involves reviewing a certain number of samples of a prescribed list of activities (e.g. KYC files, screening hits, transaction monitoring alerts, SAR filings, etc.). Sample sizes can be chosen arbitrarily or calculated using complex statistical measures. Samples may be entirely random, or stratified - e.g. 60 high-risk customers files, 30 medium-risk customer files, and 10 low-risk customer files. They are reviewed to confirm if the correct procedures were followed; for instance, whether all the correct pieces of information were collected at onboarding, or whether SARs contained the right information presented in the appropriate manner. This generates numerical outputs (“what percentage of KYC files contain all the required data?”), ideally supplemented by narrative on common issues or shortcomings.
Manual testing is the most common way of evaluating fincrime controls, and it has a few advantages. It can be scaled up or down in response to available resources (although clearly scaling down decreases the quality and reliability of the output). It can focus on high-risk controls or customers, in line with a risk-based approach, and it is effective at identifying clear-cut issues. It ultimately asks straightforward binary questions about surface components of the program (“were all PEPs escalated per the policy - yes/no”), so the outputs are easy to understand and failings are easy to define.
However, there are clearly serious problems with this approach:
Automated Assurance
So, if dip testing is not the answer, what is?
There are clear benefits of an automated approach:
Automation can also help ensure consistency, another topic regulators are super keen on. This applies both internally, across different pools of analysts and over time, and across programs for sponsor banks and Banking-as-a-Service providers. Using an automated solution means you can monitor the controls of every program in your portfolio, giving you complete oversight and total coverage even as the number of customer programs grows.
There may not yet be a universal definition of financial crime controls effectiveness or specific regulatory instructions, but now you hopefully know that this needn’t stop you introducing a robust process to measure and understand your firm’s effectiveness.
The benefits of automating this process should also be evident; ensuring you have 100% coverage, understand the breadth and depth of any issues, and catching problems in real-time. This is the next step in the evolution of financial crime. The first wave of automation targeted the operations of the first line of defense (KYC, screening, transaction monitoring etc.). The next leap forward will be harnessing the power of technology to revolutionize assurance and enable financial institutions to understand and evidence their financial crime controls effectiveness.
Anything less than full monitoring of controls leaves room for error - finally you can do away with manual dip sampling. Cable provides automated evidence of your compliance, risk management and effectiveness, allowing you to:
Why manually test 100 accounts when you can automatically monitor 100%?
Regulators, international standard setters and private sector groups have all started to talk about prioritising financial crime effectiveness over technical compliance. There is growing momentum as more people come to the realisation that financial institutions must be able to prove that what they’re doing is not just legally compliant, but is actually working to reduce financial crime.
The overall message from these organisations is clear, and it is only a matter of time before firms around the world are obliged to demonstrate the effectiveness of their financial crime controls.
Whilst there are clear financial benefits to measuring and evidencing financial crime effectiveness, how to do so remains unclear.
This series provides a deep dive into the world of financial crime effectiveness, covering:
If you’ve read the first two parts in this Series, you now understand the importance of assessing and understanding the effectiveness of your financial crime controls. It’s good for financial institutions, because they can eliminate ineffective controls, reduce costs and improve stakeholder communication. It’s good for regulators, because it gives them a better understanding of whether firms are just ticking boxes or actually managing their risks. And it’s good for society, because it helps us understand how much financial crime we are stopping, and tells us how to stop more.
Regulators are increasingly promoting the concept of effectiveness. It’s only a matter of time before this is embedded in regulations themselves, with firms obliged to prove to the regulators that not only do they have controls in place, but that those controls actually work.
Firms can frequently find themselves playing catch-up with regulatory developments and changing systems or processes which they’ve only recently implemented or reviewed. So when regulators signpost a change in direction, as they are now, there are clear advantages to getting out in front of those changes.
But, how do you measure the effectiveness of your financial crime controls?
It’s all very well to say that firms should measure effectiveness, but what does that actually mean? What specifically should you be measuring, and what metrics and data will tell you if your program is effective?
Regulatory Compliance
The basic starting point is establishing whether your program is compliant with regulatory requirements. If not, you’re in trouble.
You need to ensure that you’re collecting all the mandatory KYC information, all customers and payments are correctly screened, and all appropriate actions are taken when you identify suspicious activity. The question you’re asking here is essentially a binary one - are we meeting our regulatory obligations, yes or no? Provided you understand your obligations and have access to the relevant data and information about your program, you should be able to answer this question.
Internal Control Compliance
As well as regulatory compliance, you need to ensure your financial crime program complies with your internal policies and procedures. If you commit to implementing certain controls, you must adhere to those controls. The basis of any good program is a clear risk appetite, so this is a good place to start. You should confirm your risk appetite is being met through both prohibitions such as “we won’t onboard customers in certain sectors” as well as limits such as “no more than 5% of our customers will be high-risk”. It’s relatively simple to tell what ‘good’ looks like; you established a defined risk appetite statement, so it is important to be sticking to it.
Adherence to your Risk Based Approach
A cornerstone of all financial crime programs is the adoption of a risk-based approach, so you need to confirm this approach is being implemented in practice. Are you correctly applying different levels of KYC to different types of customers? Are you conducting periodic reviews at the correct intervals? Are higher-risk customers being escalated for senior approval?
As well as confirming if you’re implementing a risk-based approach, you should consider whether it’s giving you the results you expect.
Are you classifying the right customers as high-risk, or do you actually identify more suspicious activity amongst low or medium-risk customers? If you’re applying different transaction monitoring rules to different customers, are you missing suspicious activity by low-risk customers, or drowning in unhelpful noise from high-risk customers?
Performance of Controls
Most assurance processes involve confirming if controls are being implemented as expected. But not all consider if the controls are actually performing. Are they doing what is expected, and are the right outcomes being generated, for the right reasons?
For instance, did suspicious activity that you deem connected to money laundering get flagged by transaction monitoring rules looking for money laundering, rather than fraud or terrorist financing? If you amend your transaction monitoring rules to reduce unnecessary false positives, or conversely to capture more activity, you need to measure whether the number of alerts has gone up or down as expected. And at a deeper level, you need to measure whether the activity now triggering alerts is relevant - i.e. has an increase in the number of alerts highlighted your awareness of suspicious activity, resulting in more investigations and more SARs? Has a reduction in the number of alerts led to fewer investigations and SARs, meaning you’re not just cutting out noise but also suspicious activity?
Unknown Unknowns
An additional challenge is measuring not just what you are doing, but also what you’re not doing. Your controls may be working well, but are there other controls which you lack which you can’t even begin to measure? Are there risks that you are experiencing that you don’t know about?
Fortunately, there are ways to assess your program holistically to identify where there might be gaps. For instance, if transaction monitoring regularly flags fraudulent activity, consider how other elements of your program address fraud, to prevent such activity happening in the first place. Can you refine your customer risk assessment to better identify fraud indicators? Can you invest in new tools or integrate additional data feeds to calculate fraud scores? Gaps may appear over time, so you need to continuously monitor what your assurance process is showing you. For instance, if you start filing fewer fraud-related SARs, this could be explained by changes to KYC or monitoring controls, or by the fact you’ve hired a new team of analysts, who need more training on fraud typologies.
Keeping on top of industry trends and typologies will also help you understand the unknown unknowns. Unfortunately financial criminals are always coming up with new methods, so your financial crime controls need to be updated just as frequently.
Financial Crime
Finally, we come to a paradox. The metric which should be the most important and the most integral is also the one which is hardest to assess.
If we can’t tell, how can we possibly assess whether what we’re doing is working? The good news is, while it’s hard to accurately calculate comprehensive figures, there are ways we can assess progress by looking at whether controls are becoming more (or less) effective. SARs are a key indicator, e.g. the number of SARs filed over time, the number filed compared to your peers, the value of the transactions covered by SARs, whether your SARs cover all known typologies, and the number of SARs triggered by internal alerts vs. those triggered by intelligence from law enforcement or other financial institutions. Fraud losses are another good metric - e.g. the amount of money identified as the proceeds of fraud which you stop before it leaves an account, vs. the amounts reported which you do not stop.
Having looked at what to measure, let’s think about how.
Manual Assurance
Traditionally, this has been an intensive manual exercise. Regular testing generally comprises financial crime compliance analysts performing monthly or quarterly dip testing, supplemented by annual testing by the third line of defense or an external party.
Dip testing involves reviewing a certain number of samples of a prescribed list of activities (e.g. KYC files, screening hits, transaction monitoring alerts, SAR filings, etc.). Sample sizes can be chosen arbitrarily or calculated using complex statistical measures. Samples may be entirely random, or stratified - e.g. 60 high-risk customers files, 30 medium-risk customer files, and 10 low-risk customer files. They are reviewed to confirm if the correct procedures were followed; for instance, whether all the correct pieces of information were collected at onboarding, or whether SARs contained the right information presented in the appropriate manner. This generates numerical outputs (“what percentage of KYC files contain all the required data?”), ideally supplemented by narrative on common issues or shortcomings.
Manual testing is the most common way of evaluating fincrime controls, and it has a few advantages. It can be scaled up or down in response to available resources (although clearly scaling down decreases the quality and reliability of the output). It can focus on high-risk controls or customers, in line with a risk-based approach, and it is effective at identifying clear-cut issues. It ultimately asks straightforward binary questions about surface components of the program (“were all PEPs escalated per the policy - yes/no”), so the outputs are easy to understand and failings are easy to define.
However, there are clearly serious problems with this approach:
Automated Assurance
So, if dip testing is not the answer, what is?
There are clear benefits of an automated approach:
Automation can also help ensure consistency, another topic regulators are super keen on. This applies both internally, across different pools of analysts and over time, and across programs for sponsor banks and Banking-as-a-Service providers. Using an automated solution means you can monitor the controls of every program in your portfolio, giving you complete oversight and total coverage even as the number of customer programs grows.
There may not yet be a universal definition of financial crime controls effectiveness or specific regulatory instructions, but now you hopefully know that this needn’t stop you introducing a robust process to measure and understand your firm’s effectiveness.
The benefits of automating this process should also be evident; ensuring you have 100% coverage, understand the breadth and depth of any issues, and catching problems in real-time. This is the next step in the evolution of financial crime. The first wave of automation targeted the operations of the first line of defense (KYC, screening, transaction monitoring etc.). The next leap forward will be harnessing the power of technology to revolutionize assurance and enable financial institutions to understand and evidence their financial crime controls effectiveness.
