Natasha Vernier
Oct 28, 2024

Will AI + Automated Testing Eliminate the Third Line of Defense?

 

Anthropic just announced a new public beta feature for its Claude AI, “computer use”, which means Claude can use computers the way people do – moving cursors, looking at screens, typing text, etc. Put another way, AI agents will soon operate across multiple screens, move data between applications, make comparisons, fill out forms, and more. This advancement drastically increases AI agents’ ability to handle more banking and compliance tasks, such as identity verification, screening, form completion and data movement. 

Claude’s computer use feature is still in its early stages, but it has potentially huge ramifications for the banking industry including the traditional 3LOD model.

A Brief Overview of the 3LOD

The 3LOD framework was developed in the late 1990s and early 2000s as a way for financial institutions to mitigate risk in certain processes.

As the name suggests, there are 3 layers of oversight:

 

The 3LOD model is therefore all about people testing whether other people did their work accurately and effectively - “checking the checkers”. As such, one of the biggest issues with the 3LOD is that it relies on employees to actively monitor and assess risks and compliance controls. Not only are people highly fallible when it comes to repetitive tasks, these kinds of manual processes tend to be painfully slow.

How AI and Automated Testing Fits Into the 3LOD

With AI now able to reasonably complete a significant amount of the work in the 1LOD, and automated testing becoming standard for the 2LOD, are we going to see a shift away from a 3LOD model to a 2LOD model?

AI for 1LOD

AI can assume the work of the 1LOD because it’s much better at predictive analysis. AI can interpret large amounts of data and recognize patterns to more efficiently, accurately and effectively detect fraud, assess credit worthiness, conduct compliance checks, and monitor transactions in real-time. We’re already seeing this starting to become standard with companies like Greenlite

Automated Compliance Testing for 2LOD

Traditional dip sampling methods of compliance testing are already not cutting it, and that will become even more true with AI in the 1LOD. With humans doing less of the 1LOD work, sample sizes will need to increase, and with computers able to perform the 1LOD work more efficiently, the corresponding volume of 1LOD work to review will also increase. Therefore the only way to confidently ensure compliance is to automate the testing of your compliance controls. 

Tools like Cable’s Automated Assurance and Transaction Assurance provide 100% testing coverage of your compliance controls, 24/7 – without the need for additional manpower. Regulatory breaches and control failures are immediately flagged, so you can be proactive in fixing any issues. 

And while AI is perfectly suited for the more predictive work in the 1LOD, we definitely want to avoid the scenario where AI in the 2LOD tests AI in the 1LOD. Humans have built our biases into AI and it’s extremely difficult to completely eliminate that bias. If we used AI to test the effectiveness of another AI, it likely wouldn’t catch any deficiencies because it’s operating on the same fallacies. Furthermore, and unlike in the 1LOD, automated compliance testing is very binary - was a regulation or policy complied with, yes or no? AI is not needed in order to determine a binary output. 

Will the Future be Two Lines of Defense?

Together, AI and automated testing are powerful tools for enabling growth, reducing costs, and ensuring your financial institution remains safe and compliant.

AI supplements and enhances human capabilities in pattern recognition and analysis by integrating with disparate systems originally designed for human operation. This allows the 1LOD to scale with fewer resources. With automated testing as part of the 2LOD, it provides complete, independent coverage of the actions and data flowing through financial systems, ensuring that any issues are quickly caught and corrected.


In summary, the coverage of our testing surface has the potential to expand exponentially. With expanding coverage, the 3LOD, whose job it is to catch things that fall through the cracks because the 1LOD and the 2LOD are unable to test everything, becomes redundant. You no longer need to check the checkers. 

 

So in a world where AI is prevalent in the 1LOD and automated testing the standard in the 2LOD, there may not be significant benefit from the 3LOD.
When it comes to risk management and compliance, the financial industry has long relied on the “three lines of defense” (3LOD) model. Could that be about to change?

Recent Posts

Compliance Team Resources
Why Fintechs Need 24/7 Automated Compliance Testing
By
Natasha Vernier
.
November 26, 2024
Company News
Expanding Compliance Testing: New Modules for Reg E, Reg B, and Reg Z
By
Natasha Vernier
.
November 22, 2024

 

Anthropic just announced a new public beta feature for its Claude AI, “computer use”, which means Claude can use computers the way people do – moving cursors, looking at screens, typing text, etc. Put another way, AI agents will soon operate across multiple screens, move data between applications, make comparisons, fill out forms, and more. This advancement drastically increases AI agents’ ability to handle more banking and compliance tasks, such as identity verification, screening, form completion and data movement. 

Claude’s computer use feature is still in its early stages, but it has potentially huge ramifications for the banking industry including the traditional 3LOD model.

A Brief Overview of the 3LOD

The 3LOD framework was developed in the late 1990s and early 2000s as a way for financial institutions to mitigate risk in certain processes.

As the name suggests, there are 3 layers of oversight:

  • 1LOD: Operational Management: Usually, the first line of defense are front-line employees and management. They’re tasked with identifying risks, and designing and implementing controls.
  • 2LOD: Risk Management and Compliance: Compliance and risk management teams form the second line of defense. They provide internal oversight and testing to ensure that controls are working.
  • 3LOD: Independent Audit: Oftentimes, a financial institution will rely on their internal audit team or hire an independent auditor to conduct an audit to test the effectiveness of the organization’s controls implemented by 1LOD and 2LOD. These auditors report their findings to the institution’s board and senior management.

 

The 3LOD model is therefore all about people testing whether other people did their work accurately and effectively - “checking the checkers”. As such, one of the biggest issues with the 3LOD is that it relies on employees to actively monitor and assess risks and compliance controls. Not only are people highly fallible when it comes to repetitive tasks, these kinds of manual processes tend to be painfully slow.

How AI and Automated Testing Fits Into the 3LOD

With AI now able to reasonably complete a significant amount of the work in the 1LOD, and automated testing becoming standard for the 2LOD, are we going to see a shift away from a 3LOD model to a 2LOD model?

AI for 1LOD

AI can assume the work of the 1LOD because it’s much better at predictive analysis. AI can interpret large amounts of data and recognize patterns to more efficiently, accurately and effectively detect fraud, assess credit worthiness, conduct compliance checks, and monitor transactions in real-time. We’re already seeing this starting to become standard with companies like Greenlite

Automated Compliance Testing for 2LOD

Traditional dip sampling methods of compliance testing are already not cutting it, and that will become even more true with AI in the 1LOD. With humans doing less of the 1LOD work, sample sizes will need to increase, and with computers able to perform the 1LOD work more efficiently, the corresponding volume of 1LOD work to review will also increase. Therefore the only way to confidently ensure compliance is to automate the testing of your compliance controls. 

Tools like Cable’s Automated Assurance and Transaction Assurance provide 100% testing coverage of your compliance controls, 24/7 – without the need for additional manpower. Regulatory breaches and control failures are immediately flagged, so you can be proactive in fixing any issues. 

And while AI is perfectly suited for the more predictive work in the 1LOD, we definitely want to avoid the scenario where AI in the 2LOD tests AI in the 1LOD. Humans have built our biases into AI and it’s extremely difficult to completely eliminate that bias. If we used AI to test the effectiveness of another AI, it likely wouldn’t catch any deficiencies because it’s operating on the same fallacies. Furthermore, and unlike in the 1LOD, automated compliance testing is very binary - was a regulation or policy complied with, yes or no? AI is not needed in order to determine a binary output. 

Will the Future be Two Lines of Defense?

Together, AI and automated testing are powerful tools for enabling growth, reducing costs, and ensuring your financial institution remains safe and compliant.

AI supplements and enhances human capabilities in pattern recognition and analysis by integrating with disparate systems originally designed for human operation. This allows the 1LOD to scale with fewer resources. With automated testing as part of the 2LOD, it provides complete, independent coverage of the actions and data flowing through financial systems, ensuring that any issues are quickly caught and corrected.


In summary, the coverage of our testing surface has the potential to expand exponentially. With expanding coverage, the 3LOD, whose job it is to catch things that fall through the cracks because the 1LOD and the 2LOD are unable to test everything, becomes redundant. You no longer need to check the checkers. 

 

So in a world where AI is prevalent in the 1LOD and automated testing the standard in the 2LOD, there may not be significant benefit from the 3LOD.
When it comes to risk management and compliance, the financial industry has long relied on the “three lines of defense” (3LOD) model. Could that be about to change?

There’s more to read!

Resources