SR 21-8, SR 11-7 and NYDFS Part 504: How a Top US Bank Achieved Best Practice Baseline Model Validation of Its Anti-Money Laundering Model
With Anti Money Laundering (AML) systems being classified as a model falling under the purview of SR 11-7 requirements, banks are required to institute model governance and validation processes independently and in addition to the testing mandated by DFS Part 504. A top US Bank was searching for a partner who could conduct a baseline validation as per SR 11-7 requirements.
Our Model Risk Management (MRM) and anti money laundering teams performed a detailed review of the fragmented documentation on the model system, which included Business Requirement Documents, technical material from the vendor, and user guides, among others. We also had discussions with the client’s legal & compliance team and their IT model implementation teams. This enabled us to develop a comprehensive validation work plan that took conflicts and overlaps between BSA/ AML regulations and interagency guidelines under SR 11-7 into consideration.
The validation work plan was executed to provide the model risk assessments in the following focus areas:
• Input Data Elements: assessing the appropriateness and integrity of the input data used in the model by performing data lineage and data quality testing.
• Conceptual Soundness and Effective Challenge: assessing the soundness and appropriateness of the anti money laundering rules/scenarios, model limitations, and assumptions in consultation with our team of AML Experts. This was done in keeping with the scope of product and services categories offered to the institutional businesses and corporate clients.
• Model Implementation: verifying the accuracy and completeness of model implementation by replicating scenarios and test cases during a sample testing period using our independently coded SQL testing modules
• Outcome Analysis: validating the appropriateness of $ thresholds set for event-based rules by performing distribution and sensitivity analysis (Above the line and Below line testing). The Performance and Tuning Calibration testing also included the model operational performance such as SAR yield analysis.
• Ongoing Model Governance: assessing the internal model governance framework and change management procedures.
• Model Documentation: assessing the quality and completeness of model documentation. The primary objective was to provide a clear view of the model’s processes in our validation report in line with SR 11-7 requirements for third-party models. Our description included the documentation of complex model architecture, upstream source systems providing inputs on transaction/customer data and pre-processing activities, and rules for filtering out the suspicious transactions.
Model owners were made aware of critical gaps in model documentation, implementation, and governance including gaps in the production environment for certain high-risk rules associated with sensitive jurisdictions. A time-bound plan was agreed with the model owners to plug gaps in model implementation and documentation pertaining to technical specifications and data transformations. The ongoing monitoring framework was enhanced to conduct back-testing on a sample basis on unfiltered transactions to establish true negatives.
Reduced Turnaround Times for Change-based Validation Based on Re-usable SQL-based testing framework
allowing the bank to swiftly implement change to rules, keywords, and thresholds to rapidly adapt to the evolving threat environment.
Helped Model Owners Identify Areas for Process Improvements,
which among others, included resolution time for past due cases and non-compliance with the SAR filing process.
Enabled Model Owners to Improve Change Management Procedures
through recommendations to consolidate model information scattered over multiple documents into a single master document.