Federal banking regulators are set to release new guidelines Monday that they hope will restore the battered reputation of risk models — widely disparaged as a result of the financial crisis — by forcing banks to improve how they design and implement them.
Although previous regulatory guidance had focused mostly on validating whether models worked properly, the Office of the Comptroller of the Currency (OCC) and the Federal Reserve Board are expected to release a 21-page bulletin detailing extra steps that banks will be required to take to ensure models are being implemented correctly and have proper attention from a bank's management and board of directors.
Under the bulletin, a copy of which was obtained by American Banker, banks must ensure models have a strong governance framework, beef up staffing and resources devoted to risk modeling management, as well as conduct internal audits and ensure comprehensive documentation.
Risk models have been used for years, and many did not adequately predict losses in a crisis scenario. But regulators say the problem was not that the models failed, but that they were misused.
"We had this sense that the models didn't work, but that's like saying, 'We have car accidents, so if we just all walked instead of drove cars, then we wouldn't have car accidents,' " Mark Levonian, senior deputy comptroller for economics for the OCC, said in an interview.
"Well, yeah, but we wouldn't want to live without cars," he said. "It's kind of the same thing here. … It's this emphasis that not only do you have to make sure the model works, but you have to look at how that model is being used in your business, in your banks, and make sure in that use what risks it's creating and how you are controlling those risks and who's responsible for it."
The new bulletin replaces May 2000 guidance by the OCC that largely focused on validation of models, a process that dealt mostly with the soundness of the math and assumptions involved but did not look at the broader context of how banks were using them.
Although the OCC and Fed began working separately on updated guidance in 2009, they eventually combined efforts. The agencies hope the Federal Deposit Insurance Corp. (FDIC) will consider issuing the guidelines to its own banks.
The updated guidelines emphasize that model developers should use rigorous assessment of data quality and relevance and provide appropriate documentation followed by testing to ensure the right models are used for the right products.
Levonian said many models blamed for failing during the crisis worked fine, but were being used to test the wrong products.
"They did what they were designed to do," he said. "The problem was they were being used for things that they weren't designed to be used for. An example would be some of the problems with structured mortgage credit. People were using models that were fine models, but they were developed for corporate bonds."
Under the previous regulatory guidance of validation, "the models were fine," he said.
"The models worked," Levonian said. But "if you looked at it in a risk management framework, somebody said, 'Let's use a model that wasn't designed for this purpose for this other purpose.' That's the kind of thing good governance should pick up."
The new guidelines emphasize that point.
"Developers should ensure that the components work as intended, are appropriate for the intended business purposes, and are conceptually sound and mathematically and statistically correct," the guidelines say. "But it is not enough for model developers and user to understand and accept the model. Because model risk is ultimately borne by the bank as a whole, the bank should objectively assess model risk and associated costs and benefits using a sound model-validation process."
Once implemented, the guidelines will force many banks to beef up staffing to address model risk assessment. The bulletin requires banks to have a model owner responsible for developing, running it and updating it; a risk control person or group to look at the risk around that model; and a compliance officer to oversee the other two roles. It orders banks to conduct an annual review of models and develop a process for regular monitoring. In addition, a model-validation framework should include evaluation of conceptual soundness and outcomes analysis.
Additionally, the guidelines ask banks to review vender and third-party products as part of broader risk management. Institutions would need to develop and maintain strong governance policies and controls over model risk framework, including oversight from a bank's board and senior management. Banks also must conduct internal audits on their models and maintain a documentation inventory.
Levonian said the industry is already moving toward such risk management changes.
"It varies across the large banks," he said. "There are some that have really strong practices. Others will have some work to do. I don't think any of them will see things in here that they look at and are surprised. … I think the industry is going to see this as something that, yes, they have some work to do, but it's the direction they were going anyway because it makes sense."
The guidelines also emphasize a number of the requirements should already be in place.
"Many of the activities described in this document are common industry practice," the guidelines say. "But all banks should confirm that their practices conform to their principles in this guidance for model development, implementation, and use, as well as model validation. Banks should also ensure that they maintain strong governance and controls to help manage model risk, including internal policies and procedures that appropriately reflect the risk management principles described in this guidance."
Levonian acknowledged, however, that the guidelines will require updating resources and staff for many institutions. Post-Dodd-Frank, banks have complained that they are already overburdened with regulatory requirements. Levonian said banks should tailor the guidance to their complexity, reducing the burden on community banks or institutions that are less complex. "We've seen some problems that need to be fixed and that might require more resources, but in the other direction, one of the things we emphasize in this guidance is treat this like risk management," he said. "What you have to do depends on the nature of the risk. With these models, it's how complex are they. How material is it to your business? If it's not too complex and not that material then you don't have to do that much. But as a matter of good risk management, if they are complicated and important to your bottom line then, yeah, you better be putting resources into it."