An endgame to the foreclosure crisis is emerging and it centers on Fannie Mae and Freddie Mac.

To date, the GSEs have been the scapegoats for everything that is wrong in the mortgage market and the solution for few of its problems.

But what if Fannie and Freddie, which know more about servicer practices than anyone else, pushed firms to improve their systems and hire enough people to modify the loans that can be saved and foreclose on the borrowers who will never be able to pay their debt?

Senate Banking Committee Chairman Chris Dodd asked this question at a Dec. 1 hearing, prodding executives from both Fannie and Freddie to lean harder on servicers.

"Why haven't you done more to insist upon servicer reform?" Dodd asked Terry Edwards, Fannie's executive vice president of credit portfolio management. "Don't you believe that they would jump back through hoops to respond to your concerns?"

The question must have surprised Edwards, because initially all he came up with was: "So our approach has been to understand the problem, get behind the problem, and try to solve the problem."

Recovering temporarily, Edwards assured the Connecticut Democrat that Fannie had "moved hundreds of thousands of loans where servicers weren't getting it done. They were at the bottom of the barrel, if you will, in terms of servicer performance, and we moved servicers to where there was capacity in the industry."

But after another witness said he was thrilled to hear Fannie had "fired" lousy servicers, Edwards backed off. "I don't want to make any headlines here today," he told Dodd. "When we moved this servicing, we actually got together and worked on an arrangement where the servicer agreed to move some of these loans to a place where we had better capacity. We did it in such a way that was a win-win for Fannie Mae and the servicer, and ultimately families."

Clearly Fannie and Freddie do exert some influence over servicers. They both monitor performance and press for improvements. Each has lengthy guidelines outlining what servicers are expected to do when a borrower gets into trouble.

But three years into the mortgage meltdown, it is time for Fannie and Freddie to do more. It is time for the government to turn from its piecemeal, voluntary loan-mod programs that haven't worked to something more centralized and comprehensive.

"The government has the perfect vehicle in the GSEs. Along with [Federal Housing Administration], they are the ones that can really turn this around," said Clifford Rossi, the managing director of the Center on Financial Policy at the University of Maryland. "With Fannie and Freddie in conservatorship, the government has the power to pull some levers."

The government took over Fannie and Freddie in September 2008 and has covered some $300 billion in losses since then. Obviously, the government wants to contain its losses, and that's a big reason principal writedowns are not occurring on loans owned by Fannie and Freddie.

But the GSEs have plenty of weight to throw around in this market. Freddie pays servicers $5 billion a year; Fannie refused to provide a comparable number. But together the companies own 55% of the 55 million single-family mortgages outstanding in this country. Fannie owns 18% of the seriously delinquent mortgages and Freddie owns 10%.

The biggest stick they could swing would be to stop buying loans from originators with subpar servicing operations.

Fannie and Freddie could shun any servicer that had not invested in the technology and the manpower, and even the cultural change needed to refocus on resolving the mortgage mess rather than simply slogging through it. Alternatively, the GSEs could continue to do business but cut the fees they are willing to pay laggard firms on new business.

As Edwards indicated, the GSEs also could yank servicing business from companies, and could put a dent in the second-lien problem by transferring servicing from any company that holds the second on a mortgage. Servicers would then be free of a conflict of interest that has thwarted foreclosures and modifications.

But there are other, less drastic steps Fannie and Freddie could take.

For instance, Freddie ranks servicers by various factors and slots them into four tiers, ranging from "superior" to "unacceptable." These reviews are updated monthly, but the results are available only to the individual servicer. Freddie pitches it as a service to servicers. "The profile exists as a tool to help you improve your institution's efficiency and performance," Freddie says on its Web site.

But what if Freddie decided to use those reviews as a lever to encourage servicers to improve? Imagine the impact if Freddie made those rankings public, and it noted every month in which companies had improved and which had been downgraded.

That would be a big incentive to servicers, much the way getting an "outstanding" Community Reinvestment Act grade used to be.

Asked for details on the program, Freddie would not say how many servicers had been slotted into each of the four categories, much less what rating it had assigned to individual companies like Bank of America or JPMorgan Chase.

Freddie would not even say if any servicer was currently ranked "superior." Fannie does not have a similar system for ranking servicers.

(A Congressional Oversight Panel report released Tuesday lists even more ways the GSEs could be used to cure the mortgage market; go to

For now, the debate remains stuck on things like whether loan modification efforts should take place parallel to the foreclosure process. This confuses borrowers and enrages their representatives in Congress; servicers claim Fannie and Freddie force them to use this "dual track," while the GSEs say it isn't true.

The other current hot button is a "single point of contact," which Edwards described as a counselor for the servicer, putting "their arms around the person who's in a jam and explains to them what is going on."

Servicers say this is unrealistic, expensive and inefficient, and note call centers pool information so borrowers can be helped by whomever answers. Customers would get much slower service, bankers say, if they could only speak to one person.

This finger-pointing isn't helping, and it's blurring the larger issue. Tweaks haven't — and won't — help. If the government, if the industry, wants to put the mortgage crisis behind it, the GSEs' market power should be used to improve both the foreclosure and the modification process.

"If Fannie and Freddie don't do it, who else will?" asked Kurt Eggert, a professor at Chapman University School of Law, near Los Angeles. "What we are in right now is just a holding pattern, which is bad for both borrowers and investors. "What we have to do is move toward speedier resolution of problem loans."

Subscribe Now

Access to a full range of industry content, analysis and expert commentary.

30-Day Free Trial

No credit card required. Access coverage of the securitization marketplace, including breaking news updated throughout the day.