Measuring the measurers

Almost 2,000 years ago Juvenal, the Roman poet, asked: Quis custodiet ipsos custodies? - which may be freely translated as who measures the measurers? When it comes to investment consultants, pension funds are still waiting for an answer.

Effective management of advice given by investment consultants - probably the only profession in the financial services industry without external, independent metrics - would seem to be a relatively rudimentary demand. Indeed, Myners had identified a need for such a service.

Yet in December of Edinburgh-based Blacket Research, the first company to evaluate investment consultants’ advice to pension funds, went into liquidation.

Since then it has been suggested that consultants scotched the firm’s research on their largely questionable performance in advising on fund manager selection by claiming proprietorship over the data used.

Roger Brown, one of the company’s founders rejects this theory. Consultants did effectively prohibit publication of Blacket’s report by claiming the data it was based on were their property, he confirms. But that’s not why Blacket went bust, he adds.

The reasons for Blacket’s demise are more mundane. “The demand just wasn’t there - or at least it wasn’t widespread,” he says. “Looking back, we made mistakes. We launched just before A-day [the introduction of a new set of pension rules last April] and we didn’t market to pension fund trustees - most of our communication was aimed at fund managers and they aren’t the decision-makers.

“In any case, wider governance issues just weren’t on trustees’ agenda at the time,” he adds. “That’s changing - we made enough of a statement. But it’s a conservative industry. Change is slow in pensions.” A more likely concern for investment consultants was that, for the first time, they were open to scrutiny against their peer group. Few were willing to discuss the issue of independent metrics with IPE. They pointed out that there is no lack of measurement: all major consultants claim they provide detailed performance information to their clients. The problem is, rather, that there is no standard measurement across consultants. Brown claims the methodologies are biased to give consultants a better chance of success, “even if it isn’t deliberate”.

“It’s only recently - in the past few years - that the question [of independent measurement] has come up,” says Keith Faulkner, managing director of Pensions Adviser Review, an adviser on advisers to pension funds. “The big firms - Mercer, Hewitt and Watson Wyatt - all have their own measurement systems but because they’re all different there’s no way to compare. Other, smaller consultants don’t even bother trying.”

If Blacket suffered from first mover’s disadvantage in a market crammed with other priorities, is the market for independent metrics likely to grow significantly? It isn’t clear that, with A-day absorbed, trustees are rioting for transparency on their consultants and there are good reasons - other than consultants’ reluctance - for the failure to introduce transparent performance measurement.

First, it is not clear whose decisions pension funds will be measuring. Nick Sykes, senior consultant at Mercer, points out there is a risk of asking consultants to answer for trustees’ decisions.

“It isn’t a trivial problem,” says Sykes. “There isn’t a problem for clients assessing what we do. We’re advisers; we don’t have power delegated to us. A scheme’s trustees can take the advice or not take the advice; or they might act on some of the advice - say, x and y but not z. Who is accountable for what isn’t clear.” Even if the consultants were right, they’d likely be too late. The lack of an appropriate time horizon remains an issue because a manager’s entry onto the fund manager short-list might take 18 months to reach pension funds’ manager selections.

Second, the potential negative consequences of independent measurement is the likely impact it would have on client relationships. If pension funds choose investment consultants on an ad hoc basis, Sykes argues, the danger is that they will undermine the long-term character of those relationships.

“The trustees wouldn’t feel that they had a retainer relationship,” says Sykes. “We do have clients who come to us ad hoc but it’s constructive to have an overview of long-term objectives and changes to those objectives. It makes for better decisions if you aren’t the flying squad.” A third potential problem is the
likelihood that consultants, faced with trustees holding them to account, will opt for the lowest-risk option. As rationales go, this amounts to: if you watch them, they’ll take no risks; if you do not, you’ll have no idea what they’re up to. Yet even some with less of an obvious interest agree that liability increases risk aversion.

“They’re so worried about their insurance premiums that they don’t give good advice,” says Richard Stroud, CEO of The Pensions Trust fund in London. “Pension fund managers could say: ‘If your advice isn’t good, we’ll sue’. Or they could say: ‘Even if it isn’t correct, we won’t sue’. But they need to come to an agreement. If the advice is to put it all into bonds, it takes the risk out of it but what you’re going to get is a lousy investment return.”

What trustees want, says Sykes, is not super-scrutiny but some reassurance that they’re getting their money’s worth. “We put a lot of time and effort into sifting the universe for A-rated fund managers but we don’t tend to have long discussions with trustees about how we arrived at the short-list,” he says. “It’s not that there isn’t a robust basis for the process, but trustees don’t necessarily ask what that process was.”

Likewise Chris Armitage, former pensions investment controller at the J Sainsbury pension fund, one of Blacket’s erstwhile clients, suggests the demand is more for reassurance than metrics. After a recent review, the scheme stuck with incumbent consultant Russell.

“Both Russell and [competitor] Mercer had measurement systems in place - but it wasn’t uppermost in our minds. All major consultants know what they’re doing,” he says.

“If we were offered the ability to track consultants’ performance, we’d take it,” Armitage adds. “Around 18 months ago, we asked them for proof that they were adding value, and they gave it. We’d been happy with the advice they gave us and happy with the risk and controls. Even so, we wondered whether we might have done it better.”

But Sykes’ suggestion that pension funds aren’t very interested in measuring consultants begs the question why consultants were so reluctant last year to hand over the data that would have allowed Blacket to release its research. Yet it may be that the whole business of measuring advice comes down to this: do you, the client, trust them or not? The market penalty for losing trust will be client defection.

“Ultimately, it is our clients that hold us to account for what we have delivered and, in a free market, surely this is a very good form of measurement,” Paul Trickett, European head of investment consulting at Watson Wyatt, said in a written statement.

How well did they do?

“Of all the professionals in the financial services industry, investment consultants are the only ones not measured,” says Keith Faulkner, managing director of Pensions Adviser Review, an adviser on advisers to pension funds. “But consultants will tell you how well fund managers have done to the third decimal point.”

Blacket’s claims centred on two issues: the extent to which past performance influenced fund manager selection and consultants’ ability to predict outperforming managers.

Blacket claimed consultants’ advice was often driven by past performance - despite their claims to see beyond historical data. Specifically, it found that 56% of Watson Wyatt and Hyman Robertson recommendations had top-quartile form over a five-year pre-selection period, compared with 38% for Hewitt and 32% for Mercer.

Both Hyman’s and Watson Wyatt declined to be interviewed by IPE.

According to Blacket’s research, after three years no consultants proved more effective than random selection at selecting the best fund managers. Watson Wyatt, the strongest median consultant in terms of avoiding the worst and identified the best, produced a short-list with both the worst and the best performance.

Blacket has arguably already affected the way pension funds evaluate investment consultants’ advice on fund managers - or at least caught on to an emerging trend. Its fund manager selection survey report 2006 favourably compared manager selection decisions taken by investment committees with those selected by trustees.

Donny Hay, director of UK institutional investment at Edinburgh manager Martin Currie, sees more CalPERS-style investment committees making the decisions in what he calls the “do-it-yourself model”.