Performance and risk analysis have gone a long way since the dark ages of investment, when returns were measured in rather raw form, while risk was hardly measured at all. These days, pension plan trustees are enlightened by slick performance reports, including hundreds of statistics, thoroughly calculated and well-presented with coloured charts.
However, something interesting is happening in pensions management. Despite all this information provided, many trustees, sponsors and members still find it difficult to understand what is happening to their assets, particularly in recent turbulent times. Are the traditional tools adequate to help steer the ship through the wild sea?
This poses new challenges not only to the providers of performance and risk analysis but also to pensions plan fiduciaries. The question of how to better organise an effective quantitative monitoring system has become a key element in the investment governance of pension schemes.
Before we go into this further, let’s have a look at the state-of-the-art is in this field. In many ways, performance measurement has developed very well in the 1980s and 1990s. With modern portfolio theory, a more scientific approach was adopted, spreading out from the US, aiming at a much more systematic - and correct - measurement of investment returns and risk.
All this culminated in agreed industry standards (national and global), on how fund performance should be reported and verified. It is now broadly agreed that there is no meaningful measurement of performance without measurement of risk, and that one objective of analysing history is to learn something useful for the future.
This process was also accompanied by substantial organisational changes in the industry. New performance measurement departments were set up, offering a specialist career in performance or risk analysis. In addition, specialist firms were formed to provide in-depth services to pension funds. Pension plan directors can now expect - almost as a matter of routine - a pretty solid service from their advisers. In general terms they should be able to:
q Master the mathematics of portfolio return and risk;
q Help define relevant benchmarks for investment portfolios;
q Provide a meaningful performance attribution;
q Apply appropriate risk and risk-adjusted performance statistics;
q Cover all asset classes and currencies globally;
q Provide more sophisticated analysis where and when required;
q Report on the amount and impact of (explicit and implicit) transactions costs, and all other investment-related costs;
q Give explanations and qualitative assessments in simple words;
q Present an integrated, well-structured report, including executive summary and detailed appendices.
However, this is often easier said than done. For example, many pension funds still experience piecemeal reporting received in different formats from different providers at different times. In practice, there are still plenty of other unsatisfactory situations. To name a few:
q Inconsistent figures, confusing text, unlabelled charts;
q Time frames are often too short, or opportunistically chosen;
q Attribution analysis irrelevant to the actual investment style;
q Risk statistics that are not properly understood even by those who present them;
q Cut-and-paste comments, ie, not client-specific, or fudging important issues;
q Bad explanation of performance, in particular outperformance;
q Analysis poor or missing on certain assets (eg, corporate bonds, derivatives);
q Lack of a coherent analysis from an overall pension fund perspective.
Surely, such problems should be manageable when approached in a business-like manner.
However, new and bigger issues have arisen in recent years, creating considerable confusion in the industry. Falling equity markets and rising liabilities led to a deterioration of funding levels and a sometimes substantial mismatch of assets and liabilities. Trustees, sponsors and pension plan members question what has gone wrong with their control tools. Unfortunately, even with a well-developed performance and risk reporting system, they would not easily find the answers.
The traditional analysis may well have shown excess returns generated by their fund managers as well as tracking errors within the limits given. But how can you pay rising pensions with falling returns?

Many of the problems stem from the concentration on relative performance, measured against some market indices or peer groups, while (almost) ignoring the absolute values in hard currency. Also people realise that ‘performance’ and ‘risk’mean different things to different parties in the pension deal. Another problem is that a major gap has become manifest between the ongoing investment performance/ risk measurement and the actuarial valuation of liabilities (which is still often only undertaken as a snapshot every three years).
In a nutshell, real world problems demand a rethink, both technical and organisational, of the whole area. To start with some more technical issues:
q Spurious accuracy: Are we over-analysing decisions of little relevance (eg, some marginal stock selection) and under-analysing factors of high relevance to the pension fund (eg, the plan’s asset allocation)?
q What are (historic) statistics worth? There are frequent structural breaks in the economy but normally we find out only some time after the event. An example: What do historic credit default statistics really tell you?
q Non-normal distributions: For example, ‘extreme’ events seem to occur more often than they should according to typical model assumptions;
q Simple risk measures such as standard deviation of returns are often blamed and people are now trying to introduce alternative ones, such as VAR. However, how to make sure they will be applied and interpreted adequately?
q What about the other interesting risks in your portfolio, eg, concentration, extreme valuations, survivorship bias, liquidity risks, mismatch risks, and so on?
q How to best measure the relevant risks of alternative asset classes, eg, private equity or hedge funds? How much transparency is desirable?
q How exactly to define liability-based benchmarks? What does performance and risk mean in the context of liability-plus-alpha investment?
Academics are currently working on how better to model the real world. Performance measurement experts are expanding their range of tools and services. But it will take some time to get new concepts and systems applied in practice, and this will require some learning time by the ultimate users.
But the bigger challenge for pension trustees seems to be ‘organisational’, the overall direction and control of performance and risk reporting. It is time for pension plans not just to take what they are being offered from providers but to define a clear policy for their own particular scheme, top to bottom. Key issues include:
q Relevance: What exactly do we want to have measured? Performance and risk analysis needs to be relevant to the objectives, design, risk tolerances and time horizons of the specific pension plan;
q Structure: All levels of decision-making in a pension fund need to be adequately measured and assessed (eg, asset allocation, fund structure, manager selection and tactical decisions). This includes the performance of those accountable at each step (including consultants, investment committees and pension boards) and not just fund managers;
q Responsibility: Who should the job be delegated to? Managers, custodians, actuaries, auditors, specialist shops, in-house management? How to best consolidate and integrate data reports coming from different sides?
q Liabilities: In times of low nominal returns and high volatility, the asset side needs to be linked more consistently and timely with the liability side, which in itself is ever-changing, and so difficult to capture. Such issues are even more intricate with cash-balance and defined contribution plans.
Some of these questions go well beyond classic performance and risk measurement in its narrow sense. This just shows how this area is bound to evolve with the changes in markets and investment approaches. As the fiduciary duties of pension trustees are only increasing, it is essential they have the appropriate monitoring tools, whichever direction they choose to go in.
Georg Inderst is an independent consultant based in London
info@georginderst.com

Topics