Performance attribution and self-assessment

The final analytical exercise is applied to the Fellow's own record, and it is weighted more heavily in assessment than investment returns.

Decomposition of returns

Returns should be decomposed into market exposure, sector and factor exposure, and security selection. In a rising market, much of what appears to be skill is attributable to beta; in a declining market, much of what appears to be failure is attributable to the same source.

Only the residual, the contribution of security selection after accounting for exposures, constitutes evidence of skill.

Classification of closed positions

Each closed position is assigned to exactly one of four categories:

1. Correct for the stated reason. The thesis developed as documented.
2. Correct for an unanticipated reason. The position was profitable for a reason not identified in the thesis. This represents a favourable outcome from a flawed process, and its classification as a success is a common form of self-deception.
3. Incorrect for the stated reason. The thesis was testable, was tested and failed. This is the most informative category of loss.
4. Incorrect for an unanticipated reason. A development occurred that the Fellow did not anticipate. The Fellow should assess whether the development could not have been anticipated or was simply not investigated.

The second and fourth categories are the most instructive, and are those most frequently misclassified.

Quantitative measures

The hit rate and the win/loss ratio, being the ratio of the average gain to the average loss, should be reported. A hit rate of 45% with a win/loss ratio of 3:1 indicates a strong process; a hit rate of 70% with a ratio of 1:3 indicates a process exposed to infrequent but severe losses.

The concentration of contribution should also be reported. Where three positions account for the whole of the return, the process in effect depends on identifying such positions, and the Fellow should be able to state their common characteristics.

Self-assessment

The Fellow prepares a one-page self-assessment, retained on the Fellow's profile, addressing:

• The judgements that proved correct, and the grounds for concluding that the reasoning, rather than the outcome alone, was sound
• The judgements that proved incorrect, and whether the error lay in the analysis or the timing
• The Fellow's characteristic error: the error that occurred more than once
• The specific changes the Fellow will make, stated with sufficient precision to be verified

The identification of the characteristic error is the principal objective. Most investors exhibit a small number of recurring errors that persist because each occurrence is remembered as an isolated case. Six weeks of dated updates make the pattern observable.

Assessment

Overstatement is penalised more heavily than investment losses. A Fellow who reports an unfavourable period with rigorous attribution is assessed above one whose favourable results were attributable to chance and who reports them imprecisely. The committee states this criterion explicitly before the programme begins.