Picture review — Clear Lines on Audit and Risk

Review each picture on its own. Each picture must be fully defined. Each picture must describe the outcome, not something the on the way to the outcome. Each picture must describe the outcome, not an indicator of the outcome. A picture may refer to future prospects as they will look at the end of the planning period. Each picture must become factually true or false, regardless of point of view. Each picture should produce an emotion, instantly. The picture should match the way the year is remembered in the future. Each picture shows the period outcome, not a specific incident during the period. The period outcome can be an accumulation of specific incidents. If you keep the success and failure levels, make sure the picture matches the level.
What to read first: Prove and consolidate the outcome pictures
Everyone	Version 3.0 Beta

Review each picture on its own.

Check the descriptive pictures in each cell on each of these questions, and make changes until you are happy with the answer.

Each picture must be fully defined.
Each picture must describe the outcome, not something on the way to the outcome.
Each picture must describe the outcome, not an indicator of the outcome.
A picture may refer to future prospects and the end of the planning period.
Each picture must become factually true or false, regardless of point of view.
Each picture should produce an emotion, instantly.
Each picture shows the period outcome, not a specific incident during the period. The period outcome can be an accumulation of specific incidents.
If you keep the success and failure levels, make sure the picture matches the level.

Each picture must be fully defined.

Does the picture define the outcome fully, so that in a year from now, no-one will be left wondering what you were thinking? Clarify it as necessary.

Unit Objective: Maximise the timeliness and accuracy employee payments employees in line with proper management decisions and legal entitlements			Unit Objective: Maximise the timeliness and accuracy employee payments employees in line with proper management decisions and legal entitlements
Success/Failure level	Outcome pictures		Success/Failure level	Outcome pictures
Qualified Success	There were a few minor exceptions to paying everyone on time.	BECOMES	Qualified Success	There were a couple of avoidable events in which some employees were paid late, but within 48 hours of the usual time.

Each picture must describe the outcome, not something the on the way to the outcome.

Does the picture describe the outcome that matters, or something that leads in that direction?

The outcome is the final consequence of any event you have in mind.

First thoughts often capture a step on the way to the outcome. For a positive outcome, it might be something that you think you need to do to be successful. For a negative outcome, it might be something that will set you back, or put you on a track to the unwanted outcome.

Neither prerequisites to success, nor setbacks, are outcomes in themselves. They are steps along the way to an outcome. They may not even decide the outcome. You can recover from setbacks, and things can still go wrong after you achieve the most important milestones.

If you’ve captured a step on the way, replace the description of the step with a description of the year-end outcome to which the step is leading.

Unit Objective: Stick to the approved unit budget			Unit Objective: Stick to the approved unit budget
Success/Failure level	Outcome pictures		Success/Failure level	Outcome pictures
Partial Success	Carelessness with financial planning and spending	BECOMES	Partial Success	Some surprise year-end variances that the boss or CFO would have wanted to avoid.

You can also approach this test by asking why the described outcome is important. There are two kinds of answer:

(a) Because if we don’t do that, there’s no way that …X… will be achieved.

(b) Because that’s why we exist. If we don’t do that, we will have had no reason to exist, or at least no reason for the organisation to have funded us to do that part of the plan.

These are both good reasons for the outcome to be important. However, if the answer is of the (a) kind, replace your existing outcome description with a description of the ‘…X…’, the ultimate purpose to which the original ‘outcome’ was clearly essential. You may want to keep a note of the original description for possible re-use as a strategy, measure, or indicator.

Each picture must describe the outcome, not an indicator of the outcome.

Does the picture describe the outcome that matters, or an indicator that predicts the outcome?

An indicator is something that is easily seen and measured now, that is supposed to correlate with (predict) a reality that will be known in the future. The reality is not yet known.

In risk management we are concerned with the unknown future reality. The indicators may be important, but they are not the reality itself.

The outcome picture must refer to the reality itself.

Outcome pictures may include measures. Measures are often confused with indicators. Measures are numbers that are supposed to represent the important attributes of something that has already happened. That something might be the outcome itself (outcome measures), or something with predictive value (indicator measures). There is no prediction involved in the measure itself. Measures are not possible before the event has occurred, and may not be possible until well afterward. The numbers used as measures may be actual quantities, proportions, statistics, or ordered classifications (ordinal variables) such as high, medium, low. Measures hardly ever represent all the characteristics that matter, and may be misleading for evaluation. That is one reason why you are painting outcome pictures, and not measuring success as a number.

In this example, the initial ‘outcome’ is actually an indicator – a bad one, as it happens. The important thing is the (absence of) actual fraud. Detection of fraud always comes later, or not at all. At best, not detecting any frauds might suggest (indicate) that no frauds have occurred. At worst, it could mean that fraud is going on all the time, and no-one even notices. Detection isn’t even a good indicator.

Unit Objective: Prevent fraud against the organisation by employees or by payroll unit members

Success/Failure level

Outcome pictures

Success/Failure level

Outcome pictures

Success

No frauds detected during the year.

BECOMES

Success

Good evidence that there have been no undetected payroll-related frauds during the year, beyond minor unauthorised and unrecorded absences.

Any fraud attempts that occurred were promptly detected and referred to appropriate investigation.

Fraud prevention measures are convincing.

The new pictures must be entirely different from the originals, as there is no simple trick to convert an indicator into a picture of the underlying outcome. The technique is based on asking ‘why’.

A picture may refer to future prospects as they will look at the end of the planning period.

You draw pictures of the world on the last day of the year being planned. The character of that picture is influenced by expectations for the future beyond that day. Expectations for the future are influenced by what has happened during the year, and by the position on the day, especially in the objectives relating to Capabilities.

Unit Objective: Maintain the organisation’s capability to pay employees correctly indefinitely, even if there are changes in the way payroll services are provided
Success/Failure level	Outcome pictures
Success	Recent experience demonstrates continuing capability. Changes been planned and implemented to protect rather than erode the ongoing capability.	OK AS IS

Each picture must become factually true or false, regardless of point of view.

Is each outcome picture described so that it will be an agreed fact that the real outcome does or does not match the picture?

It is common to use subjective terms like ‘major’ or ‘minor’. Yet your idea of ‘major’ and ‘minor’ could be very different to those of (say) an executive in a different branch of your organisation, or those of an external auditor. Replace any pictures built from subjective terms with factual statements about what will or will not have happened at the end of the year.

You may want to include some measures of success or failure that don’t depend on anyone’s emotions or point of view. You have some measures defined in your business plan already. There is no sense in becoming rigidly numerical, and even less sense in agonising about how the measurements might be made. You don’t need any of that for risk management purposes.

Unit Objective: Prevent and avoid violations of confidentiality expectations around payroll			Unit Objective: Prevent and avoid violations of confidentiality expectations around payroll
Success/Failure level	Outcome pictures		Success/Failure level	Outcome pictures
Partial Success	There were some minor incidents around confidentiality.	BECOMES	Partial Success	Several times during the year, something leaked from the payroll unit through carelessness with talk, email, or documents. The unit basically understands what is to be kept private and makes an effort.

And here’s another example. The trouble with the old outcome picture is that while it’s bad for the payroll unit manager (and possibly for the IT team), it would be invisible and unimportant to most of the organisation. It isn’t an agreed fact that this outcome would represent Qualified Success. If the benefits are delivered anyway, the project is an unqualified Success, even if there is continued bickering.

Unit Objective: Move to a new payroll system and service arrangements with long term benefits			Unit Objective: Move to a new payroll system and service arrangements with long term benefits
Success/Failure level	Outcome pictures		Success/Failure level	Outcome pictures
Qualified Success	Payroll team members unacceptably hostile and resentful toward the IT team responsible for the project.	BECOMES	Qualified Success	The change project is still expected to produce benefits. There has been some delay caused by payroll unit issues.

Each picture should produce an emotion, instantly.

Each outcome picture should be instantly meaningful to people working in your part of the organisation. It should produce a specific emotional response, even if that response is boredom, complacency, or irritation. The language should be of the same kind used in corridor conversations—the kind of conversation that reports good and bad news in a few seconds. Subject to clarity and objectivity, it is best to avoid a complicated list of measures and criteria requiring analytical thought.

Unit Objective: Maximise the timeliness and accuracy employee payments employees in line with proper management decisions and legal entitlements – Timeliness of Response			Unit Objective: Maximise the timeliness and accuracy employee payments employees in line with proper management decisions and legal entitlements – Timeliness of Response
Success/Failure level	Outcome pictures		Success/Failure level	Outcome pictures
Success	Timeliness and accuracy score of at least 87 using the index developed by the performance metrics people last year.	BECOMES	Success	Almost all authorisations and requests reaching the unit by advertised cut-off times were processed for the next payday. On the few occasions that was not possible, processing was sensibly prioritised and notice was given to anyone affected.

If you don’t see anything wrong with your picture, but it fails to produce any emotion, there may be a problem with the objective. Is the objective really what matters? What related objective would get the emotions flowing more freely? You can experiment with different pictures and work back to the underlying objective.

The picture should match the way the year is remembered in the future.

If the outcome is reached at the end of the year, how will it be remembered a year or two after that?

Chances are, that memory will be a visual image and a feeling, rather than numbers. That visual image, with its emotion, is the outcome picture you are trying to draw now.

Time tends to help with balance and objectivity, too. It may help to imagine the future not just at the end of the current planning period, but a year or two beyond that date. Think about how you might remember the year’s outcomes. Any feeling of relief or surprise might be gone, but the solid reality of the year’s results will remain. Draw a picture of that future solid reality, and use that drawing now as an outcome picture for the coming year.

Each picture shows the period outcome, not a specific incident during the period. The period outcome can be an accumulation of specific incidents.

Can the ‘outcome’ happen more than once during the year?

If the outcome can happen multiple times, it isn’t really the outcome for the period. It may be a type of event that can happen repeatedly. In that case, the description as it is does not capture the net outcome effect of the occurrence(s) of the event type. The repeated events may have consequences, but they are not themselves the consequence.

In this situation, you may be able to draw the period outcome in ways like this:

The pattern and level of …X-type… incidents was within normal expectations for this type of work (Success)

The pattern and level of …X-type… incidents was far worse than would be tolerated by stakeholders, and threatened the organisation’s viability, even if a few of those incidents are to be expected. (Failure or Worst imaginable outcome)

The period outcome can be understood as the accumulation of specific events during the year. A simple example is an annual sales figure, which is the accumulation of separate sales transactions, or the count of instances in which employees were paid late.

If you keep the success and failure levels, make sure the picture matches the level.

You don’t need the pre-defined success and failure levels after the pictures have been drawn. The seven pre-defined success and failure levels ran from ‘far better than expected’ through to ‘worst imaginable’. For the later stages of the risk work, you define consequence with the pictures, not the pre-defined levels.

If you want to keep the levels as a way of classifying pictures, you should check that each picture is fair relative to the level.

The reference table gives definite boundaries for the types of outcome that belong at each success or failure level.

If not, you may need to re-locate a picture. You can move a given outcome picture between levels, as in this example. Having made such a move, you will probably want to make some edits to the other pictures for the same objective.

Unit Objective: Maximise the employee perception that this organisation is a good employer		Unit Objective: Maximise the employee perception that this organisation is a good employer
Success/Failure level	Outcome pictures	Success/Failure level	Outcome pictures
Far better than expected		Far better than expected
Excellence	Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque lorem purus, iaculis vitae ultrices vel, hendrerit eu leo. Nulla vitae interdum enim.	Excellence	Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque lorem purus, iaculis vitae ultrices vel, hendrerit eu leo. Nulla vitae interdum enim. Records have been checked thoroughly and are known to be in good order and to support all payroll transactions during the year.
Success	Records have been checked thoroughly and are known to be in good order and to support all payroll transactions during the year.	Success
Qualified Success		Qualified Success