Monty & Co — Blog

How much contingency should a project carry?

Daniel Atkin — Sun, 16 Aug 2026 01:06:42 GMT

Ask ten project teams how they set contingency and most will tell you a percentage. Ten percent. Fifteen for something unfamiliar. Five if the sponsor is watching costs closely. I have used those numbers myself and they were usually about right, which is exactly what makes them hard to argue with.

The problem shows up later. When the project overruns by 22%, nobody can point to which assumption failed, because no assumptions were written down. The number was defensible in the room and unexaminable afterwards.

What the percentage is actually telling you

It is not nothing. A practitioner who has delivered thirty projects in one sector has real prior experience of how badly they tend to run. What the percentage carries is information about the reference class, not about this register. That distinction matters, because two projects with identical budgets and very different risk profiles get identical contingency, and the number scales with the size of the estimate rather than with the shape of the exposure.

What a percentile says instead

Run a Monte Carlo simulation across a register and you get a distribution of possible outcomes: thousands of versions of the same project. A percentile is a position in that distribution.

One project, nine risks, 200,000 simulated runs. Each bar is a band of possible contingency outcomes, and its height is how often that band came up. The percentile is simply a position along the bottom: $1.9M at P80 means eight runs in ten finished at or below that figure. Notice how much further P90 sits from P50 than P50 sits from zero. That asymmetry is the tail, and it is why the choice of percentile moves the money so much. Simulated with the same engine that runs Project Risk Quantification, 200,000 iterations, seed 20260809. Bar heights use a square-root scale so the tail stays visible next to the peak. View the chart.

P80 means 80% of the simulated outcomes came in at or below this figure.

That is the whole definition, and the looser paraphrases are wrong in ways that matter. P80 is not 80% confident. It is not an 80% chance of coming in under budget. It cannot be, because the distribution is of the risks you modelled, not of the project. It excludes scope change, escalation outside your ranges, and estimating error you did not capture. It is a statement about the simulation, and it inherits every assumption the register was built on.

The useful part is not the number. It is that you can take the number apart in front of someone. The judgement has not gone away, it has moved upstream into the ranges, the probabilities and the scope of the register, where it is visible and can be argued with.

From a percentile to a number in the budget

This is the step most explanations skip, and it is worth being concrete about it, because everything that follows depends on it.

You start with a base estimate: the cost of the work as scoped, carrying no allowance for things going wrong. The simulation tells you what the risks might add on top of that. You choose a percentile, and the contingency is the amount sitting at that point in the distribution. Base estimate plus contingency is the number you ask to have funded.

Infrastructure Australia puts it in a single sentence. A P50 cost estimate is the project cost with sufficient contingency to provide 50 per cent likelihood that this cost would not be exceeded, and P90 is the same sentence with ninety in it.

The Commonwealth's cost estimation guidance carries a worked example that shows the arithmetic. On a $100M base estimate it sets out a P50 contingency allowance of 15%, giving a risk-adjusted estimate of $115M, and a P90 allowance of 40%, giving $140M. Escalation is then applied on top of whichever figure you funded, to get to an outturn cost.

Two things fall out of that example that are worth sitting with.

The P50 allowance in that table is 15% of the base estimate, which is roughly what the ten teams at the top of this article would have told you off the top of their heads. The percentage heuristic is not arbitrary. On an ordinary project it lands somewhere near P50. What it cannot do is tell you where it landed, or move when the project is not ordinary.
The step from P50 to P90 is 15% to 40%. Nearly three times the contingency for exactly the same scope. That is not a technical refinement, it is a materially different ask, which is why the next question is the one that actually matters.

So which percentile?

A common reference point worth challenging is P80 as the default marker. For some projects that will be the right level. It is not the standard, though, and the guidance most often cited in support of it does not actually say that.

AACE International's Recommended Practice 41R-08 mentions 80% once, and not as advice:

The more conservative, risk-averse attitude used by many profit-making companies, is to specify a probability of 80% or higher that the project will not overrun. This is a safer route but by specifying a high probability, the required contingency will increase and with it the project cost.

It describes the practice, then warns about it. The same paragraph goes on to say that large contingencies across a portfolio sequester money that could have funded other projects.

The UK Green Book treats a P90 as a way of expressing uncertainty around a central estimate rather than as a funding rule, and the example it works through is a reference-class exercise rather than a simulation: line up comparable projects by cost and take the ninetieth percentile of that list (paragraph 6.72). It separately requires an explicit adjustment for optimism bias (paragraph 2.25). Neither is a mandated percentile.

Closer to home, Australian public infrastructure does not work in P80 at all. It works in P50 and P90. Infrastructure Australia presents cost estimates at expected value, P50 and P90. The Infrastructure NSW Cost Control Framework is blunter: agencies must provision contingency at P90 for Tier 1 high-profile high-risk projects and P50 for Tier 2, unless Cabinet has approved a different level.

Which means the line I have heard many times, that funding to P50 is a coin flip nobody would knowingly authorise, is not right either. Funding at P50 is frequently deliberate, and the P50 to P90 increment is held somewhere else rather than not held at all. Queensland Transport and Main Roads says it plainly: project managers are provided with a contingency reserve of up to P50 levels, with the difference between P90 and P50 contingency provisions being kept at portfolio levels. Infrastructure NSW does the same, giving the project director delegation to draw down against the P50 contingency and holding the P90 minus P50 delta at agency senior executive level. What is not defensible is funding at P50, calling it prudent, and holding nothing behind it.

Setting the risk appetite level is an organisational decision, not something an analyst should do in isolation.

ISO Guide 73 defines risk appetite as the amount and type of risk that an organization is willing to pursue or retain. ISO 31000:2018 puts the establishing of it with top management, whose responsibilities include setting the amount and type of risk that may or may not be taken to guide the development of risk criteria (clause 5.2). AACE reaches the same place from a different tradition: the selection of desired probability depends upon the risk attitude of management.

Computing a percentile is technical work, and it should be reproducible by anyone holding the same inputs. Choosing which one the organisation funds is governance. Conflating the two is how analysts end up quietly setting appetite for people who never delegated it to them.

What the number still does not cover

Three things, and the third is the one I see go wrong most often.

Risks nobody identified. The simulation quantifies the register it was given and says nothing about what is missing from it. That gap has a name and an instrument. PMI's Lexicon separates a contingency reserve, time or money allocated in the schedule or cost baseline for known risks with active response strategies, from a management reserve, time or money that management sets aside in addition to the schedule or cost baseline and releases for unforeseen work that is within the scope of the project. Contingency covers the risks you found. Management reserve covers the fact that your register is incomplete. Carry only the first and you have funded the part you could see.
Correlation you assumed away. If two risks share a driver, one supplier or one weather window or one scarce skill set, and the model treats them as independent, the tail is understated. Note that it is the tail specifically. The mean is unaffected by dependence and the median can move either way. Independence is an assumption, not a neutral default.
Aggregation across projects. You cannot add the P80s of five projects together to get a portfolio P80. Percentiles do not sum.

The aggregation one is worth dwelling on

It would be normal to assume this works as a diversification benefit: the portfolio figure comes in below the sum, because five projects are unlikely to all have a bad day at once, so adding them up is the conservative move. That is true sometimes. It is not a rule, and it is very easy to carry it as one.

Take five independent projects, each carrying a handful of moderate risks plus one low-probability, high-impact item at 12% and $4M. Each project's own P80 lands at about $299k, comfortably below its own tail risk, because there is an 88% chance that item does not fire. Across five projects, though, there is only a 53% chance that none of them fires.

So add the five P80s and you get $1.5M. Simulate the portfolio properly and P80 is $4.8M. Adding up understates what you need by more than three times, and it understates in the direction that leaves you short.

The same five projects, simulated together rather than separately. The separate humps are the tail risks firing: the left cluster is runs where none fired, the next where one did, and so on. Add the five projects' own P80s and you get $1.5M. Simulate the portfolio properly and P80 is $4.8M. The shaded band is the difference, and it is a shortfall rather than a cushion. Simulated with the same engine that runs Project Risk Quantification, 200,000 iterations, seed 20260809. Bar heights use a square-root scale so the tail stays visible next to the peak. View the chart.

Make the five projects move perfectly together instead, so that a bad day on one is a bad day on all of them, and the two figures land exactly equal. That is the single case in which adding percentiles up is correct, and it is not the case anyone is actually in.

You do not have to take the simulation's word for it either. AS/NZS IEC 31010:2020, the risk assessment techniques standard in the ISO 31000 family, says it outright in its clause on aggregating measures of risk (6.3.7.2):

If correlation is not taken into account appropriately the outcomes will be inaccurate and may be grossly misleading. Consolidating risks by simply adding them up is not a reliable basis for decision making and could lead to undesired results.

Which direction the error runs depends on the dependence structure and on the shape of the registers, and it can flip between percentiles in the same portfolio. The safe conclusion is the narrow one: a portfolio number has to be simulated at portfolio level, not assembled from parts.

None of this needs expensive software. A Monte Carlo over a risk register is not a difficult model: a probability and an impact range per risk, a few thousand iterations, and a sort. I have built plenty of these in Excel over the years and they were perfectly defensible.

The reason to reach for a tool is time, not capability. If you want to test any of this against your own register, what the tail does to P90, what correlation does to the shape, what happens when you aggregate, Project Risk Quantification runs in the browser, is free, and will get you there faster than rebuilding the spreadsheet. One thing to know if you do: it reports the percentile as the contingency figure itself, sitting on top of your base estimate, rather than as a total with the base estimate already inside it.

The method matters more than the tool though. The number you hand a sponsor should be one you can take apart in front of them, and you should be able to say what it does not cover without being asked.

I would be interested in how others are handling the portfolio question in particular. If you aggregate across a programme, do you simulate it at portfolio level, or add the percentiles up and assume it errs on the safe side?

The register is one technique. The standard lists more than forty.

Daniel Atkin — Sat, 15 Aug 2026 21:30:00 GMT

Here is an uncomfortable audit you can run on your own risk framework in about a minute. List the techniques your organisation actually used in its last full assessment cycle. For most, the honest list has two entries: a risk register, and a consequence/likelihood matrix to rate what is in it. Identification, analysis, evaluation and reporting, four different jobs, all done with the same pair of tools.

Now put that against the standard. IEC 31010, the assessment techniques companion to ISO 31000, catalogues more than forty techniques, organised by the question each one answers: eliciting views, identifying risk, finding sources and causes, analysing controls, understanding consequence and likelihood, analysing dependencies, measuring risk, evaluating significance, selecting between options, recording and reporting. ISO 31000 itself says plainly that an organisation "can use a range of techniques for identifying uncertainties that may affect one or more objectives". The range exists because the questions differ. A framework that answers every question with the same two tools has stopped asking which question it is on.

The standard's own warning label

This is not an argument against the matrix. It is an argument the standard itself makes, on the matrix's own page. IEC 31010's entry for the consequence/likelihood matrix lists its strengths, it is easy to use, fast, and communicates clearly, and then a limitations list that is longer than the strengths. Two entries deserve to be read aloud in every framework review.

First: its use "is very subjective and different people often allocate very different ratings to the same risk. This leaves it open to manipulation." The standard, not a critic, is telling you that your primary analysis tool can be gamed.

Second: "Risks cannot be directly aggregated", one cannot say whether some number of Low risks equals a Medium. Which means the matrix cannot answer the question executives most want to ask of a register: how much risk are we carrying in total?

A tool with those limitations is still worth having. Sixty risks need triage, and the matrix triages. The failure is not using it; the failure is asking it to do jobs it was never designed for, when the catalogue next to it holds tools that were.

What choosing a technique is supposed to look like

IEC 31010 clause 7.2 describes a selection discipline most frameworks skip entirely. The choice "should be tailored to the context and use", and, critically, "the number and type of technique selected should be scaled to the significance of the decision". Not the size of the risk: the significance of the decision. A minor risk feeding a major investment decision deserves more technique than a major risk nobody is deciding anything about.

The clause lists what should drive the choice: the purpose of the assessment, stakeholder needs, legal and regulatory requirements, the importance of the decision, the time available, the information available, the complexity of the situation, the expertise on hand. It also contains a sentence worth keeping for the next time someone dismisses quantitative methods for lack of data: "in some cases where data is not sufficient, the rigour needed to apply a quantitative technique can provide an improved understanding of the risk, even though the result of the calculation might be uncertain". The discipline of specifying a model teaches you things even when the output carries wide error bars.

And the clause blesses plurality outright: "applying more than one technique can sometimes provide useful additional understanding".

A short tour, by question

The catalogue is too large to summarise honestly, so here is a deliberately small sample: one or two techniques per question, chosen for being usable without specialist software or a consulting engagement.

When the room cannot speak freely. The Delphi technique (B.1.3) runs expert judgement in structured anonymous rounds, and the nominal group technique (B.1.4) has participants write before anyone speaks. Both exist because the standard's authors knew what everyone knows: in a live workshop, the most senior voice sets the anchor and dissent arrives pre-softened. If your risk workshops always converge comfortably, the comfort is probably the hierarchy, not the risk profile.

When you suspect the register is missing things. Structured what-if technique, SWIFT (B.2.6), works a system through prompted what-if questions with the people who run it; HAZOP (B.2.4) does the disciplined version for processes, driving guidewords through every part of a design; scenario analysis (B.2.5) builds "models of how the future might turn out" and walks your objectives through them. All three exist because a register populated by asking "what are our risks?" mostly collects last year's answers.

When the argument is about whether the controls actually hold. Bow tie analysis (B.4.2) maps every pathway from cause to consequence and forces an effectiveness judgement on each control along it; layers of protection analysis (B.4.4) asks, quantitatively, whether the barriers stacked between cause and consequence are enough. Both belong to a category, "techniques for analysing controls", that many frameworks have no technique in at all, despite controls being where most of the money goes.

When one number will not carry the decision. Fault tree analysis (B.5.7) decomposes an event into the combinations of failures that produce it; event tree analysis (B.5.6) plays a single initiating event forward through the barriers that might stop it; Monte Carlo simulation (B.5.10) does the arithmetic once inputs are ranges rather than points, which is the honest shape of most cost and schedule uncertainty. This is the family that answers "how much?", the question the matrix's aggregation limitation concedes it cannot.

When the assessment must end in a choice. Cost/benefit analysis (B.9.2), decision tree analysis (B.9.3) and multi-criteria analysis (B.9.5) are the techniques for the step everything else feeds: selecting between options. A framework that assesses risk exhaustively and then chooses treatments by advocacy has stopped one technique short.

Risk is not the only thing on the table

One more push for breadth, from Australian guidance rather than the international standard. Infrastructure Australia's Assessment Framework guide draws a working line between risk, "events that have probabilities of occurrence that are predictable and outcomes that can be estimated with some confidence", and uncertainty, "events where probabilities of occurrence are difficult to predict and outcomes are challenging to quantify", and it structures its tooling advice around the difference: tools to analyse risk, and separate tools, scenario planning and real options among them, to analyse uncertainty. In practice, the guide notes, there is "a spectrum between risk and uncertainty", and investigation can move things along it.

The register-and-matrix pair lives entirely at the risk end of that spectrum. The strategic questions that most deserve assessment, market shifts, technology transitions, climate horizons, mostly live at the other end, where probability estimates are not honestly available and scenario-based techniques are the ones that work. A framework with no uncertainty tools has quietly limited itself to the questions its one tool can hold.

Broadening without boiling the ocean

Nobody should read IEC 31010 cover to cover and attempt all of it. The practical discipline is smaller: for the next assessment that matters, ask what question you are actually trying to answer, check which category of the catalogue owns that question, and add exactly one technique from it alongside the register you were going to use anyway. One Delphi round before the workshop. One bow tie on the risk the committee keeps arguing about. One Monte Carlo model where a contingency number has to survive scrutiny. One scenario exercise where nobody can honestly state a probability.

Clause 7.2's test is the one to keep: scale the technique to the significance of the decision. Some of the decisions crossing your desk are significant. The catalogue is written, the techniques are in it, and two tools were never going to be enough.

_Several of the techniques above have free browser implementations on this site: bow tie analysis with control effectiveness ratings, Monte Carlo cost-risk simulation across a register, and single-event quantification with treatment cost-benefit. The knowledge base covers the methods behind them, with the standards cited by clause._

The bow tie is a controls technique, not a poster

Daniel Atkin — Sat, 15 Aug 2026 21:00:00 GMT

Ask where the bow tie lives in the standard and you get an answer most practitioners find mildly surprising. IEC 31010, the risk assessment techniques standard, writes its entry for the technique in clause B.4, "Techniques for analysing controls", alongside HACCP and layers of protection analysis. The standard knows the diagram does two jobs, its process map also lists bow ties among the recording and reporting techniques next to the register and the heat map, but the home clause, the one that defines the method, is the controls one. The definition there is a diagram with a job: "a graphical depiction of pathways from the causes of an event to its consequences", showing "the controls that modify the likelihood of the event and those that modify the consequences if the event occurs".

That placement is the whole argument of this piece. The standard gives the bow tie two jobs, analysis and communication, and most organisations have kept only the second. A bow tie is a test you run on your controls. The picture is what the test leaves behind.

The wallpaper problem

Here is the version that actually gets produced in a lot of organisations. The risk assessment is done, the register rows exist, and someone draws a bow tie for the board pack because bow ties look rigorous. Causes on the left, consequences on the right, and a reassuring picket fence of barriers across every line.

You can recognise this version by what is missing. No control on the diagram carries an effectiveness judgement, so the picket fence reads as uniformly solid. Nothing distinguishes a control that exists from one that is planned, or half-implemented, or was last tested three years ago. And nothing changed because the diagram was drawn: no control owner got a question, no gap got a treatment, no rating moved. The diagram decorated a conclusion the team had already reached.

That is wallpaper. It is not worthless, because the picture genuinely is a good communication device, and the standard says so: it is "simple to understand and gives a clear pictorial representation of an event and its causes and consequences". But communication is the by-product. If the diagram never made anyone uncomfortable, the technique was not applied.

What the technique is actually for

The standard's use clause is blunt about the job. A bow tie "is used when assessing controls to check that each pathway from cause to event and event to consequence has effective controls, and that factors that could cause controls to fail (including management systems failures) are recognized".

Read as a procedure rather than a description, that sentence generates the workshop. For every pathway on the left: what stops this cause reaching the event? Not "what have we written down", but which named controls, owned by whom, working how well. For every pathway on the right: once the event has happened, what limits this consequence? And for every control on either side: what would make this control fail, and is anything watching for that?

The last question is the one that earns the technique its keep, because it is the one a register row cannot hold. The standard's drawing steps include two elements that rarely survive into the wallpaper version: escalation factors, "factors that might cause the controls to fail", drawn with their own controls, and management functions "which support controls (such as training and inspection)", linked to the controls they support. A barrier diagram with escalation factors on it stops being a reassurance document. The training that lapsed, the inspection regime that quietly stopped, the single person who administers the critical system: they appear on the page, attached to the exact controls they undermine.

The honesty mechanism is effectiveness

The fastest way to turn a poster into an analysis is to force an effectiveness judgement onto every control, on a scale, in the room, with the control owner present.

The uncomfortable pattern that emerges is nearly always the same one: the pathways that matter most are anchored by the controls rated weakest. Awareness training rated partially effective. A patching process rated partially effective. A crisis comms plan nobody has exercised. The picket fence was never uniform, and rating it says so in a way everyone in the room has to look at. That, not the tidy final image, is the deliverable. The residual rating you carry out of the workshop should be arguable from the effectiveness spread on the page, and if it is not, one of them is wrong.

When the bow tie is the right tool

The standard gives it a specific slot, and the slot is worth quoting because it prevents both overuse and underuse. The bow tie "is used when the situation does not warrant the complexity of a full fault tree analysis and event tree analysis but is more complex than can be represented by a single cause-event-consequence pathway". One line in a register: too simple to need it. A system safety case with interdependent failure logic: too complex for it. The wide middle band, one serious event with several credible causes and several consequences that matter: exactly it. The standard adds that it "is particularly used for analysing events with more serious consequences", which is where the investment of a workshop pays.

Two more properties from the use clause deserve more attention than they get. It works "proactively to consider potential events and also retrospectively to model events that have already occurred": a bow tie of an incident that actually happened, drawn against the bow tie you would have drawn beforehand, is a controls audit with nowhere to hide. And it can be used "for desirable consequences as well as undesirable ones", which almost nobody does and which the opportunity side of your register is quietly waiting for.

What it cannot do

The same clause that defines the technique limits it, and the limits are real. A bow tie "cannot depict a situation where pathways from causes to the event are not independent", the situations where a fault tree would carry AND gates: two things that must both happen. If your event needs combination logic, you need the fault tree.

And the standard is careful about numbers. Some quantification is possible "where pathways are independent, the probability of a particular consequence or outcome is known and the probability that a control will fail can be estimated", but "in many situations, pathways and barriers are not independent, and controls may be procedural and their effectiveness uncertain". In other words: the diagram is a map of mechanism, not a calculation. When you want the calculation, the standard points you at fault trees, event trees or LOPA, and for cost exposure you are better served by a Monte Carlo model of the event. The bow tie's honest output is the standard's stated one: the pathways, "the controls in place, and the factors that might lead to control failure".

Run it as a test

The practical version, next time a serious risk crosses your desk: one event, one workshop, the people who own the controls in the room. Draw the pathways, put every control where it acts, and rate every one of them for effectiveness before anyone is allowed to admire the picture. Then read the weakest ratings on the busiest pathways aloud. That list is your treatment plan, and it existed nowhere before the diagram forced it out.

If you want to try the method without procurement getting involved, Beau-Tie is a free browser-based bow tie editor built around exactly this discipline: typed controls, effectiveness ratings on a five-point scale, existing versus planned status, and residual, target and appetite ratings on a matrix, with the diagram exportable once it has done its real job. There are ten complete templates to start from, and the knowledge base has a fuller introduction to the technique with the ISO references attached.

The picture at the end will still look good in the pack. It will just have earned it.

One risk, quantified properly

Daniel Atkin — Sat, 15 Aug 2026 20:30:00 GMT

There is a moment in most risk committee meetings that everyone recognises and nobody enjoys. A risk is up for review, the matrix says High, and the argument starts about whether the consequence is really a 4 or actually a 3. Twenty minutes later the argument has not converged, because it cannot: both sides are right. The plausible bad outcome for that risk runs from an inconvenience to a company-defining loss, and the matrix has asked the room to compress that whole range into one cell.

The standard knows this. Buried in IEC 31010's own limitations list for the consequence/likelihood matrix is the exact defect: it "requires a single indicative value for consequence to be defined, whereas in many situations a range of consequence values are possible and the ranking for the risk depends on which is chosen". The twenty-minute argument is not a failure of the people in the room. It is the technique running out of road.

For most of the register, that is fine. A matrix rating is cheap, fast, and good enough to sort sixty risks into the ones that need attention and the ones that do not. The trouble starts when a real decision, usually a spending decision, hangs on one specific risk. Approve the $400k control program or not. Accept the vendor's limitation of liability or negotiate. Self-insure or transfer. At that point "High, probably" is not an input a decision deserves, and the honest move is to take that one risk out of the matrix and model it.

What modelling one event means

Single-event quantification is a Monte Carlo model of one risk event: the likelihood of the event occurring in a period, the range of consequences if it does, and the controls and treatments that modify both. Instead of one indicative consequence value, you give each impact a distribution, a low, likely and high with a shape, and instead of one likelihood word, a probability the drivers support. The simulation then plays the year out thousands of times and hands back the distribution the matrix could not hold: the chance of a quiet year, the typical loss when it is not quiet, and the tail you actually fear.

Monte Carlo earns its place here for the reason the standard gives: once inputs are distributions rather than numbers, "it is often not possible to derive analytical solutions", and simulation is the practical way of doing the arithmetic. Nothing about the method is exotic. It is the same technique cost engineers run across whole project registers, pointed at a single event.

The output that changes meetings is not the headline exposure figure. It is what happens when you run the model twice.

The two-pass trick

Run the model once with the controls you have. Run it again with the proposed treatment in place, reducing the likelihood, the impact, or both. The difference between the two runs is the treatment's expected benefit, in dollars, on the same assumptions both times. Put the treatment's cost next to it and you have the thing the committee was actually trying to decide: whether this control earns its keep at the margin.

This is where single-event modelling beats both of its neighbours. The matrix cannot do it at all: move a risk from High to Medium and you have recorded an intention, not measured a benefit. A full register quant does it but at the cost of modelling everything, when the decision in front of you concerns one risk. IEC 31010's guidance on technique selection says the effort "should be scaled to the significance of the decision", and a significant decision about a single risk is precisely the case for quantifying that risk alone.

One warning from honest experience of the pattern: sometimes the two-pass result says the celebrated treatment does not pay. The reduction is real but the cost is larger than the expected benefit at any plausible reading of the inputs. That result feels like the model failing. It is the model working. A cost-benefit table that only ever endorses the proposal is decoration, and the whole point of doing the arithmetic is that it is allowed to come back negative.

Keeping it honest

Quantifying one event does not make the inputs true, and a model this small has nowhere for bad inputs to hide. Three disciplines keep it defensible.

Elicit ranges, not points, and write down whose ranges they were. A distribution built from one person's guess is one person's guess with better formatting. The gain over the matrix is that the guess is now explicit, inspectable and arguable line by line.

Report percentiles as statements about the model. "The P90 annual exposure is $2.1M" means: in 90% of simulated years, on these assumptions, the loss was at or below $2.1M. It is not a promise about next year, and it collapses the moment the assumptions do.

And resist the slide into false precision. The matrix's other documented limitation, that its use "is very subjective and different people often allocate very different ratings to the same risk", does not vanish because the subjectivity now enters through distribution parameters. What changes is that the subjectivity is on the record, attached to named inputs, where a reviewer can find it and a better estimate can replace it.

Where to draw the line

Not every risk deserves this. The matrix remains the right tool for sorting the register, and a full register simulation remains the right tool for funding a project contingency. The single-event model owns the middle case: one risk, materially uncertain in its consequence, with a genuine decision attached. If the twenty-minute argument about 4-versus-3 has a budget line waiting on its answer, that is the signal.

Event Risk Exposure is a free browser tool built for exactly this shape: multi-driver causes, impact distributions, shared controls, treatments with per-treatment cost-benefit and ROI from the two-pass comparison, and a seed field so a result you put in a paper can be reproduced to the dollar. The worked example walks a customer data breach through the whole method, including a treatment the arithmetic declines to endorse. The methodology page covers the model in detail, including the parts that deserve scepticism.

The next time the room starts arguing about which single number to give a risk that obviously has a range, the productive answer is on the table: stop compressing, and model the range.

How to read a country risk rating without fooling yourself

Daniel Atkin — Sat, 15 Aug 2026 20:00:00 GMT

Sooner or later, most organisations that trade, source or expand across borders end up looking at a country risk index. Something has put a country on the table, and somebody wants a number. ISO 31000 puts this squarely in scope: understanding the external context means examining "the social, cultural, political, legal, regulatory, financial, technological, economic and environmental factors, whether international, national, regional or local" that bear on your objectives. A country rating is an attempt to compress most of that sentence into one figure.

Compression is not the sin. Nobody can weigh forty indicators in their head, and a defensible summary is genuinely useful for deciding where to look harder. The sin is compression with the workings hidden, because a rating you cannot interrogate is not analysis, it is a vibe with a colour scheme. Five questions separate the two, and they apply to every index on the market, including the free one we publish.

1. Is the method public?

Not the marketing page: the method. Which indicators, from which sources, weighted how, normalised how, refreshed when. If the answer is proprietary, you cannot know what the rating is sensitive to, which means you cannot know when it is wrong. A rating built from published weights over named public sources can be argued with, and being arguable-with is the entire value of writing a method down.

The licence trail matters for a second reason. An index built on data its publisher is not licensed to redistribute has a compliance problem baked into the product, and one day it becomes your compliance problem. The test is simple: does every indicator name its source and the terms it is used under?

2. Are the bands absolute or relative?

Most indices band by rank: a country is "High" because it sits in a particular slice of the scored cohort, not because it crossed an absolute line. There is nothing wrong with that, quartiles are a perfectly honest way to band, but it has a consequence you must know about: a country's band can move without anything in that country changing, because the cohort moved around it. If the index does not tell you which kind of band you are reading, you cannot know whether a downgrade is news about the country or news about everyone else.

The follow-on discipline: compare like with like. An overall band ranked against every scored economy and a category band ranked against the smaller set with data for that category are different statements wearing the same four words.

3. What happens when the data is missing?

This is the question that most reliably separates careful indices from confident ones. Data coverage is wildly uneven across economies: the same indicator set that covers Germany densely covers a Pacific micro-state barely at all. Something has to fill the gap, and there are only two honest answers: disclose it, or refuse to rate.

The unhonest answer is silent imputation, filling the hole with a regional average and rating as if the data existed. A rating built that way tells you about a country's neighbours, in the tone of a statement about the country. What you want to see is a coverage figure on every rating, floors below which the index declines to publish a number at all, and a visible "insufficient data" state that is allowed to be the answer. An index with no such state is asserting that it can rate everywhere, which is another way of saying it is estimating somewhere and not telling you where.

4. Does statistics capture legal reality?

A country can score moderately on its indicators while being legally difficult or outright prohibited to do business with. Sanctions regimes, severed correspondent banking and government travel advisories are facts about your ability to operate that no weighted indicator average will surface in time, because the data that feeds indices lags the gazette by months or years.

A usable index needs a deliberate, documented gate for this: a rule that comprehensive sanctions force the worst band regardless of score, and that serious legal constraints floor the band, with the override visibly disclosed so you can see what the data alone would have said. If the index treats legal reality as just another indicator, the one fact that should dominate the rating is being averaged away.

5. Does the rating hold still when you quote it?

You will put the rating in a paper, the paper will be read in a month, and the decision will be revisited in a year. If the index recomputes continuously, the number in your paper is unverifiable the day after you wrote it: nobody can check what you saw, and you cannot check what changed. Ratings should come from dated, frozen snapshots, so that "High, as at June" is a statement anyone can reproduce, and so that a change between snapshots is itself information.

The as-at date is not a technicality. It is what makes the rating quotable in a governance process at all.

The same questions, pointed at ours

We publish GeoRisk, a free country risk dataset built for Australian businesses, and it was built by asking these five questions in the mirror. The method is published down to each indicator's source, weight and licence. Bands are quartiles of the scored cohort, cut at the 25th, 50th and 75th percentiles, and the pages say so rather than implying an absolute scale. Nothing is imputed: every rating carries its coverage, categories below their floors are not scored, and in the current snapshot 34 of 218 economies show "insufficient data" because that is the true answer. A documented sanctions gate sits after the scoring, with any override disclosed next to what the data alone computed. And everything is generated from a frozen, dated snapshot that is named on every page and stamped into every export.

None of that makes the ratings correct. It makes them checkable, which is the property the five questions are really testing for. A country rating, ours included, is a place to start looking, a way to structure the questions for people who know the market, and a means of keeping the conversation anchored to evidence with a date on it. Any index that offers you more certainty than that is offering you its confidence, not its method.