Providence, RI · Independent public-finance research & analytics

How we build indicators, and what we refuse to build

A tool used in public finance has to survive a question from an auditor, a council member and a reporter. This page describes the process that is meant to make that survivable.

Process

Six stages, applied to every indicator

Borrowed largely from model risk management practice in banking supervision, with the parts built for capital adequacy removed.

01

Question

Every indicator starts from a question a public official actually asks — is this budget structurally balanced, which of these transactions should I read, does this contract belong in the lease register. Methods are selected to answer the question, not the reverse.

02

Definition

The indicator is defined in writing before it is computed: what is included, what is excluded, what basis of accounting applies, and how it behaves at the edges. Definitions are published.

03

Computation

Computed from the entity's own records with every transformation documented and reproducible. Where a judgement is required, the judgement is surfaced for a person rather than embedded in a parameter.

04

Validation

Tested against a labelled sample and against a deliberately naive alternative — a simple threshold, last year's value, a random sample of equal size. Where the naive alternative performs as well, we ship the naive alternative.

05

Disclosure

Results are presented with their limitations attached, and with the specific records and provisions that produced them. Nothing resolves only to a score.

06

Review

Every model carries a validation date, a next-validation date, and a sunset at which continued use requires an affirmative decision supported by evidence of value.

Risk tiering

The tier is a property of the deployment, not the product

The same model used advisorily is low tier; used to hold a payment it is high tier. Frameworks that tier by product rather than by use are gamed within a month. Everything RIGFOA currently offers sits in the first two tiers by design.

Where an entity wishes to deploy an output in the third tier, we will say so, and we will ask to see the governing body approval, the published notice and the appeal route before supporting it.

TierDescriptionApproval expected
LowAdvisory; a human independently reaches the decisionDepartment head
ModerateDetermines what a human reviews, or in what orderFinance director and internal audit
HighBlocks, delays or determines an outcome affecting a third partyGoverning body, with published notice
Questions we are asked

Method, data and governance

Do you use entity data to train models?

No. Entity data is used to compute that entity's own indicators and to establish that entity's own baselines. It is not pooled, not used to train shared models, and not shared with other members. This is a contractual commitment, not a policy statement, and it survives termination.

How do you handle model drift?

Every deployed model has a scheduled revalidation and a set of event triggers: a change of ERP, a change to the chart of accounts, a material change to the underlying model, or an override rate that crosses a threshold. Revalidation means testing against a labelled sample, not asking whether anything changed.

What happens when a reviewer disagrees with the system?

The reviewer's decision stands and is recorded with a reason code. Override data is the single most valuable artefact a deployment produces: a model overridden four times in five is not being used, it is being tolerated, and we would rather find that out from the log than from an annual report.

Do your tools make decisions?

No. Every tool we publish is advisory: it ranks, registers, or checks. Nothing blocks a payment, scores a vendor for award purposes, or determines an outcome affecting a third party. Those uses require governance, a published notice and an appeal route, and are decisions for the entity's governing body rather than for a vendor's product roadmap.

Can we see the detection logic?

Indicator definitions and detection logic are published. Specific numeric thresholds used in fraud triage are configurable by the entity and are not published, for the same reason audit sampling criteria are not published — the general approach is public, the specific selection criteria are not. We say so rather than implying the method is secret.

Who reviews your research?

Research is reviewed internally by staff who did not write it and, for technical accounting positions, by a practitioner with recent preparer or audit experience. Research staff do not report to commercial functions, and no research is reviewed by a client or vendor before publication.

What this is, and what it is not

RIGFOA provides analytical tools and published research. We are not a law firm, an accounting firm, a registered municipal advisor or an audit organisation, and nothing we produce constitutes legal, accounting, audit or financial advice. Our tools are designed to support the judgement of qualified public finance professionals, not to replace it.

Talk to us about your oversight programme

Walk through the platform with your own chart of accounts, or start with the research library. Both routes are free to begin.