Defined purpose
The question being studied must be stated before the sample is selected.
Measure 4.05 · independence preserved
Legal File 4.05: Analysing Decision Consistency by Sampling
A statistic may flag a question; it must never become an order to the judge.
Detecting a pattern is not scoring a judge
Measure 4.05 proposes analysing decision consistency through sampling. The legitimate purpose may be to identify an unexplained divergence, a procedural weakness, a training need or a data-quality problem.
The mechanism would become dangerous if it produced secret rankings of judges, encouraged statistical conformity or treated a lawful difference of reasoning as an error.
Conditions for legitimate use
Representative sampling
The method must avoid choosing only cases that support a predetermined conclusion.
Contextual review
Differences in facts, procedure and law must be examined before comparing outcomes.
Collective analysis
The primary objective should be system improvement, not individual punishment.
Independent governance
Judicial, statistical, legal and data-protection expertise must be represented.
Published method
Indicators, exclusions, limitations and correction rules must be documented.
What must be prohibited
No hidden league table of judges or courts.
A statistical signal cannot establish professional misconduct.
Consistency analysis must not erase judicial independence or lawful differences.
Personal data and professional evaluation require a clear legal basis and safeguards.
Sampling protocol
- Define the legal or procedural question.
- Describe the population of cases and the sampling method.
- Remove or protect identifying data when appropriate.
- Review relevant factual and legal context.
- Submit findings to qualified human analysis.
- Publish aggregate results, limitations and corrective action.
Core safeguard. A statistical anomaly is a question to investigate, not proof that a judge or court has acted improperly.