SETUMO MOHAPI | AI lessons for justice from the Andrea Johnson debacle

Artificial intelligence (AI) systems such as predictive robotic actuators and large language models are, in essence, trained computational inference machines. Human beings, possessing ordinary intelligence, are inference machines also. Individuals and communities continually observe, process information, learn and make decisions from experience. The criminal justice system likewise performs inference: complaints generate suspicion, investigations test it, and arrests or prosecutions may follow. Because the system exercises coercive public power, every decision must be grounded in evidence.

The criminal justice system performs inference continuously. Complaints generate suspicion, investigations, and, ultimately, arrests or prosecutions. Because it exercises coercive public power, every decision must be grounded in evidence.

The controversy involving Investigating Directorate Against Corruption (Idac) head Andrea Johnson and police crime intelligence official Brig Dineo Mokwele illustrates what can happen when decisions are made on incomplete information.

In June last year, Idac arrested Lt-Gen Dumisani Khumalo and six co-accused, including Mokwele, on fraud and corruption charges linked to her appointment in crime intelligence. The National Prosecuting Authority publicly characterised the matter as the appointment of an allegedly unqualified civilian to a senior post.

During Johnson’s evidence before the Madlanga commission, however, she acknowledged relying on incorrect information supplied by her team and conceded she did not know the full extent of Mokwele’s qualifications. Reporting from the hearing identified qualifications in industrial engineering, mechatronics and several technical fields relevant to the position. Johnson subsequently withdrew her description of Mokwele’s appointment as a “token appointment” and apologised.

The sequence resembles a poorly governed AI system. A label was generated and treated as fact before the underlying evidence had been properly tested.

In machine learning, decisions are based on features — pieces of information supplied to a model. Recruitment works in much the same way. Qualifications, experience and job requirements form the feature set on which decisions are made.

The concern raised by the Mokwele matter is that some features appear to have carried more weight than others. Public discussion focused on her background outside the police service and her previous employment at BMW, while insufficient attention appears to have been paid to her qualifications and the requirements of the post.

“Former BMW employee” became a suggestive classification. “Unqualified civilian” became the label. “Token appointment” became the narrative.

The word “token” is particularly revealing. It presupposes a lack of merit and turns an inference into a premise before the evidence has been fully examined.

Poorly designed classifiers make similar mistakes. They treat proxies as facts: a postcode becomes a measure of creditworthiness, a name a measure of organisational fit, and a private-sector career evidence of unsuitability for public service.

Commissioner Sesi Baloyi questioned the factual basis of the corruption theory. While the exchange did not constitute a finding, it exposed what appears to be the central governance problem: a coercive decision without a clear chain from evidence to conclusion.

AI governance teaches that high-risk systems require accountability, transparency and mechanisms to correct errors. Comparable controls belong in investigative and prosecutorial systems.

Every authorisation involving arrest or prosecution should contain a claim-to-evidence matrix. Each factual assertion should identify its original source, its verifier and any contradictory material. The file should record the applicable legal threshold, why it is considered satisfied, the likely consequences of a false positive and the evidence still outstanding.

An independent reviewer should construct the strongest innocent explanation consistent with the known facts. Scope changes should generate a written record. Additional suspects should enter the process only through an identifiable evidential link. Uncertainty should be recorded rather than converted into confident prose.

An effective correction protocol is equally important. When a material factual premise collapses, the event should be treated as a serious institutional incident. Related cases and court documents should be reviewed. Affected persons should be notified where legally appropriate. The institution should preserve a record of the error, identify its cause and determine whether the same investigative team or information source produced similar defects elsewhere.

Safety-critical engineering already applies this principle. A structural engineer does not approve a bridge because a subordinate says the load calculations probably exist. A medical-device regulator does not accept a verbal assurance that adverse events were checked. A prosecutor exercising coercive state power requires at least comparable evidential discipline.

Some of the most dangerous inference machines contain no code. They are institutions and human beings whose assumptions remain hidden, whose inputs are undocumented, whose confidence is uncalibrated and whose outputs carry badges, court dates and headlines.

The question for AI safety is simple: who is accountable when the inference machine is wrong?

The South African question is equally direct: who is accountable when poorly tuned inference machines embedded in recruitment processes are conditioned by the presumption that black African candidates are likely to steal and then malfunction as publicly as the system exposed through Johnson’s evidence? Responsibility rests with those who deny the material and continuing effects of prejudice against black South Africans, disclaim responsibility for the constrained upward mobility of the disadvantaged, and nevertheless sustain evaluative systems primed to question the capability and integrity of the black African child at first sight.

Those machines can malfunction too. In public.