As artificial intelligence (AI) is increasingly adopted and integrated into justice systems worldwide, UNESCO has released a new global resource to help judges and judicial professionals understand the technology, its applications and the legal and ethical challenges it presents.

The United Nations Educational, Scientific and Cultural Organization (UNESCO) has introduced the “Global Toolkit on AI and the Rule of Law for the Judiciary“, a practical resource for judges, prosecutors, state attorneys, public lawyers, law universities and judicial training institutions. Funded by the European Union, the toolkit builds on UNESCO’s Recommendation on the Ethics of Artificial Intelligence and aims to support responsible AI use in judicial systems.

“Human in Loop”: Why UNESCO Warns against Automation Bias

In a UNESCO video shared on X, Prateek Sibal, UNESCO’s Programme Specialist for Digital Policies and Digital Transformation, said AI affects legal systems on two levels: its broader societal effects on privacy and discrimination, and its more direct effects on judicial principles like impartiality and fair trial when deployed inside courts. He pointed to “automation bias,” the tendency of judges to defer to an AI system’s output rather than question it, as a central risk the toolkit addresses, and said this is why it emphasizes keeping a “human in the loop” in AI-assisted decisions.

The toolkit responds in part to documented demand. A 2022 UNESCO needs assessment found that 90% of the 32 African countries surveyed requested capacity-building support for their judiciaries on AI, while a separate global survey of judicial actors across 100 countries identified significant gaps in understanding of AI’s implications for the administration of justice.

The toolkit examines how AI technologies, including generative AI, predictive analytics, algorithmic decision-making and machine learning, are being adopted across the justice sector for applications such as legal research, document review, drafting assistance, case management, dispute resolution and risk assessment. It is organized into four modules covering AI fundamentals, its adoption across justice systems, the legal and ethical risks it raises, and its intersection with human rights.

Global Case Studies: How Courts are Using AI

It draws on examples from courts and agencies in Brazil, Argentina, Colombia, the Netherlands, Australia, the United Kingdom and the United States, among others, to illustrate both efficiency claims made for AI tools and cases where their use was later challenged in court or by regulators.

Brazil: AI Streamlines Appeal Screening

On the efficiency side, the toolkit cites Brazil’s VICTOR system, developed with the University of Brasília, which the Supreme Federal Court uses to screen appeals for “general repercussion,” a step the toolkit says previously took court staff around forty minutes per case.

Argentina: Faster Legal Processes with AI

It also cites Argentina’s Prometea system, used by the Prosecutor’s Office in Buenos Aires, where the toolkit reports the time needed to resolve a tender process fell from ninety minutes to one minute, and trial preparation time fell from 167 days to 38 days. Both sets of figures originate from the systems’ developers and partner institutions rather than independent audits, according to the toolkit’s own citations.

Colombia: From Privacy Concerns to ChatGPT in Court

The rollout of AI tools in courts has not always proceeded without difficulty. In Colombia, a pilot of the Prometea system was put on hold after civil society organizations raised concerns about sensitive case data, including information on minors and sexual offence cases, being shared with third-party software developers. The Colombian Constitutional Court subsequently developed a replacement system, PretorIA, using topic modelling rather than neural networks, specifically because it could be fully explained, interpreted and audited, the toolkit says.

The more recent emergence of general-purpose AI tools has introduced a new set of concerns and challenges for courts and judicial actors. In January 2023, Judge Juan Manuel Padilla of Cartagena, Colombia, issued a ruling on a health insurance case involving an autistic child in which he transcribed his ChatGPT exchanges directly into the judgment, with the AI-generated text making up roughly two of the ruling’s seven pages.

Padilla told local radio the tool helped him draft the decision more efficiently and that using it did not make him any less a judge. According to UNESCO’s toolkit, judges in Peru and Mexico also disclosed using ChatGPT to support judicial decisions around the same period.

United States: Fake AI Citations Prompt Judicial Safeguards

In the United States, the risks of unverified AI use in legal filings became concrete in the Mata v. Avianca case. An attorney submitted a brief to a federal court in New York containing six case citations generated by ChatGPT, none of which existed.

Following the disclosure of the fabricated citations in 2023, Judge Brantley Starr of the Northern District of Texas issued a standing order requiring attorneys appearing before him to certify that any AI-generated content in their filings had been verified against authoritative legal sources by a human before submission. In June 2023, Judge P. Kevin Castel imposed a joint US$5,000 penalty on the attorneys involved, finding they had acted in bad faith.

The toolkit also documents cases where algorithmic tools drew formal regulatory or judicial scrutiny. A 2016 ProPublica analysis of the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) tool, used in some Florida courts, found the system was more likely to flag defendants of colour as future criminals than white defendants with comparable records, and that just 20% of those predicted to commit violent crimes actually did so. Northpointe, the tool’s developer, disputed the findings, arguing the ProPublica report was “based on faulty statistics and data analysis and failed to show that the COMPAS itself is racially biased.”

United Kingdom: Algorithmic Risk Assessment under Scrutiny

In the United Kingdom, Durham Constabulary uses an algorithmic tool called the Harm Assessment Risk Tool (HART) to assess reoffending likelihood and inform decisions on whether to offer a suspect a place in a rehabilitation programme as an alternative to being charged. The toolkit notes that HART is deliberately calibrated to underestimate eligibility for rehabilitation, which it says runs counter to the legal principle that ambiguity in criminal proceedings should favor the defendant.

Netherlands: Welfare AI Ruled Unlawful

The toolkit references the Dutch System Risk Indication (SyRI), a welfare-fraud detection system which a Hague court ordered halted in 2020, concluding it violated Article 8 of the European Convention on Human Rights, which protects the right to respect for private and family life.

Australia: Automated Decisions Face Legal Consequences

It also references Australia’s Robodebt scheme, an automated data-matching system used to identify welfare overpayments. According to the toolkit, the system generated many incorrect debt notices by failing to account for irregular incomes, shifting the burden of proof onto welfare recipients.

UNESCO cites estimates that authorities attempted to claim back nearly AUD 600 million through the scheme. The toolkit also notes that, in June 2021, the Federal Court approved an AUD 112 million class action settlement, under which the Commonwealth agreed to pay eligible group members, stop raising or recovering certain invalid debts, and accept declarations that some administrative decisions were not validly made.

Following the 2021 settlement, an appeal commenced in September 2024, with the applicants seeking additional compensation for group members. In July 2025, the parties reached a further AUD 548.5 million settlement, which received final court approval in June 2026. According to Gordon Legal, the overall value returned to group members since the class actions were issued exceeds AUD 2.4 billion.

Separately, the Australian Government released its response to the Royal Commission into the Robodebt Scheme, accepting or accepting in principle all 56 recommendations.

Beyond Courtrooms: AI’s Broader Human Rights Implications

Beyond criminal justice, the toolkit raises concerns about automated gender recognition systems, which it says infer a person’s gender from facial features or legal names rather than self-identification, reducing gender identity to a binary. It notes this can have practical consequences for access to social assistance, housing and healthcare for people with diverse gender identities, and flags that datasets built around a binary gender categorization effectively misidentify individuals whose identities fall outside it.

 Also Read: “AI Cannot Replace the Trained Mind of a Lawyer”: India’s Supreme Court Judge Vikram Nath