Artificial intelligence (AI) is no longer a distant prospect for the insurance industry, it is already embedded in operations, making decisions, and generating litigation. Insurers globally are integrating AI tools into their claims processes, promising faster determinations, improved efficiency, and greater consistency. However, a landmark class action in the United States has brought into sharp focus the risks that arise when algorithmic systems replace human judgment in high-stakes coverage decisions, and the legal consequences that may follow.
For South African insurers considering the adoption of AI in claims processing, the lessons from this case are both timely and instructive.
The US Class Action Case: Estate of Gene B. Lokken v. UnitedHealth Group1
On 13 February 2025, a Federal Court in the United States delivered a significant ruling in a class action (the Lokken case) finding that claims against a medical insurer for breach of contract and breach of the duty of good faith could proceed to trial.
This case was brought by patients (and their families) who had been denied insurance cover for ongoing medical care after hospitalisation. They alleged that these denials led to worsening health outcomes and, in some cases, death.
At the centre of the dispute was an AI tool.
The plaintiffs claimed that the insurer used an artificial intelligence system called the nH Predict AI Model (nHPredict) to make decisions about their cover, when the decisions should have been made by doctors. The insurer denied this. The AI model in question works by comparing a patient's profile against similar cases and then recommending how long their post-hospital care should last. According to the plaintiffs, the insurer treated these AI recommendations as the final word, overriding the opinions of treating doctors.
The plaintiffs further alleged that the insurer was aware of the model’s inaccuracies, citing that more than 90% of claim denials were overturned on appeal, along with more than 80% of pre-authorisation denials.
The plaintiffs' claims are grounded in a straightforward yet compelling premise: The insurer had represented in the insurance contract that claims would be reviewed by “clinical services staff” and “physicians” and there was no reference to the use of artificial intelligence.
On this basis, the Federal Court allowed the plaintiffs' claims for breach of contract and breach of the duty of good faith to go ahead, finding that adjudicating these claims would require only the application of basic contractual principles, specifically whether the insurer adhered to its own process. As at the date of this article, the matter has not been set down for trial. The case remains in the discovery stage, with the Court ordering in March 2026 that the insurer is compelled, in part, to produce documents relating to its development and use of nH Predict.
Lessons from the judgment
The case underscores the importance of ensuring that human (and in this case medical professional) judgment remains part of the claims process, supported by robust safeguards.
It is likely that courts will scrutinise the use of AI tools in claims processing, particularly where such tools result in systematic or automated rejections.
Several key risks emerge:
- Contractual misalignment: If insurance policies promise physician-led reviews, the use of AI as a de facto decision-maker may create exposure for breach of contract claims.
- Governance and oversight: AI systems must support rather than supplant human-led reviews. A lack of meaningful oversight increases litigation risk.
- Transparency and bias: The “black box” nature of AI and the potential for algorithmic bias is a significant concern; particularly in sensitive decision-making processes where the exercise of human discretion remains essential.
South African perspective
Although South Africa has not yet experienced comparable litigation, similar foundational risks exist. As South African insurers explore AI in underwriting, fraud detection, and claims processing, they should remain cognisant of the existing legal frameworks.
South African insurers that deploy AI in claims processing must consider policy wording in light of legal exposures such as those raised in the Lokken case. It must be borne in mind that the Protection of Personal Information Act, 2013 (POPIA) contains provisions relevant to solely automated decision-making that may carry legal consequences for individuals.
Moreover, a failure to address the use of AI in policy documentation and communications may give rise to questions of transparency and good faith under South African insurance legislation and within the Treating Customers Fairly (TCF) framework.
The Lokken case serves as a warning light for the South African market that the use of AI in claims processing gives rise to substantial risks. South African insurers should be considering these risks proactively. The critical question is not whether AI will be used, but whether it will be deployed responsibly and transparently.
We are following the Lokken proceedings closely as they develop. As the case progresses toward trial and further judicial determinations are made, particularly regarding the use and operation of nHPredict.We will publish a further analysis of the implications for the South African insurance market.
1 - The Estate of Gene B. Lokken et al. v. UnitedHealth Group, Inc. et al., Civil No. 23-3514 (JRT/DJF), United States District Court, District of Minnesota, 13 February 2025.
