LLM let you down? Human frauds might win compensation for AI mis-selling

Disputes arising from the outputs of large language models and generative artificial intelligence are inevitable. At the moment lawyers seem to be the canary in the coal mine. Flaws in legal research and legal submissions prepared with the input from these tools are now being exploited by opponents in litigation and publicly exposed in court judgments. An initial trickle of occurrences has not abated. There are now nearly 1900 reported instances of AI hallucinated cases in legal proceedings globally. While obviously having direct impacts as between lawyer and client, wider reputational and professional consequences are also following.

The challenges the legal profession is facing with its adoption of these tools will inevitably affect other professionals – the adversarial nature of legal proceedings just means lawyers’ mistakes are being exposed first. 

How does AI marketing create legal risk?

Disputes in this area will necessarily focus on why. Generative AI excels at reproducing human writing patterns based on examples in its training data, yet it is not able to meaningfully verify the accuracy of any content it produces.  Even if it purports to cite its sources, the sources may not actually exist or may not support the statement for which they are being cited. 

The inherent limitations of generative AI are only becoming appreciated well after widespread adoption and use has occurred. Companies globally which took steps to reduce their overall employee headcount in anticipation of realising significant benefits from deploying such technology have begun to backtrack on those AI-driven redundancies.

This is where the human element appears: exaggerated marketing claims about the abilities of the tool, sales patter down-playing known limitations, half-truths being told in order to give a reassuring answer to a prospective customer’s questions, failing to correct an obvious misunderstanding, recommending use cases that exceed the actual capabilities of model, can all be potential bases for advancing a successful claim against software vendors and other advisers with an interest in hard-selling prospective customers the latest model.

This “AI mis-selling” will not necessarily lead to compensation. Claims against software vendors are notoriously difficult to successfully make. Individual consumers must grapple with unequal means and often a one-sided “clickwrap” contract containing terms highly favourable to the vendor. Although the balance of power is slightly shifted in a business context, despite larger sums and a negotiated contract being involved, disputes between a software vendor and customer still rarely reach a court. Often the extensive contractual protections favouring the vendor are effective at limiting or excluding the vendor’s liability to their customer altogether, making the game not worth the candle.

Why a claim in fraud might work

However, those contracts cannot exclude liability for fraud. Success in a fraud claim (such as fraudulent misrepresentation or the tort of deceit) often means restoring the innocent party to the position they were in before the fraud occurred. In the context of a claim against a software vendor, this might mean that all licence fees paid to date for use of the large language model are refunded and, in addition, compensation for any additional losses suffered as a result. 

While identifying a knowingly false statement in commercial marketing about generative AI’s capabilities would be highly probative evidence to advance a claim based on fraud, identifying a knowingly false representation is likely to be extremely difficult. We think one rarely considered aspect of fraud claims – the possibility of a claim based upon implied representations – will be the likely genesis of future AI mis-selling claims. 

Formulating a fraud claim in this way has gained the most traction in claims brought by investors in complex financial products. Those claims have arisen in contexts where contractual protections are similarly comprehensive and there has been a need to succeed in a fraud claim to unlock meaningful compensation. For example, the decision in Lorely Financing (Jersey) No 30 Limited v Credit Suisse Securities (Europe Limited) & Ors [2023] EWHC 2759 (Comm) ruled on a deceit claim based, in part, on implied representations as to honesty and implied representations as to the underlying quality and characteristics of underlying loans securitised into residential mortgage-backed securities. However, just because the argument is run does not mean that such claims are necessarily successful at trial (as occurred in Lorely).

Where to from here?

In the context of AI mis-selling, something at least seems to have gone badly wrong with customer and user understanding of the capabilities of commercialised large language models being marketed as having “artificial intelligence”. In particular, the occurrence of errors, omissions and hallucinated outputs and the need to verify the output seem to have been drastically underappreciated by users of those tools. Express disclaimers in the outputs about the possibility of the mistakes and the need to verify the output against the source are seemingly ignored or misunderstood. Understanding what is actually known by the creator of the tool, what was said (or left unsaid) in marketing literature and what was done by the innocent party in reliance upon those representations will be critical to success in any AI mis-selling claim.

While AI mis-selling claims will each ultimately be fact dependent, the smoke from the repeated reoccurrence of issues arising from use of generative AI suggests there might just be fire. In circumstances where contractual terms excluding liability for pre-contractual statements are likely to be effective, a future-facing firm might make all the difference.