The use of artificial intelligence (AI) is also revolutionizing the legal sector. However, the opportunities are accompanied by considerable risks - one of the most serious being the phenomenon of so-called AI hallucinations. What does this mean, how do such errors arise and what consequences does this have for the legal process?

What are AI hallucinations and how do they occur?

AI hallucinations are erroneous or fictitious content created by generative AI systems such as ChatGPT or Claude, even though they appear credible at first glance. There are many causes:

• Training data: Incorrect, outdated or incomplete data can lead to false conclusions.
• Training methods: Technical weaknesses such as "attention glitches" or "exposure bias" lead to models learning incorrect patterns or adapting too strongly to assumed user expectations.
• Inference: When generating answers, AI models often rely on statistical probabilities, which leads to realistic-sounding but incorrect statements.

These mechanisms are inherent to the system and cannot be completely avoided, as current research and practice show.

Consequences for the determination of rights

In practice, AI hallucinations can have dramatic consequences. Examples from the USA and Israel illustrate this:

• In one US case (T3N, 2025), an elite law firm used Google Gemini to cite non-existent judgments, resulting in a $31,000 fine.
• In Israel, a motion for evidence was filed with fictitious laws generated by an AI system.
• In one case in New York (Golem, 2023), an attorney filed briefs containing precedents invented by ChatGPT.

Courts rely on precise and verifiable information. If AI-generated errors go unnoticed, they can lead to wrongful convictions. Even when errors are noticeable, they undermine trust in the legal process.

Responsibility and liability

The legal responsibility for errors caused by AI is clear:

• According to a ruling by the Regional Court of Kiel (2024), companies are liable for the content generated by their systems, even if it was created automatically.
• A disclaimer or reference to automation does not release the user from the obligation to check the information carefully.
• Legal bases such as Section 1004 of the German Civil Code (BGB) and corporate personality rights apply here, as do data protection and personality rights.

This means that anyone using AI remains responsible and liable for the results.

Significance for the determination of law (general)

The judiciary and legal practitioners are faced with the challenge of critically reviewing AI-generated content. While AI systems can speed up research and reduce the workload, their use requires a high degree of skepticism and control mechanisms. Without these, there is a risk that erroneous content will flow into the legal process without reflection.

What does this mean in concrete terms for lawyers?

For lawyers, the use of AI means:

• Control is mandatory: All documents created with AI must be carefully checked before they are submitted.
• Risk management: Clear guidelines on the use of AI must be developed and liability risks minimized.
• Training and awareness-raising: Lawyers must learn to scrutinize AI-generated content and not be blinded by the "authority" of AI.
• Technological support: The use of hallucination checking tools, such as Retrieval Augmented Generation (RAG), and strategies such as chain-of-thought prompting can help minimize errors.