The use of artificial intelligence (AI) is also revolutionizing the legal field. But these opportunities come with significant risks—one of the most serious being the phenomenon known as “AI hallucinations.” What does this mean, how do such errors arise, and what are the implications for the administration of justice?

What are AI hallucinations, and how do they occur?

AI hallucinations are erroneous or fabricated content generated by generative AI systems such as ChatGPT or Claude, even though they appear credible at first glance. There are many causes:

• Training data: Incorrect, outdated, or incomplete data can lead to incorrect conclusions.
• Training Methods: Technical weaknesses such as “attention glitches” or “exposure bias” cause models to learn incorrect patterns or to adapt too strongly to presumed user expectations.
• Inference: When generating responses, AI models often rely on statistical probabilities, which leads to statements that sound realistic but are incorrect.

These mechanisms are inherent in the system and cannot be completely avoided, as current research and practice show.

Implications for the Administration of Justice

In practice, AI hallucinations can have dramatic consequences. Examples from the U.S. and Israel illustrate this:

• In a U.S. case (T3N, 2025), a top-tier law firm cited non-existent court rulings using Google Gemini, resulting in a $31,000 fine.
• In Israel, a motion to introduce evidence was filed based on fictitious laws generated by an AI system.
• In a case in New York (Golem, 2023), a lawyer filed pleadings that contained precedent cases fabricated by ChatGPT.

Courts rely on accurate and verifiable information. If errors generated by AI go unnoticed, they can lead to miscarriages of justice. Even when errors are detected, they undermine confidence in the legal process.

Responsibility and Liability

The legal responsibility for errors caused by AI is clear:

• According to a ruling by the Kiel Regional Court (2024), companies are liable for the content generated by their systems, even if it was created automatically.
• A disclaimer or reference to automation does not relieve one of the obligation to conduct a careful review.
• Legal principles such as Section 1004 of the German Civil Code (BGB) and the corporate right of personality apply here, as do data protection and personality rights.

This means that anyone who uses AI remains responsible and liable for the results.

Significance for the Administration of Justice (in General)

The judiciary and legal practitioners face the challenge of critically reviewing AI-generated content. While AI systems can speed up research and reduce workloads, their use requires a high degree of skepticism and robust oversight mechanisms. Without these, there is a risk that erroneous content will be incorporated into the legal process without proper scrutiny.

What does this mean specifically for lawyers?

For lawyers, the use of AI means:

• Review is mandatory: All documents created using AI must be carefully reviewed before they are submitted.
• Risk Management: Clear guidelines on the use of AI must be developed, and liability risks must be minimized.
• Training and Awareness: Lawyers must learn to question AI-generated content and not be blinded by the “authority” of AI.
• Technological Support: The use of tools to verify hallucinations, such as Retrieval-Augmented Generation (RAG), and strategies such as chain-of-thought prompting can help minimize errors.