AI in the Courtroom: Promise, Peril, and the Future of Indian Justice
Following Supreme Court interventions against AI-'hallucinated' judgments, an analysis of how Artificial Intelligence is being integrated into the Indian judicial system, the risks involved, and the rules being framed to govern its use.
The Groundwork: Key Terms, Timeline, and Institutions
To understand the debate surrounding Artificial Intelligence (AI) in the Indian legal system, familiarity with core concepts, the timeline of its adoption, and the key institutions shaping policy is essential.
KEY TERMS
- Generative AI: A category of artificial intelligence algorithms that can create new content, including text, images, and code. Large Language Models (LLMs) like ChatGPT are a prominent example, capable of generating human-like responses and, in a legal context, drafting documents or summarizing case law.
- AI Hallucination: A phenomenon where an AI model produces information that is factually incorrect, nonsensical, or entirely fabricated, yet presents it with confidence. In the judicial context, this manifests as fictitious legal citations or non-existent case precedents.
- Adjudication: The formal legal process of resolving a dispute and passing a judgment. It is the core function of a judge, involving the evaluation of evidence, interpretation of law, and the exercise of judicial discretion to arrive at a decision.
BACKGROUND & TIMELINE
The integration of technology into the Indian judiciary began with basic computerisation and has now entered a phase of intelligent automation, driven by the need to manage immense caseloads.
- 2007: The e-Courts Mission Mode Project is launched in phases by the Government of India to computerise district and subordinate courts, focusing on digitising records and streamlining case management.
- 2019: The Supreme Court launches its neural translation tool, SUVAS (Supreme Court Vidhik Anuvaad Software), to translate judgments from English into multiple regional languages, enhancing access to justice.
- April 2021: Then-Chief Justice of India S.A. Bobde launches the SUPACE (Supreme Court Portal for Assistance in Court’s Efficiency) portal, an AI-driven tool to assist judges with legal research.
- February 27, 2024: In Apoorva Arora v. State of Uttar Pradesh, a Supreme Court bench takes serious note of a trial court relying on AI-generated fictitious case laws, terming the act as potential judicial “misconduct”.
- April 2024: The Supreme Court sets aside orders from the National Company Law Tribunal (NCLT) and its appellate body (NCLAT) in an insolvency case involving SpiceJet. The decision came after discovering the NCLT's judgment was based on fictitious AI-generated legal citations, prompting calls for formal regulations.
INSTITUTIONAL FRAMEWORK
Two primary bodies are at the forefront of navigating the integration of AI into the Indian legal system.
- Supreme Court of India (and its e-Courts Committee): As the apex judicial body, the Supreme Court guides the technological modernisation of the judiciary. Its e-Courts Committee, currently headed by Justice Dhananjaya Y. Chandrachud, is the nodal body for implementing technology, including AI-based tools, within the court system.
- Bar Council of India (BCI): The BCI is a statutory body under the Advocates Act, 1961, that regulates legal practice in India. It prescribes standards of professional conduct for advocates and has been tasked by the Supreme Court with formulating norms and disciplinary actions for lawyers misusing AI.
The use of Artificial Intelligence in justice delivery has moved from a theoretical possibility to a practical reality, bringing both the promise of efficiency and the peril of fundamental error. Recent interventions by the Supreme Court of India have brought this tension into sharp focus, compelling a nationwide conversation on the rules of engagement between human judges and intelligent machines.
What recent events have brought AI in courts into focus?
Two specific cases in 2024 demonstrated that the risk of AI 'hallucinations' entering the official judicial record is no longer hypothetical. In April 2024, the Supreme Court set aside orders passed by the National Company Law Tribunal (NCLT) in an insolvency case involving the airline SpiceJet. A bench of Justices Hima Kohli and Ahsanuddin Amanullah found that the NCLT had relied on completely fictitious, AI-generated legal citations to arrive at its judicial determination. Earlier, on February 27, 2024, a different Supreme Court bench had addressed a similar issue. In the case of Apoorva Arora v. State of Uttar Pradesh, Justices P.S. Narasimha and Alok Aradhe observed that a trial court's reliance on AI-generated content could amount to judicial “misconduct.” The bench described the potential for such AI-induced miscarriages of justice as being akin to methyl isocyanate, the toxic gas behind the 1984 Bhopal tragedy— “invisible, insidious, and catastrophic by the time anyone notices” (Source: Supreme Court Oral Observations).
How is AI currently being used in the Indian judiciary?
The Indian judiciary, burdened by a pendency of over 50 million (5 crore) cases across all levels (Source: National Judicial Data Grid, July 2024), has cautiously adopted AI in strictly assistive roles. The official position, articulated by the Supreme Court's e-Courts Committee, is that AI should augment human intelligence, not replace it. This approach is exemplified by two flagship initiatives. The first is SUVAS (Supreme Court Vidhik Anuvaad Software), an AI-powered tool launched in 2019 for translating judicial documents from English into regional languages to improve accessibility. The second, SUPACE (Supreme Court Portal for Assistance in Court’s Efficiency), was introduced in 2021 as a research assistant for judges. It processes case files to extract relevant facts and suggest precedents, but its output is intended only as a starting point for a judge's own analysis. The rationale is that such tools can reduce time spent on mechanical tasks, freeing up judicial resources for the core function of adjudication.
What are the primary concerns and the Supreme Court's stance?
The central concern is the unreliability of generative AI, which can 'hallucinate' and fabricate information. The Supreme Court has adopted a firm, cautionary stance, drawing a clear line between permissible assistance and the impermissible delegation of judicial functions. The Court has explicitly stated that AI can be an assistive tool but can never replace “independent human reasoning, judicial discretion or professional accountability.” Any judgment influenced by fabricated AI material is considered “no decision in the eyes of law,” establishing that reliance on unverified AI output is a ground for appeal. For advocates, presenting machine-generated fabrications is now being framed as a serious breach of professional ethics, prompting the Court to direct the Bar Council of India (BCI) to formulate disciplinary norms. Beyond hallucinations, experts from institutions like the Centre for Internet and Society (CIS) have raised concerns about algorithmic bias. Since AI models are trained on past legal cases, they can perpetuate historical societal biases, potentially leading to discriminatory outcomes that violate constitutional guarantees under Article 14 (Equality before Law) and Article 21 (Right to a Fair Trial).
What regulations are being considered to govern AI use?
In response to these challenges, a formal regulatory framework is being developed. The Supreme Court's e-Committee is in the process of framing rules to govern the use of AI in the judicial domain. Based on judicial observations, these forthcoming regulations are expected to draw a prohibitive boundary around core judicial functions. The use of AI will likely be explicitly prohibited for the final act of adjudication, the determination of sentences in criminal cases, decisions on bail eligibility, and the evaluation of witness credibility. This 'human-in-the-loop' approach aligns with international frameworks like the European Union's AI Act, which classifies AI systems used in the administration of justice as 'high-risk' and subjects them to stringent requirements of transparency, accuracy, and human oversight. The Indian regulations, combined with the BCI's forthcoming rules for advocates, aim to create a system where AI serves as a tool, but final authority and accountability rest unequivocally with the human judge and lawyer.
The recent judicial pronouncements and the move towards formal regulation mark a pivotal moment for the Indian legal system. The debate over AI in the courtroom is now a pressing governance challenge with profound implications for the rule of law and the fundamental right to a fair trial.
The 2024 NCLT case, in particular, served as a stark demonstration of the real-world consequences of unregulated AI use, transforming a theoretical risk into a documented miscarriage of justice. With generative AI tools becoming universally accessible, the frequency of such incidents is likely to increase, making immediate regulatory intervention and the setting of professional standards an urgent necessity. The forthcoming rules from the Supreme Court's e-Committee provide a critical opportunity for stakeholders—including the judiciary, the bar, technologists, and civil society—to shape a framework that balances innovation with constitutional safeguards.
The near-term trajectory will likely involve a two-pronged evolution. First, the finalisation and adoption of a uniform code of conduct for AI use by the judiciary. Second, the Bar Council of India is expected to amend its professional conduct rules to include specific provisions on the ethical use of AI, with clear penalties for citing unverified or fabricated material. The judiciary will likely continue to invest in vetted, closed-system AI for administrative tasks and research, while shunning open-ended generative models for any function that touches upon judicial reasoning.
The challenge of integrating AI into the justice system requires building not just technological capacity but also 'AI literacy' among judges and lawyers. It necessitates a legal framework that is both technology-neutral and future-proof, ensuring that automation does not erode human agency and accountability. Ultimately, the Indian judiciary's journey with AI will test its ability to innovate while preserving the core tenets of justice—which, as the Supreme Court has affirmed, must be delivered by accountable human beings, not hallucinated by an algorithm.