When an AI hallucination enters an executive order

On 2 September 2026, the Supreme Court encountered a problem concerning the use of Artificial Intelligence (AI) by a Customs officer who relied on non-existent cases and fake citations to impose a penalty of ₹425.28 crore. The error, this time, was outside the court and tribunal system. 

An Additional Commissioner of Customs had imposed the penalty on Vijay Ghanshyam Gadiya, after he found that a consignment of natural diamonds had been misdeclared as lab-grown diamonds to secure a lower tariff. The penalty was challenged before the Gujarat High Court. Gadiya approached the Supreme Court after the High Court dismissed his appeal in January 2026.

A Division Bench of Justice Dipankar Datta and Justice Sheel Nagu did not examine the merits of the alleged misdeclaration. Instead, it identified three kinds of errors in the customs officer’s order: some of the cited cases did not exist, some carried fake citations, and genuine judgements were cited for holdings that they did not lay down. The Bench described the last category as an apparent AI hallucination.

The Court set aside both the customs order and the High Court’s decision. It directed a different officer of the same rank to decide the proceedings afresh. The appointing authority was given the discretion to consider action against the officer who authored the defective order.

From the NCLT to the Customs Department

In July 2026, the Supreme Court had confronted a similar issue in Pooja Ramesh Singh v Jammu and Kashmir Bank Ltd. where it adopted a “zero-tolerance” standard to AI-generated judgements. The case concerned the use of six purported precedents by the National Company Law Tribunal (NCLT). These included wholly fabricated decisions, incorrect citations and non-existent paragraphs attributed to genuine judgements.

The Supreme Court held that any decision contaminated by even a small amount of fake or hallucinated legal material must be set aside. This was irrespective of whether the false material had a direct or indirect bearing on the outcome.

The Court added that AI may help with research and improve efficiency, provided that a human retains control of adjudication and verifies its output. In Gadiya, Justice Datta reiterated the distinction: AI may serve as “training wheels”, but placing it in the “pilot’s seat” would be dangerous.

The two cases show that the risk travels across institutions. Pooja Ramesh Singh involved a tribunal and its appellate body. Gadiya concerns a departmental officer exercising statutory adjudicatory powers.

Section 114 of the Customs Act, 1962 states that a person may face penalty for any act or omission that results in the confiscation of goods. The power to adjudge confiscations and penalties is with the custom officers as per Section 122. Section 122A requires the adjudicating authority to offer a hearing when the party seeks one.

These officers form the executive branch and exercise quasi-judicial functions when deciding liability and imposing penalties. Therefore, their orders must contain applicable statutes and authentic precedents. An invented judgement acts as more than a faulty footnote and obscures the basis of the decision. The affected person has no opportunity to answer.

The uncertain edge of the draft regulations

On 3 June, the Supreme Court published the Regulations for Use of Artificial Intelligence in Courts, 2026. The regulations are built around human primacy, transparency, accountability, data protection and judicial independence.

It defines “Court” to broadly include the Supreme Court, High Courts, district courts, tribunals and statutory commissions performing adjudicatory functions.

The regulations also state that while AI-assisted legal research, precedent retrieval, citation verification and document summarisation are permissible, they simultaneously require human supervision. Any officer using AI must take responsibility for the resulting decision. Judicial outcomes cannot be reached solely through algorithmic decision-making.

The Regulations, however, are organised around courts, tribunals and statutory commissions. It adds an oversight mechanism on national tribunals and commissions by setting up the Supreme Court’s AI Committee and state-level committees at the relevant High Courts. It does not identify an equivalent supervisory mechanism for adjudication conducted within government departments like the customs officers.

Gadiya demonstrates why the boundary requires clarity. The injury caused by a fabricated precedent does not change with the designation of the official.

Verification as part of giving reasons

A reasoned order tells the affected person why they won or lost, permits an appellate body to test the decision and restrains arbitrary exercises of power. AI-generated material can weaken each safeguard while creating the appearance of elaborate legal reasoning.

Verification must therefore become part of the process. Every cited authority should be traced to an authenticated judgement database. The relevant paragraph should be checked against the proposition for which it is used. Where AI materially assists with research or drafting, the officer should record its use and the verification undertaken.

Departmental frameworks could also identify approved tools, prescribe disclosure requirements and create a process for reporting AI-related errors. Training must cover subtler hallucinations, including genuine cases paired with fabricated passages. These are harder to detect than a wholly invented citation because the case name itself inspires confidence.

The next stage of AI governance in adjudication must look beyond the courtroom. Gadiya shows that human control, source verification and accountability are requirements of fair decision-making wherever the State determines a person’s rights or liabilities.