The First FDA Warning Letter Referencing AI: Why the Failure Was Not the Technology, but the Lack of Control

The use of artificial intelligence is increasingly reaching regulated areas of pharmaceutical manufacturing, quality assurance, and documentation. What has so far primarily been discussed as a matter of efficiency is now gaining a clear regulatory dimension.

In a Warning Letter dated April 2, 2026, the FDA explicitly criticized the inappropriate use of AI agents in a GMP environment for the first time. Specifically, the case involved AI-generated specifications, procedures, and production and control records that were apparently incorporated into GMP-relevant processes without sufficient human review.

The case is noteworthy because the FDA did not criticize the use of AI itself. Rather, the issue was that AI outputs were used without appropriate control, review, and approval. The authority made it clear that outputs or recommendations from an AI agent must be reviewed and approved by an authorized representative of the Quality Unit when they are used in cGMP activities.

This case marks a turning point. It is not the use of AI that is being criticized, but its uncontrolled use in quality-relevant processes.

What the FDA Specifically Criticized

The Warning Letter is based on an inspection conducted in October 2025 and documents several serious cGMP violations. In addition to more traditional deficiencies, such as inadequate sanitary conditions and missing testing, the FDA explicitly highlighted the handling of AI for the first time.

From a quality perspective, two key deficiencies can be identified.

First, there was apparently no review of AI-generated documents. According to the FDA, documents such as drug product specifications, procedures, and production and control records were created using AI and then incorporated into regulated operations. The FDA considered the lack of review of these documents to be a violation of 21 CFR 211.22(c), which describes the responsibility of the Quality Unit for reviewing and approving quality-relevant activities.

Second, the FDA criticized excessive reliance on AI. This became particularly evident in the lack of process validation. The company reportedly stated to the FDA that it had not been aware of the corresponding legal requirement because the AI agent used had not pointed it out. The FDA viewed this as a violation of 21 CFR 211.100.

This highlights a central point: regulatory responsibility cannot be delegated to an AI system. AI can support, prepare, structure, or provide indications. However, it cannot assume professional responsibility and cannot replace quality-relevant approval.

AI Is a Tool – Not an Approval Authority

The case does not show that AI is fundamentally unsuitable for use in GMP environments. Rather, it shows that AI, like any other tool, must be used in a controlled manner. Especially in regulated environments, the usability of a system is not determined by its technical performance alone, but by its integration into clear processes, responsibilities, and evidence.

An AI-generated document draft can be helpful. It can save time, reduce repetition, and serve as a starting point for expert work. However, it becomes problematic when such a draft is transferred into the quality management system without qualified review. At that point, the result is not efficiency, but a compliance risk.

The Quality Unit remains the decisive authority. It must ensure that quality-relevant documents, processes, and decisions are reviewed, traceable, and approved — regardless of whether the content was created by a human or supported by AI.

The Real Pain Point: Shadow AI in Regulated Environments

Many companies are currently in a transitional phase. AI tools are available, easy to access, and promise fast results. At the same time, binding rules for their use in GMP- or GxP-related processes are often still missing.

This is where the actual risk arises: the problem is not the controlled use of AI, but its informal use outside established quality structures. When employees use AI without clearly defined rules on which use cases are permitted, which data may be processed, who reviews the results, and how the use of AI is documented, “shadow AI” emerges.

For regulated companies, this is particularly critical. GMP compliance is based on traceability, accountability, and controlled processes. AI must not bypass these principles; it must be integrated into them.

Where AI Can Provide Meaningful Support

When used correctly, AI can make a significant contribution in regulated environments. Meaningful use cases include, for example:

• Drafting SOPs, specifications, or test plans
• Supporting research and summarizing regulatory requirements
• Preparing risk analyses and gap assessments
• Creating review checklists or test cases
• Supporting code generation and technical documentation
• Optimizing, translating, and structuring language and documents

However, one point is crucial: these results remain preparatory work. They must be professionally assessed, reviewed, and approved before they are incorporated into regulated processes.

Where Is Human Review Mandatory?

• Approval of any GxP-relevant documentation
• Code reviews and technical verification
• Assessment of regulatory compliance
• Decisions on software validation and qualification
• Risk assessments and their final classification
• Release/deployment approvals and quality-relevant decisions

As a general rule, every AI-generated result that flows into a regulated process must be reviewed by qualified personnel.

Particularly critical are specifications, SOPs, validation documents, traceability matrices, risk assessments, test protocols, and approval decisions.

What Companies Should Define Now

The FDA case makes it clear that companies need clear guardrails for the use of AI in GxP environments. At a minimum, these include:

Clear Delimitation of Use Cases

Companies must define where AI may provide support and where it may not. Not every use case is suitable for AI — especially not without a risk assessment.

Responsibilities and Approval Processes

It must be clearly regulated who reviews, assesses, and approves AI-generated results. Responsibility remains with the company and with the relevant roles in quality management.

Human-in-the-Loop as a Mandatory Principle

AI outputs must not be automatically incorporated into GxP-relevant processes. A competent human review is required.

Documentation and Traceability

The use of AI must be traceable. This includes documenting when AI was used, for what purpose it was used, and how the results were reviewed.

Risk Assessment and Validation Strategy

For specific AI use cases, companies should assess the risks associated with their use and determine which controls are required.

Employee Training

Anyone using AI must understand the limits of the technology. This includes hallucinations, incomplete responses, missing context, and the risk of seemingly plausible but technically incorrect results.

The Position of the anic GmbH

The anic GmbH sees AI as a valuable tool for supporting development, documentation, and research. At the same time, the principle is clear: AI replaces neither expertise nor responsibility. A result only becomes quality- or process-relevant once it has been reviewed and approved by qualified personnel.

This basic approach is already reflected in existing control mechanisms. AI-generated code must not be adopted without review. Code reviews are performed manually by qualified reviewers, and documents follow a multi-stage approval process — regardless of whether content was created with or without AI support. Confidential information, personal data, and company IP may also only be processed in approved and licensed AI tools.

In this way, AI is not excluded, but made usable in a controlled manner. The decisive difference lies in process integration. AI must not operate outside the quality management system, but must be embedded into existing review, approval, and documentation processes.

From a Technology Question to a Governance Question

The first FDA Warning Letter explicitly referencing AI sends a signal to the industry. Regulators will not reject AI across the board. However, they will expect companies to control its use. Anyone using AI in regulated processes must be able to demonstrate that results have been reviewed, decisions are traceable, and responsibilities are clearly defined.

This shifts the discussion. It is no longer only about which AI tool is used. What matters is whether companies have created structures that enable safe and audit-ready use.

For pharmaceutical, biotechnology, and other regulated companies, this means that AI projects must not be treated in isolation as purely efficiency-driven initiatives. From the outset, they must be aligned with QA, IT, specialist departments, and compliance.

Conclusion

The case shows that compliance does not fail because of AI, but because of a lack of control. AI can be valuable in GMP environments — but only if it is understood as a supporting tool. As soon as AI outputs are adopted without review or regulatory decisions are effectively delegated to a system, a significant risk arises.

The future in regulated environments is therefore not “AI or no AI.” The future is controlled AI with clear responsibilities, documented reviews, traceable approvals, and a Quality Unit that actively fulfills its role in digital transformation.

For the anic GmbH, this is precisely the decisive approach:

"AI is a valuable tool in regulated environments — provided that its outputs are not adopted blindly, but are professionally reviewed and taken responsibility for by authorized personnel. AI itself is not the problem, but uncontrolled use without human oversight. At anic, the principle is clear: AI supports, while humans decide and take responsibility."