Generative AI in pharma: Opportunities and compliance pitfalls

Introduction

Generative AI has revolutionized many industries over the past year. But in the pharma world, where we at Hyperbolic have extensive experience, enthusiasm is often mixed with concern. Can we use ChatGPT to write SOPs? Can we generate report text with AI? What about confidential data?

The answer is: Generative AI has enormous potential in pharma, but there are serious compliance traps to navigate. This article reviews both the opportunities and the risks.

Where generative AI can create value in pharma

1. Documentation and reporting

Pharma produces huge amounts of documentation: Protocols, SOPs, validation reports, deviation reports, batch records, etc.

Opportunities:

• Draft generation: AI can generate the first draft of standard documents based on templates

• Summary: Condense long clinical trial reports into executive summaries

• Translation: Automatic translation of documents into different languages

• Standardization: Ensure consistent terminology and structure

Case from our work: We implemented AI-assisted SOP writing for a pharmaceutical company. AI generates the first draft based on a template and input from SMEs, followed by human review and approval. Result: 60% faster SOP development.

2. Literature review and regulatory intelligence

Pharma must constantly keep up with scientific literature and regulatory changes.

Opportunities:

• Automatic screening of thousands of papers

• Extraction of key information

• Summarization of regulatory guidelines

• Alert for relevant changes

3. Clinical trial design

Opportunities:

• Generating protocol drafts based on previous studies

• Suggestions for inclusion/exclusion criteria

• Power calculations and statistical design support

• Patient recruitment materials

4. Regulatory submissions

Opportunities:

• Generating CTD (Common Technical Document) modules

• Consistency check across submission documents

• Formatting to different regulatory requirements (FDA, EMA, etc.)

The serious compliance traps

Trap 1: Data confidentiality and IP

The problem: Public LLMs like ChatGPT send your data to external servers. All text you enter can potentially be used to train future models.

What you must NEVER share:

• Proprietary formulations or processes

• Clinical trial data

• Patient information

• Unpublished research data

• Commercial information

The solution:

• Use enterprise versions with data protection agreements (ChatGPT Enterprise, Claude for Work)

• Implement private/on-premise LLMs for sensitive data

• Establish clear policies on what employees may/may not share

• Apply data anonymization before AI use

At Hyperbolic, we typically set up a two-layer strategy: Public AI for non-sensitive text, private models for everything else.

Trap 2: Accuracy and hallucinations

The problem: LLMs often "hallucinate" - they invent facts, references, and data that sound plausible but are incorrect.

Example: An AI might generate a reference to a study that never existed. In a regulatory context, this is catastrophic.

The solution:

• Never trust blindly in AI-generated content

• Implement human review of all AI-generated material

• Require source verification for all facts and references

• Use RAG (Retrieval Augmented Generation) to anchor outputs in verified sources

Trap 3: Regulatory acceptability

The problem: There is still no clear guidance from the FDA, EMA, etc. on the use of generative AI in regulatory submissions.

Questions without clear answers:

• Should it be disclosed if AI has written parts of a submission?

• How is AI-assisted work documented?

• Is AI-generated text acceptable in critical documents?

Our approach:

• Be transparent in the audit trail of AI use

• Treat AI as a drafting tool, not final author

• Ensure human oversight and accountability

• Keep an eye on emerging guidance from regulators

Trap 4: Validation and 21 CFR Part 11

The problem: If AI output is used in GxP documents, the system may need to be validated.

Considerations:

• Is the AI system a "computerized system" under Part 11?

• Is validation required?

• How is reproducibility ensured when LLMs are updated?

Our approach:

• Assess GxP impact before implementation

• For GxP-critical applications: Use validated, versioned models

• Implement change control for AI system updates

• Document all AI-assisted processes in QMS

Practical framework for safe use

Step 1: Classify use case

Not all AI use carries the same risk. We categorize:

Low risk (can use public AI with caution):

• Brainstorming and idea generation

• Learning and training materials

• Non-confidential communication

Medium risk (requires enterprise AI):

• Draft generation of standard documents

• Literature search and summarization

• Internal documentation

High risk (requires private/validated AI):

• GxP documentation

• Regulatory submissions

• Clinical data analysis

• R&D documents with IP

Step 2: Choose the right tool

Public LLMs (ChatGPT, Claude): Only for non-sensitive information

Enterprise LLMs (ChatGPT Enterprise, Claude for Work):

• Business associate agreements

• Data not used for training

• GDPR-compliant

• Audit logging

Private/On-premise models:

• Full control over data

• Possibility of validation

• Integration with GxP systems

At Hyperbolic, we help choose and implement the right solution based on the use case.

Step 3: Establish governance

Policies: Who may use AI for what? We typically develop:

• AI Usage Policy

• Approved tools list

• Data classification guide

• Review requirements

Training: All users must understand:

• What they may/may not share

• How to verify output

• When human review is required

• How to document AI use

Audit: Regular review of AI use to ensure compliance.

Case studies

Success: AI-assisted literature review

Challenge: To screen 10,000+ papers annually for relevance to ongoing research.

Solution: Private LLM trained on the company's research focus. Screens papers, extracts key information, flags highly relevant ones.

Compliance: Uses only published, public information. Human scientists review all AI-flagged papers.

Result: 70% time savings in initial screening, more relevant papers identified.

Error: AI-generated SOP without review

What happened: An organization used ChatGPT to generate an SOP without thorough review. The SOP contained procedures that did not match the actual equipment and process flow.

Consequence: Found during audit, classified as a major finding, extensive remediation.

Lesson: AI can draft, but humans must write the final document.

The future: Regulatory development

FDA and EMA are working on AI guidance. We expect:

Likely requirements:

• Disclosure of AI use in submissions

• Validation of AI systems in GxP

• Audit trails for AI-generated content

• Human oversight requirements

Best practice now: Be transparent, document thoroughly, and build processes that can adapt to upcoming requirements.

Conclusion

Generative AI has enormous potential in pharma: Faster documentation, better literature review, more efficient regulatory intelligence. But it must be implemented thoughtfully.

Key principles:

1. Protect confidential data - use the right tools

2. Always verify output - AI can fail

3. Document AI use - be transparent

4. Human accountability - AI assists, humans decide

At Hyperbolic, we help pharma companies navigate this landscape. We combine AI expertise with deep GxP understanding to implement solutions that create value while maintaining compliance.

Contact us to discuss how generative AI can be used safely in your organization.