LLMs (Large Language Models) that power ChatGPT and the various “co-pilot” products in the market are very powerful tools to use in healthcare.

Healthcare organizations can use these tools to make their people more effective and to speed up various processes in their organization.

However, the LLM vendors’ Terms of Use can allow them to use any content provided by your users for various purposes. This opens up the possibility that your company confidential information may become available to other companies including your competitors.

While one option is to ban all use of LLM technologies in your company, this approach will prevent your organization from taking advantage of this technology.

A better approach is to define and communicate a clear policy around the usage of ChatGPT, Copilot and other LLM technologies in your organization. This will enable your organization to leverage this technology which protecting against your sensitive data becoming available to others.

What is Content You Have to Worry About?

Content when using an LLM consists of three parts:

1. Question that was asked by the user plus any instructions that were included (Called Prompt in LLM lingo).
2. Knowledge provided to LLM to answer. This can be included in the instructions or uploaded to LLM to use in the form of documents etc.
3. Output of the LLM. This is the reply from the LLM which may includes parts of the Prompt and knowledge that you provided to LLM.
   • NOTE: This may NOT use exact text from the Prompt and Knowledge since the LLM can rephrase knowledge.

Why Do You Need a Policy?

Let’s review the terms of use of some of the popular Co-pilot and LLM solutions in the market.

1- OpenAI ChatGPT Personal

OpenAI (the company behind ChatGPT and the underlying GPT LLM) owns the right in their terms of use to use any content you or your employees provide it.

Our Use of Content. We may use Content to provide, maintain, develop, and improve our Services, comply with applicable law, enforce our terms and policies, and keep our Services safe.

https://openai.com/policies/terms-of-use (Effective: January 31, 2024)

2- OpenAI ChatGPT Enterprise or Team

The ChatGPT Enterprise or Team editions add terms of use that restricts OpenAI from using any content you or your employees provide it for improving their services.

We will process and store Customer Content in accordance with our Enterprise privacy commitments. We will only use Customer Content as necessary to provide you with the Services, comply with applicable law, and enforce OpenAI Policies. We will not use Customer Content to develop or improve the Services.

https://openai.com/policies/business-terms (Effective: November 14, 2023)

“ChatGPT Enterprise is also SOC 2 compliant and all conversations are encrypted in transit and at rest.”

https://openai.com/blog/introducing-chatgpt-enterprise (Updated August 28, 2023)

3- Microsoft Copilot

Prompts, responses, and data accessed through Microsoft Graph aren’t used to train foundation LLMs, including those used by Microsoft Copilot for Microsoft 365.

https://learn.microsoft.com/en-us/microsoft-365-copilot/microsoft-365-copilot-privacy#how-does-microsoft-copilot-for-microsoft-365-use-your-proprietary-organizational-data (Updated 1/31/2024)

However notice the language that is limited to training foundation LLMs. This means Microsoft may (pursuant to other terms) be able to use your data for other purposes than training foundation LLMs.

4- Github Copilot

Github Copilot terms prohibit them from storing or using your content for any purpose other than answering your prompt.

Github Copilot sends an encrypted Prompt from you to GitHub to provide suggestions to you. Except as detailed below, Prompts are transmitted only to generate Suggestion in real-time, are deleted once Suggestions are generation and are not used for any other purpose. Prompts are encrypted during transit and are not stored at rest without your permission.

https://github.com/customer-terms/github-copilot-product-specific-terms (Version: January 2024)

5- Atlassian Intelligence

The data you submit and the responses you receive via Atlassian Intelligence are not used to fine-tune or improve OpenAI’s models or service. Each data request is sent to OpenAI individually, over an SSL encrypted service, to process and send back to Atlassian.

We process your inputs to provide you with the outputs you requested. We do not use your input or output for any other purpose.

https://www.atlassian.com/trust/atlassian-intelligence (Checked on February 12, 2024)

How To Draft Your Policy

I recommend that you classify the usage of LLMs in your organization into three buckets:

• Level 1: Where ONLY data available in the public domain is provided to the LLM.
• Level 2: Where Company Confidential data is provided to the LLM.
• Level 3: Where PHI or Customer sensitive data is provided to the LLM.

Of course, depending on the needs of your organization you may add additional levels.

Level 1- Where ONLY data available in the public domain is provided to the LLM

In this case users are only providing content that is available in the public domain.

Examples:

1. “How do you find NDC code in FHIR?”
2. “Write me a summary of the CMS Patient Interoperability Rule”.
3. “What is the annual revenue of Cigna”

This can be allowed pretty freely and your employees can use personal ChatGPT accounts.

However ChatGPT Enterprise and Team editions provides a lot of enterprise tools so unless cost is an issue, I would recommend requiring everyone to use ChatGPT Enterprise or Team edition.

Level 2: Where Company Confidential data is provided to the LLM

This is content where company confidential data is included.

Examples:

1. Pasting your company software code in ChatGPT to convert. 
2. Pasting your company’s internal documents in ChatGPT to summarize.
3. Using Copilot on a company presentation.

You should require that users use the Enterprise or Team version of ChatGPT. As mentioned above, these editions adjust the terms to prohibit OpenAI from using your content.

OpenAI ChatGPT Enterprise also provides SOC2 compliance and encryption so that is an added level of protection in case OpenAI gets hacked.

Level 3: Where PHI or Customer sensitive data is provided to the LLM

This is content that may include PHI (Protected Health Information), PII (Personally Identifiable Information) or customer sensitive data.

Examples:

1. Parts of a patient record
2. Customer lists including names and addresses
3. Contracts with customers

You should not use ChatGPT Personal AND, if you’re a healthcare organization, you should sign a BAA (Business Associate Agreement) with your vendor.

Maintaining a list of approved tools

New products and tools are coming onto the market all the time. In addition, the current vendors continue to change their terms of use.

I recommend maintaining an easily available list of approved tools that anyone in your organization can use to know which tools to use. This list should include the Levels of Usage approved for each tool.

In addition, you should define a process where someone can submit a new tool for consideration. Then your compliance people can review the Terms of Use to decide whether the tool is added to approved list and what Levels of Usage are approved.

Having this process will help prevent people from just using tools because they don’t know how to get it approved.

Policy for use of the output of LLMs

In addition to defining a policy of which tools to use for what level of usage, you should also define a policy for using the output of LLMs.

Whenever possible, require that any verbatim use of the output of an LLM includes a notice that this content was generated using AI. This is not required by the US government yet but EU is headed in this direction and many companies in the US are voluntarily doing this. This positions you as a responsible user of this technology.

When the output of the LLM is used, require that a form is included that says who, if any, verified the output of the LLM. LLMs are subject to hallucinations and before relying on the output you would like a human being to verify the output.

Summary

The tips above can get you started on drafting your company policy on LLMs.

Each organization is different so you should involve the appropriate security and compliance officers in your process early.

You can use the information above to give them confidence that you are sensitive to their concerns and are proposing a nuanced policy to balance the needs of the business while ensuring compliance.