Does Your AI Vendor Train on Your Data? Reading the Terms

Does your AI vendor train on your data? Read the terms for the plan each person on your staff signs in to, because one vendor can give two different answers. On OpenAI's plans for individuals (Free, Plus and Pro), OpenAI may use your content to train its models. On its business products, OpenAI does not train its models on your data by default, and a company with some staff on personal accounts and some on business seats is under both rules at once.

What the Terms Say, Vendor by Vendor

OpenAI's no-training default covers inputs and outputs from ChatGPT Business, ChatGPT Enterprise, ChatGPT for Healthcare, ChatGPT Edu, ChatGPT for Teachers and the API Platform. For the API, the promise covers only data sent after March 1, 2023. Free, Plus and Pro are not on the list. If a staff member pastes a client proposal into a personal Plus account, the terms that allow training apply, whatever business plan the company pays for.

Anthropic draws the line in the same place. By default, it does not use inputs or outputs from its commercial products, such as Claude for Work, the Anthropic API and Claude Gov, to train its models. The consumer plans work the other way. On Claude Free, Pro and Max, Anthropic uses chats and coding sessions for training when the user chooses to allow it, so each person's setting decides the outcome.

Under Microsoft's enterprise data protection for Copilot, prompts, responses and data accessed through Microsoft Graph are not used to train foundation models. Copilot's web search queries are not shared with advertisers and do not train Microsoft's foundation large language models (LLMs). Microsoft Copilot and Microsoft Copilot Chat offer the same enterprise terms as Microsoft 365 commercial offerings.

Google puts its promise in a numbered contract clause. Section 18 of the Google Cloud service terms, titled Training Restriction (it was Section 17 in earlier versions), bars Google from using Customer Data to train or fine-tune any AI/ML models without the customer's prior permission or instruction. The Google Workspace service-specific terms carry their own Training Restriction section. Google does not use Workspace data to train or improve the underlying generative AI and large language models that power Gemini, Search and other systems outside of Google Workspace without permission. For Gemini in Workspace, content is not human reviewed or used for model training outside your domain without permission.

The Exceptions Hidden in "By Default"

OpenAI and Anthropic both put "by default" into their business promise, and both say what overrides it. OpenAI may train on business data that the customer has explicitly opted in to share, for example through its opt-in feedback programs. For ChatGPT Business, Enterprise and Edu, it does not use prompts, responses or files to improve its models unless the organization has signed a separate agreement allowing it. A thumbs up or thumbs down on a response in those plans does not permit training on that conversation. At Anthropic, the exception covers anyone who reports feedback or bugs, for example with the thumbs up/down button, or who otherwise chooses to allow use of their data. Anthropic may then train on those chats and coding sessions.

The thumbs buttons are where the two vendors part ways. A thumbs-down on a poor answer in Claude for Work counts as feedback, and Anthropic keeps feedback conversations in its secured back-end for up to 5 years, so one click can put a client conversation into its training data. Owners of Claude Team and Enterprise plans can switch the buttons off with the Rate chats setting under Organization settings > Data and Privacy. On ChatGPT business plans a rating does not permit training, and admins can still turn response feedback off under Workspace settings > Allow response feedback in the Admin Console.

Personal accounts come with their own exceptions and switches. On Claude Free, Pro and Max, Anthropic can also use conversations that are flagged for safety review. Incognito chats are not used to improve Claude, even with Model Improvement turned on in Privacy Settings. To opt out on ChatGPT, turn off Improve the model for everyone under Settings > Data controls. The opt-out applies to new conversations only. On Free, Plus and Pro, a thumbs up or thumbs down still lets OpenAI use the whole conversation to improve its models, even after the user has opted out.

The last control to check is retention. On ChatGPT Enterprise, for Healthcare and Edu, workspace admins control how long data is kept. In Google Workspace, the Gemini app keeps prompts and responses for up to 36 months, and admins set the period.

When the Terms Change

A no-training promise covers the version of the terms in force today. On February 13, 2024, staff at the US Federal Trade Commission (FTC) warned that a company may act unfairly or deceptively if it adopts more permissive data practices, such as using customer data for AI training, and tells users only through a surreptitious, retroactive change to its terms of service or privacy policy. FTC staff also wrote that AI companies that mislead users about how their data is collected may be breaking the law, and that this covers promises not to train or update models on customer data. When businesses obtained consumer data unlawfully, the FTC has required them to delete any products built with it, models included.

The FTC has acted on changed privacy policies before. In 2004 it charged Gateway Learning Corporation, known for its Hooked on Phonics products, after the company changed its privacy policy to allow sharing consumer data with third parties without notice or consent. In June 2023, it alleged that the genetic testing company 1Health.io (formerly Vitagene) broke the law when it changed its privacy policy to retroactively expand the kinds of third parties with which it could share consumers' personal data.

None of this stops a vendor from changing its terms. It does give a business good reason to keep a dated copy of the terms it signed up under. Save the terms and privacy pages for each AI plan you pay for, and compare them with the new version whenever the vendor announces an update.

How to Check Your Own Vendor

First, check the plan each person signs in to against the business plans named above. Anyone doing client work on a personal ChatGPT or Claude account is under consumer terms that allow training. Moving that person to a business seat changes the default for everything they type from then on.

Next, find where the promise is written down. Google's sits in Section 18 of the Cloud service terms, and the Workspace service-specific terms carry their own Training Restriction section. For every other vendor, ask for the document and section number that hold the no-training promise for your plan.

Then set the controls. On Claude Team or Enterprise, turn off Rate chats unless you want staff ratings to reach Anthropic. On any personal ChatGPT account still in use, turn off Improve the model for everyone, and tell its owner that rating an answer still shares that conversation. On personal Claude accounts, leave Model Improvement off and use Incognito for sensitive chats. On ChatGPT Enterprise, Healthcare or Edu, set a retention period. In Workspace, have your admins choose the Gemini retention period, up to 36 months. Keep your saved copy of the terms alongside a record of these settings, and check both again when the vendor changes either one.

Key Takeaways

  • A rating can open a conversation to training on Claude for Work but not on ChatGPT business plans, so on Claude Team or Enterprise switch Rate chats off before staff start rating answers.
  • Ask each vendor which document and section hold its no-training promise for your plan, and save a dated copy of it.
  • If a vendor announces new terms, pause uploads of client files until you have compared the new version with your saved copy.

References

Posts in this series