Devin's Blog

Notes on LLM Privacy

10 min read

  • The OpenAI terms of use are similar to the Google Privacy Policy in that OpenAI will not train on data from developers using its API.
  • As a developer, you are responsible for both the input and the output resulting from the services you use at OpenAI, and also own the resulting input and output (may use as you please).
  • That being said, you must not do those attacks shown in LLM Security and Privacy:
    • You may not use our Services for any illegal, harmful, or abusive activity.

Terms of Use

  • You may not use our Services for any illegal, harmful, or abusive activity.
  • Attempt to or assist anyone to reverse engineer, decompile or discover the source code or underlying components of our Services, including our models, algorithms, or systems (except to the extent this restriction is prohibited by applicable law).
  • You may provide input to the Services (“Input”), and receive output from the Services based on the Input (“Output”). Input and Output are collectively “Content.” You are responsible for Content, including ensuring that it does not violate any applicable law or these Terms.
  • you (a) retain your ownership rights in Input and (b) own the Output. We hereby assign to you all our right, title, and interest, if any, in and to Output.
  • You must evaluate Output for accuracy and appropriateness for your use case, including using human review as appropriate, before using or sharing Output from the Services.

Enterprise Edition

  • OpenAI does train on ChatGPT, unless you use the Teams or Enterprise Edition ChatGPT
  • Additionally, with the enterprise edition, you maintain sole ownership to the fine tuned model, and as such may act at your discretion.
  • The enterprise addition is additionally compliant to policies such as GDPR.
  • The enterprise edition is part of a whole host of compliance and other policies provided by OpenAI, and includes a portal for access.

Rights as User of OpenAI Products in General

  • Any OpenAI service professes not to ‘sell’ or ‘share’ Personal Information for cross-contextual behavioral advertising and doesn’t process sensitive Personal Information for the purpose of inferring characteristics about a consumer.
  • However, there is still information collected, such as information on the type of content interacted with and other personal inforamation, which may be shared with 3rd parties as OpenAI pleases.
  • However, information is not sold for advertising purposes, unlike Google.
  • A OpenAI Privacy Health Center is provided for asking for privacy requests.
  • Notable is the following within the OpenAI privacy policy:
    • Depending on local law and exceptions, individuals may have rights regarding their Personal Information including understanding how it’s processed, requesting deletion, correcting information, and freedom from discrimination related to privacy rights.
  • OpenAI does provide a OpenAI Security Portal | SafeBase URL for security related inquiries.

Question

Couldn’t a user using, for example, ChatGPT, gauge information about how an LLM is used by requesting their information? Answer: Since the information is processed by the LLM, its method must be somewhat shown for the user to find out how his information was processed.

Google Terms of Use

Info

Gemini will be integrated into Android, in which the API will be used when building Android applications. Therefore, it becomes important to analyze Google’s privacy policies regarding LLMs.

Tldr

It is up to the developer to safeguard their LLMs as much as possible against concerns outlined in LLM Security and Privacy, but ultimately, any introduction of an LLM to ones app entails risks as mentioned in LLM Security and Privacy.

Main Points

  • According to Google Terms of Service for AI, the customer must affirm that the Generative AI service may in some cases hallucinate or otherwise produce similar output for multiple users.
  • Additionally, the terms of service seem to be quite open ended, such as in the following statement:
    Transclude of Service-Specific-Terms#^5e5018
  • Notably, Google, just like ChatGPT, will not train on the customer data, and will not store the prompt for longer than is needed to generate the output.
    • As a result, any content generated from the APIs won’t be trained on to further fine tune against hate speech and other forms of content violation as showcased in the Generative AI Prohibited Use Policy, such as attempts by the user to circumvent safety filters.

Responsibility

  • Google seems to be taking a hands off approach.
  • According to Safety guidance  |  Gemini API  |  Google AI for Developers, it is the responsibility of the developer to make sure that the LLMs it fine tunes are used properly.
  • Therefore, the developer must research and implement safeguards within their application, such as restricting users’ options to a dropdown list or adversarially testing their own fine tuned model.
  • All these safeguards achieve is to lower the chances of an attack such as prompt injection. There remain areas, such as regarding Google’s own safety filters.
  • Google does provide a number of safety settings as shown in Google Gemini Safety Settings, but these settings are based on the probabilistic chances of a given prompt being private, rather than this being a concrete fact.
  • Most of the settings deal with matters regarding the policing of content through means such as filters for sexual harassment and other forms of content, rather than regarding what the developer can and cannot implement within their app.
  • In fact, Google’s provided safety filters can be configured by the user through a sliding bar, in which the more safety and privacy protection, the less potentially useful the LLM is.

Organized

Responsibility of User

  • So as the application owner, you are responsible for knowing your users and the potential harms your application may cause, and ensuring that your application uses LLMs safely and responsibly.
    • Note: So the onus is on the developer to make sure that LLMs are used properly.
  • A good way to begin exploring potential safety risks is to research your end users, and others who might be affected by your application’s results. This can take many forms including researching state of the art studies in your app domain, observing how people are using similar apps, or running a user study, survey, or conducting informal interviews with potential users.

Ways a Developer Can Be Responsible

Safeguards
  • For example, you could restrict users to choose only from a drop-down list of input prompts, or offer pop-up suggestions with descriptive phrases
    • Note: Give the user less control, and additionally fine tune the model to the scenario you wish. This ensures less hazard.
  • Another safeguard is to try and protect against possible prompt injection. Prompt injection, much like SQL injection
  • Adjusting functionality to something that is inherently lower risk. Tasks that are narrower in scope.
  • an application with outputs that are reviewed by human experts prior to any action being taken might be deemed less likely to produce harmful outputs than the identical application without such oversight.
    • Note: That is true, but this itself could be a privacy violation!
  • Safety benchmarking involves designing safety metrics that reflect the ways your application could be unsafe in the context of how it is likely to get used, then testing how well your application performs on the metrics using evaluation datasets.
    • Note: The keyword here is “likely to be used”, so prompt injection remains an issue.
  • Adversarial testing involves proactively trying to break your application.

Where a Developer is not Responsible

  • The API provides built-in safety filters to help address some common language model problems such as toxic language and hate speech, and striving for inclusiveness and avoidance of stereotypes.
  • If the input is overtly adversarial or abusive in nature, it could be blocked and instead output a pre-scripted response.
    • Note: This is subject to a lot of human bias, and is likely going to be a problem for years to come (the notion of humans blocking speech could itself be a violation of the freedom of speech).
  • minimum acceptable levels of safety metrics
    • Note: So in other words: there is no such thing as “safe” whenever you are bring in an LLM to scoop up your data. There is simply a minimum acceptable amount which most users likely believe is enough.

Highlights

  • For adversarial tests, select test data that is most likely to elicit problematic output from the model.
    • Note: Similarly to software, you now interact with the LLM to see what problematic output can come about, then accordingly seeing who can win the game.
  • LLMs are known to sometimes produce different outputs for the same input prompt
    • Note: This is the biggest issue with this type of testing. How can you actually test for anything if you get nondeterministic results? This becomes a much harder paradigm to control, since you in a sense “lose control” to the whims of the AI model rather than yourself having a say in the matter of the final output. Also, this leads to a large waste of time when prompting, since you don’t know how correct the AI is, and therefore may or may not be wasting your time in trying to get it to fix your code.

Terms of Use

  • As between Customer and Google, Google does not assert any ownership rights in any new intellectual property created in the Generated Output.
  • Customer acknowledges that a Generative AI Service may, in some scenarios, produce the same or similar Generated Output for multiple customers.
    • Note: This means that the customer must sign away their rights (at least somewhat).
  • Customer will not, and will not allow End Users to, use Generated Output to: (i) develop models that compete with any Service or Software, or (ii) reverse engineer any Service, Software, or their models (or extract any components of the foregoing).
  • Google will not store outside Customer’s Account (i) Customer Data prompted to a Generative AI Service for longer than is reasonably necessary to create the Generated Output, or (ii) the Generated Output.
    • Note: Interesting. However, if the developer fine tunes these models in a way which generates hateful content, then how will Google understand whether the input generates a form of hateful speech?
  • Customer will have sole (to the exclusion of Google and other third parties) access to use any uptrained model that Customer builds by using an AI/ML Service to retrain or tune a Google Pre-Trained Model using Customer Data (as retrained or tuned, a “Fine-Tuned Model”).
  • Customer will not, and will not allow End Users to, use an AI/ML Service to: (a) develop models that compete with the specific Service being used by Customer, or (b) reverse engineer or extract components of any Service, Software, or their models. Google may immediately suspend or terminate Customer’s use of any AI/ML Service based on any suspected violation of the preceding sentence. The restriction in subsection (a) of this Section does not apply to Vertex AI Platform so long as Customer does not use a Google Pre-Trained Model.
    • Note: This is important. In other words: If you violate our terms of service, which itself includes information that is subjective in nature such as hate speech and bias or the possibility for reverse engineering, then we may suspend your use. Another way of saying this: we own you. You have no rights.
  • Google will not use Customer Data to train or fine-tune any AI/ML models without Customer’s prior permission or instruction.
    • Note: This is similar to OpenAI’s terms of service. Essentially: Google will not train on the data given by the customer. This is more specifically stated in the below highlight.
  • may provide inaccurate or offensive Generated Output, and are not designed for or intended to meet Customer’s regulatory, legal, or other obligations.
    • Note: So by using this service, you affirm that AI is subject to a whole host of problems, and therefore if there is a mistake, this is not our fault.

References

Graph View