Level 200: Using gen AI well · Module 200.5

Three coworkers talk around a closed laptop and plain folders.
Before you share

Protecting data in prompts, files and connectors · Technical gear

Reviewed and published, version 1.0.0. As of 18 Nov 2026. Not a credential. Vulnerability scores, regulator pages and guidance are dated in the text.

Objectives:

The setting (Big Widget Company, its tools, data classes and approvals) is fictional. No coding is needed; the decision procedure is given as pseudocode for precision.

1. The sharing decision as a function of four inputs

Check the data class, approval for the purpose, training and retention settings, and what connected tools can reach. If unsure, wait for the data owner or tool owner.
This fictional checklist applies minimization and purpose guidance (s4) plus disclosure and least-privilege guidance (s5). Follow your own organization’s rules.
Figure data as text

Big Widget’s fictional rule

PartWhat it means
CLASSWhat class is the information?
APPROVAL [s4]Is this tool approved for this class and this purpose?
SETTINGS [s5]Will the tool train on the data? How long will it keep it?
CONNECTORS [s5]What mail, files or systems can it reach?
IF YOU DON’T KNOWWait before sharing. Ask the data owner or the tool owner.

OWASP's Top 10 for LLM Applications 2025 lists LLM02:2025, Sensitive Information Disclosure, and gives as examples personally identifiable information, financial details, health records, confidential business data, security credentials and legal documents.1 Its mitigations include data sanitisation, strict access control on the principle of least privilege, restricting the model's access to external data sources, educating users not to input sensitive information, and transparency about data retention, usage and deletion, including a way to opt out of training.1

A licence is an input to none of these. The course's decision rule for Big Widget (a course rule, not an external standard) takes four inputs:

decide(data, tool, purpose):
    c = classify(data)                         # green | yellow | red, per company scheme
    if c is unknown: return INSUFFICIENT_INFO
    a = tool.approved.get((c, purpose))        # approval is per (class, purpose), not per tool
    if a is unknown: return INSUFFICIENT_INFO
    if a is False:  return DENY
    s = tool.config                            # as deployed: training_use, retention
    if s.training_use is unknown or s.retention is unknown: return INSUFFICIENT_INFO
    if not policy_allows(c, s):  return DENY
    k = tool.connectors                        # what the tool can read or act on
    if k is unknown: return INSUFFICIENT_INFO
    if k exceeds what purpose needs: return HOLD   # ask the tool owner to scope it
    return CONSTRAIN(minimised(data, purpose))

Two points carry the weight. First, approval is keyed on (class, purpose), not on the tool: the same assistant can be approved for yellow drafting and not for red customer records. Second, "unknown" is a distinct state that short-circuits to insufficient information, never to allow.

The outputs map onto the gate vocabulary of 100.7 (allow · constrain · hold · deny; plainly, go ahead · only this far · wait for a person · no). INSUFFICIENT_INFO and HOLD are both hold: the request waits for a named person (the data owner or tool owner), and a hold that nobody answers decays to deny, not to allow. Minimisation makes every approved share a constrain. The function proposes; a person gives the answer. (Course rules.)

The UK ICO's Guidance on AI and data protection supplies the data-protection reasoning behind minimisation and purpose. It says to identify the minimum amount of personal data needed to fulfil the purpose;2 that processing personal data, "whether to train a new AI system, or make predictions using an existing one", needs an appropriate lawful basis;2 and, on necessity, that "the mere possibility that some data might be useful for a prediction is not by itself sufficient for the organisation to demonstrate that processing this data is necessary for building the model".2 On roles, it says an organisation with no purpose of its own that only acts on a client's instructions is likely to be a processor, and that organisations will be controllers "regardless of how they are described in any contract".2 Its security chapter describes privacy attacks on trained models, such as model inversion and membership inference, and says that organisations which train models and provide them to others should assess whether those models may contain personal data or are at risk of revealing it if attacked.2 That advice is addressed to model providers, not to users; the course draws its own inference that data a tool trains on could later be exposed this way, which is one reason the training-use setting is an input to the decision. As of 18 Nov 2026 the main guidance page (updated 15 Mar 2023) and its sub-pages, including the lawfulness page (updated 28 Oct 2024), carry a notice that, due to changes made by the Data (Use and Access) Act, the guidance is under review.2

Worked example. A customer complaint containing a name and a home address is class red. Tool A (public chatbot): approved for green only → DENY. Tool B (company assistant): approved for (red customer data, customer-service reply); training off; 30-day retention; connector scoped to the customer-service mailbox → CONSTRAIN, minimised (drop the address). This is consistent with 100.7, where red is deny, or hold for a special approved process: at Big Widget, Tool B's (red, customer reply) approval is that process, so the hold has already been resolved by the person who granted it. Tool C (unlisted browser plug-in; reads all tabs; retention unknown) → DENY on approval before configuration is even evaluated. Change Tool B's retention to "unknown" and the answer becomes INSUFFICIENT_INFO.

2. Connectors widen the readable set

A connector extends the context the model can draw on (mail, files, calendars, line-of-business systems) and sometimes the actions it can take. Treat the union of connector scopes as the readable set: anything in it can appear in an output. OWASP's LLM02 mitigations are, in effect, constraints on that set: least-privilege access to sensitive data and restricted data sources.1

OWASP also notes that system-prompt restrictions on which data types may be returned "may not always be honored and could be bypassed via prompt injection or other methods".1 So the readable set must be bounded by the permission system, not by the prompt: a system prompt is not a security boundary.

3. Indirect prompt injection as an exfiltration channel

OWASP LLM01:2025 defines prompt injection as user prompts altering the model's behaviour or output in unintended ways, and distinguishes the indirect form: "Indirect prompt injections occur when an LLM accepts input from external sources, such as websites or files."1

Greshake et al. (arXiv 2302.12173, v1 23 Feb 2023, v2 5 May 2023) introduced the threat model. They argue that retrieval "blurs the line between data and instructions," and that adversaries can "remotely affect other users' systems" by "injecting the prompts into data likely to be retrieved at inference time."3 They organise the resulting threats into six categories: information gathering, fraud, intrusion, malware, manipulated content and availability.3 For information gathering they tested a synthetic application (built on OpenAI's APIs; LangChain for the text-davinci-003 version, OpenAI's chat format directly for gpt-4) and Bing Chat.3 Exfiltration routes they describe include:3

They conclude that "a clear-cut defense against these problems is, at least, difficult to achieve"3 and, in the abstract, that "effective mitigations of these emerging threats are currently lacking."3 OWASP, in 2025, says "it is unclear if there are fool-proof methods of prevention for prompt injection."1

Since retrieval blurs data and instructions,3 a defence that relies on the model choosing to ignore injected text can fail. Hence the emphasis on controls that hold even when the model is fooled.

Case: CVE-2025-32711 ("EchoLeak"). NVD's record, published 11 Jun 2025 with sourceIdentifier: secure@microsoft.com, describes "Ai command injection in M365 Copilot allows an unauthorized attacker to disclose information over a network," with weakness CWE-74.4 It carries two CVSS 3.1 vectors:4

ScorerBaseVector
Microsoft (secondary)9.3 CriticalAV:N/AC:L/PR:N/UI:N/S:C/C:H/I:L/A:N
NVD (primary)7.5 HighAV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N

Both agree on network attack vector, low complexity, no privileges, no user interaction and high confidentiality impact; they differ on scope and integrity.4 The record tags the product as an exclusively hosted service.4 The name "EchoLeak" is taken from a reference URL in the record, https://www.aim.security/lp/aim-labs-echoleak-m365; the record does not say who wrote that page.4 That page and Microsoft's advisory were not opened for this module, so the delivery vector, the assistant's reach and the data exposed are not checked and are not stated here.

Controls that bound the blast radius. From LLM01's mitigation list:1

  1. Privilege control and least privilege. "Restrict the model's access privileges to the minimum necessary for its intended operations"; give the application its own API tokens and handle those functions in code rather than giving them to the model.1 This shrinks the readable set and the callable set.
  2. Human approval for high-risk actions. Human-in-the-loop controls for privileged operations, so an injected instruction cannot complete an outbound action unaided.1 OWASP does not enumerate which operations are high-risk. Because read operations can themselves exfiltrate,3 this course scopes the approval to every egress-capable action (fetch, rendered link to an unknown domain, send, share, write), enforced in the tool layer rather than requested in the prompt: no direct model-to-tool path for consequential actions. (Course rule.)

Further layers on the same list: segregate and clearly denote untrusted content; input and output filtering; output-format validation in deterministic code; adversarial testing that treats the model as an untrusted user.1 LLM02 adds restricting data sources.1 For exfiltration specifically, note that Greshake et al.'s routes all need an egress: a fetch, a link the user follows, or a send. Controls on egress (no autonomous outbound requests, no rendered links to unknown domains, approval on sends) and on the readable set address the two ends of the channel. This pairing is the course's synthesis, not a claim from either source.

Fictional illustration (not EchoLeak). Big Widget's sales assistant has mailbox scope across all reps, read access to the whole shared drive, and a send-mail tool. An external email contains hidden text directing the assistant, when summarising, to gather dealer price lists and encode them into a link. Without controls, the readable set includes the price lists and a link is the egress. Scoping the connector to the requesting rep's mailbox removes the price lists from the readable set; requiring approval for sends and outbound fetches, and suppressing links to unknown domains, closes the egress. The injected request then ends in hold, and the rep can answer deny.

4. Regulatory findings as dated statements

Treat a regulator's announcement and the decision's later procedural history as separate records with separate timestamps. Do not infer one from the other.

Record 1: Garante press release, 20 Dec 2024. The Garante per la protezione dei dati personali published "ChatGPT, il Garante privacy chiude l'istruttoria", dated Rome, 20 December 2024, on an investigation it said began in March 2023.5 Its stated findings against OpenAI:5

Measures announced: a EUR 15 million fine, calculated taking account of the company's cooperative attitude, and a six-month institutional communication campaign (radio, television, newspapers, internet) on data collection for training and on data-subject rights including objection, rectification and erasure, described as the first use of the power in article 166(7) of Italy's Privacy Code.5 Because OpenAI established its European headquarters in Ireland during the investigation, the file was transmitted to Ireland's Data Protection Commission as lead supervisory authority under the one-stop-shop mechanism, for infringements continuing after that date.5

Record 2: note on the same page, read 18 Nov 2026. "Decision No. 755 of 2 November 2024 has been temporarily removed from the website of the Italian Data Protection Authority following judgment No. 4153/2026 of the Court of Rome", which, "published on 18 March 2026, upheld the appeal lodged against the Authority's decision."5 The note is undated on the page.

Not checked. The judgment, its reasoning, any further appeal and the decision's current legal effect. The regulator's findings are the regulator's position as of 20 Dec 2024, not adjudicated facts, and the note does not say which findings, if any, the court disagreed with.

A briefing sentence that satisfies both constraints:

On 20 December 2024 the Garante announced a EUR 15 million fine against OpenAI over ChatGPT, citing training without an adequate legal basis, transparency failures, no age verification and an unnotified breach. As of 18 November 2026 its page notes the decision was temporarily removed after a Court of Rome judgment published 18 March 2026 upheld the appeal; the decision's current status has not been checked.

The same discipline applies to guidance: the ICO's AI guidance cited above is itself flagged as under review.2

5. Get to Yes, Stay at Yes, Recover to Yes

A yes is scoped to one (tool, class, purpose) and given by a named person; it is not a blanket licence.

Receipts are not logs (course rule). A shareable record of a gate decision, a receipt, carries the decision and identifiers (for example hashes, the decision, the policy and key identifiers), never prompts, outputs, payloads, connector contents or personal data. Incident logs and any necessary prompt and output notes stay in the controlled work record. A hash in a receipt is not a licence to disclose the material behind it, and a valid receipt proves what was decided, not that the decision or the model's answer was right.

Further learning (optional, not part of this lesson):

Sources

  1. s5 OWASP Gen AI Security Project (2025). OWASP Top 10 for LLM Applications 2025. Standard. Checked 2026-11-18 by source-checker (Claude Opus 5.5, round-2 re-check session; not the writer, reviser, item writer, red team or round-1 checker). ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩
  2. s4 Information Commissioner's Office (2023). Guidance on AI and data protection. Official doc. Checked 2026-11-18 by source-checker (Claude Opus 5.5, round-2 re-check session; not the writer, reviser, item writer, red team or round-1 checker). ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩
  3. s3 Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, Mario Fritz (2023). Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. Paper. Checked 2026-11-18 by source-checker (Claude Opus 5.5, round-2 re-check session; not the writer, reviser, item writer, red team or round-1 checker). ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩
  4. s2 NIST National Vulnerability Database (2025). CVE-2025-32711 (NVD API record). Record. Checked 2026-11-18 by source-checker (Claude Opus 5.5, round-2 re-check session; not the writer, reviser, item writer, red team or round-1 checker). ↩ ↩ ↩ ↩ ↩
  5. s1 Garante per la protezione dei dati personali (2024). ChatGPT, il Garante privacy chiude l'istruttoria (press release on provision no. 755 of 2 November 2024). Official doc. Checked 2026-11-18 by source-checker (Claude Opus 5.5, separate session; not the writer, item writer or red team). ↩ ↩ ↩ ↩ ↩

Next: the quiz