Privacy, Intellectual Property, Vendors and Governance

This is Unit Seven of Generative AI at Work, on privacy, intellectual property, vendors and governance. Unit Six designed the human loop. This unit sets the boundaries around it: what data may be used, which tools may be bought, and the rules a team works by. It draws on NIST's profile and framework, the M-CaReM-AI method, and the OECD's principles as a cross-check. Carry one question through this unit: when a new AI tool arrives, how do you decide whether, and how, to let it in? The answer arrives with a human resources team assessing a recruiting tool. By the end you will be able to explain the privacy risks. To name the property, security and supply risks. To apply vendor diligence and acceptable-use policy. To place AI inside enterprise risk. And to review a tool and write team rules.

Six entries on what models learn, keep and infer about people, and on where sensitive work belongs.

Personal data in training.

GAI 151 Personal Data in Training

NIST notes that training generative AI takes huge volumes of data, sometimes including personal data, which strains privacy principles such as transparency, consent and purpose limits. Someone's old public forum posts may sit in a model's training data without their knowledge, and since you cannot check what a model was trained on, you cannot rule your own data in or out.

Models that remember.

GAI 152 Memorization

Models can memorize parts of their training data, and NIST notes that under adversarial attacks they have revealed sensitive information included in it, such as a real person's contact details reproduced from a page the model was trained on. NIST adds that even data appearing only a few times in training can be at risk.

What a model can infer.

GAI 153 Inferences Never Given

Generative AI can also infer sensitive information that was never provided, and NIST's framework notes such inference can identify individuals or reveal facts about them, like a likely health condition deduced from purchases and messages. Removing names from what you paste does not remove what the rest of the text implies.

Where sensitive work goes.

GAI 154 Governed Tools for Sensitive Work

The M-CaReM-AI method directs any work that might involve personal or sensitive data to internal, governed AI services, not public or unmanaged tools, as a hospital might provide an approved assistant inside its own systems and block public chatbots on clinical devices. Governed means someone in your organization controls what the tool keeps and who sees it.

What a provider may keep.

GAI 155 Assume It May Be Logged

The method's default assumption, when in doubt, is that a provider may log or reuse what you send, so the tool's data terms decide what is safe to paste: whether it keeps prompts, for how long, and whether it trains on them. This answers unit one's question: whether your input trains a model is set by the provider's terms.

Privacy-enhancing design.

GAI 156 Privacy-Enhanced AI

Privacy-enhanced is one of NIST's seven marks of trustworthy AI, supported by de-identification, aggregation and privacy-enhancing technologies, such as a report generator working from team totals rather than individual records. NIST warns that privacy techniques can cost accuracy, especially when data is sparse, so record the trade-off.

Four entries on copyright, likeness, poisoned data and the hidden parts behind a product.

Copyright and generated output.

GAI 157 Copyright and Generated Output

NIST notes that output reproducing memorized copyrighted material could infringe, and that the status of generated content similar to protected work is still being debated in legal forums, as with a generated logo that closely resembles a famous trademark. The course gives no legal conclusion; your organization's policy and its lawyers decide.

A person's likeness and voice.

GAI 158 Likeness and Voice

NIST adds that using or imitating a person's identity, likeness or voice without permission is also under legal debate. Picture a marketing team asked to produce a video narrated in a well-known person's synthetic voice. Whatever the law settles, generating a real person's face or voice without consent is a policy matter to raise before starting.

Attacks on the data.

GAI 159 Poisoned Data

Data poisoning, one of the attacks NIST lists, corrupts the data a model learns from or retrieves so that its outputs change as the attacker intends, such as planted pages steering an internal assistant toward a fake supplier's contact details. A tool that reads from a shared library inherits whatever was planted in it.

The parts behind a product.

GAI 160 Third-Party Components

NIST notes that generative AI's value chain is harder to vet than older software: training data is too large to review, and a few models are reused everywhere. A vendor's writing assistant may run on another company's model, trained on a third party's data, with open-source libraries. Every layer you cannot see is a layer someone else vouches for.

Four entries on checking vendors and setting the rules of acceptable use.

Checking an AI vendor.

GAI 161 Due Diligence for AI Vendors

NIST's profile says standard third-party controls apply to AI vendors: procurement due diligence, software bills of materials, service level agreements, and independent attestation reports, such as asking for the component list and an audit report. AI does not need a new procurement process so much as the existing one applied in full.

Controls that fit the model.

GAI 162 Different Controls for Different Models

NIST suggests varying controls by type of AI: foundation models, models fine-tuned for a purpose, and AI embedded in other tools each warrant different checks, so a spreadsheet's built-in AI feature is reviewed differently from a model fine-tuned on company records. Embedded AI is the easiest to miss, because it arrives as an update to approved software.

Acceptable-use policy.

GAI 163 Acceptable-Use Policy

NIST's profile notes that acceptable-use policies for generative AI help reduce misuse, abuse, inappropriate repurposing, and mismatch between what systems do and what users expect, such as a two-page policy on which tools staff may use, for what, with what data, and how outputs are reviewed. A policy nobody has read protects nobody, so pair it with training.

Restricting harmful uses.

GAI 164 Restricting Harmful Uses

NIST notes that organizations can restrict AI applications that cause harm, exceed their stated risk tolerance, or conflict with their values, such as forbidding generative AI for final hiring decisions, performance ratings or medical advice to patients. Restrictions should name uses, not tools, because tools change faster than policy.

Four entries placing AI inside the organization's own risk management and the wider frameworks.

How much risk to accept.

GAI 165 Risk Tolerance Is Contextual

NIST's framework does not set an acceptable level of risk: risk tolerance is each organization's readiness to bear risk for its objectives, shaped by law, sector and context. A marketing team may accept awkward wording that a pharmacy could never accept in dosing instructions. Without a stated tolerance, each person sets their own, invisibly.

Where AI risk belongs.

GAI 166 AI Risk Is Enterprise Risk

NIST's framework says AI risks should be folded into the organization's wider risk management, alongside cybersecurity and privacy, with AI tools in the same risk register and reviewed by the same committee as other critical systems. A separate AI process that nobody connects to the rest becomes a side project.

NIST's four functions.

GAI 167 Govern, Map, Measure, Manage: A Pointer

NIST's framework organizes AI risk work into four functions, govern, map, measure and manage, with govern running across the other three. The Artificial Intelligence course on StudyWalks teaches the functions; this course applies them to everyday generative AI use.

The international picture.

GAI 168 The OECD Agrees

NIST's framework borrows its definitions of an AI system and of AI actors from the Organisation for Economic Co-operation and Development, whose AI Principles, adopted in twenty nineteen and updated in twenty twenty-four, share the same values, including transparency, robustness, accountability and privacy. Alignment is not identity, so check which framework your organization has committed to.

Five procedures and a case on roles, reviews and rules. The opening question is answered here.

A role for AI risk.

GAI 169 The AI Safety Lead

The M-CaReM-AI method proposes an AI Safety Lead, or AI steward, responsible for AI risk across projects and working alongside each area's owners, not above them. For every initiative, the lead asks which core records it touches, which reference documents ground it, and which traceability records must be updated, and checks that features are not quietly redefining core terms. The role works only with real authority and time.

AI without anchors.

GAI 170 AI Without Anchors

The method names a failure called AI without anchors: AI features that create or change important data or decisions without reference to the records they touch. Its recovery path is to name the objects the feature touches, update their reference documents, document what the AI reads and writes, and put the feature through the gate. The failure usually starts small, which is why the gate comes before features are built.

Reviewing a vendor's tool.

GAI 171 Reviewing a Vendor's AI Tool

Reviewing a vendor's AI tool combines NIST's third-party guidance with privacy checks. Identify the type of AI, since controls differ. Read the data terms: what it keeps, where, and whether it trains on your inputs. Ask for evidence: components, service levels, audit reports and incident reporting. Test it on your own realistic cases. Record the decision and conditions, and set a re-review date, because updates change the product.

A team's own rules.

GAI 172 Writing Your Team's AI Rules

A team's generative AI rules turn acceptable-use guidance into a short document people will use. List approved tools and their uses. State what may never be pasted and where sensitive work goes. Name forbidden uses, such as final decisions about people. Set the review and logging rules. Say who owns the rules and how to report a problem. Keep it short enough to read in five minutes, because long rules are skipped.

Who does what across the lifecycle.

GAI 173 Who Does What: Roles Across the Lifecycle

NIST's framework describes many AI actors across a system's lifecycle. Find the designers and developers. Find the deployment actors, which NIST says include system integrators, who pilot the system, check compatibility, ensure compliance, manage change and evaluate the user experience. Find the operators, testers and domain experts, then procurement, third-party providers and oversight bodies, and the end users and affected communities. In contracted work, several of these roles sit with a vendor or integrator, and the course page maps each one to contracting terms.

A new tool, assessed.

GAI 174 A New Tool, Assessed

A worked example, constructed for this course, follows a human resources team assessing a tool that drafts job descriptions and screens applications. Assessed separately, drafting is moderate risk because a person reviews every posting, while screening touches privacy, bias and human–AI configuration, and an error silently turns a qualified person away. The team requires the training opt-out and shorter retention, asks for evidence on how screening performs across groups, and plans its own test. Drafting is approved; screening is held, and may only ever suggest. That answers this unit's question: let a tool in feature by feature, with the risk scan, the data terms, the evidence and a named owner for each.

Look at what you can now do. You can explain how generative AI learns, keeps and infers personal data, and why sensitive work belongs in governed tools. You can name the copyright, likeness, poisoning and supply-chain risks. You can apply vendor diligence, acceptable-use policy and use restrictions. You can place AI risk inside the organization's stated tolerance and its wider risk management. And you can review a vendor tool, write team rules and name the lifecycle roles. Unit Eight turns to documenting AI.