What Generative AI Is, and Isn't

This is Unit One of Generative AI at Work, on what generative AI is, and what it is not. This course is for professionals who use generative AI tools at work and need to use them well and safely. It draws on NIST's Generative AI Profile and AI Risk Management Framework, and on the M-CaReM-AI method by J. R. Carver. Carry one question through this unit: if these tools write so fluently, why can you not take their word, and what do you do instead? The answer arrives with the working pattern of draft, check and decide. By the end you will be able to define generative AI and foundation models. To explain why fluent output can be false. To state the method's three principles. To decide whether a task suits AI and read its answer critically. And to apply draft, check and decide to a real task.

Four entries on what generative AI is, what most tools are built on, and how a language model writes.

The definition this course uses.

GAI 001 Generative AI, Defined

NIST's Generative AI Profile defines generative AI as models that learn the structure of their input data and generate new, derived content from it, as when a writing tool drafts a paragraph that appears nowhere in its training text. Derived is not verified: nothing in the act of generating checks the output against the facts.

What most of these tools are built on.

GAI 002 Foundation Models

Most generative AI tools sit on a foundation model: trained on broad data, mostly through self-supervision, and usable across a wide range of tasks. One underlying model can draft emails for sales, summaries for legal and code for engineering inside three different products, and NIST warns that products sharing one foundation model can share its weaknesses too.

Generative AI is not only text.

GAI 003 Text, Images, Audio, Video

NIST's profile lists images, video, audio and other digital content alongside text, so one team might draft a report, an illustration and a voice-over in an afternoon. False statements are mainly a text problem; for images, audio and video, invented content is often the goal, so the checks differ by medium.

How a language model writes.

GAI 004 How It Writes: A Pointer

Language models generate text by predicting the next word, or piece of a word, again and again, following the statistical patterns of their training data. The Artificial Intelligence course on StudyWalks teaches the mechanics. The consequence that matters at work is that a statistically likely sentence is not necessarily a true one.

Six entries on why confident, fluent output can still be wrong, and on where it goes wrong most often.

Two properties that are easy to confuse.

GAI 005 Fluent Is Not True

Fluency and accuracy are separate properties of generated text; NIST notes that the same statistical prediction produces accurate outputs and inaccurate or inconsistent ones. Two polished paragraphs about a policy can be equally confident, one correct and one wrong about the deadline. Polish shows the model's skill with language, not the content's accuracy.

The name for the failure.

GAI 006 Confabulation

Confabulation is NIST's term for generative AI confidently presenting erroneous or false content; everyday speech calls it hallucination. A summary that cites a section the source does not contain, or contradicts its own earlier statement, is confabulating. NIST uses the term partly because commenters noted that hallucination attributes human qualities to software.

Why confident delivery is the danger.

GAI 007 Confident Delivery

NIST observes that the risk comes when people believe false content because it is delivered confidently, then act on it or pass it along. Its example is a confabulated summary of patient reports leading a doctor toward the wrong diagnosis or treatment. The more consequential the decision, the more a confident wrong answer costs.

Reasoning that only looks right.

GAI 008 Reasons That Look Right

Generated answers can arrive with confabulated logic or citations: reasoning steps and sources that look like justification but do not support the answer, such as three tidy steps to a wrong conclusion, or a cited report that was never published. NIST notes that such explanations can deepen misplaced trust, so a visible chain of reasoning is not proof.

Where errors are most likely.

GAI 009 Where It Is Weakest

NIST flags two conditions where errors concentrate: open-ended prompts asking for long answers, and domains that demand deep context or expertise. A one-line prompt for a ten-page legal analysis combines both. The safest requests are short, bounded and checkable; the riskiest are long, open and specialist.

Two beliefs to drop.

GAI 010 Not a Database, Not Learning From You

Two common beliefs are wrong: that a chat tool looks answers up in a database, and that it learns from talking to you. The Artificial Intelligence course covers both. A correction shapes the current conversation, not the model; whether a provider later trains on your input is a policy question for unit seven.

Five entries on what AI does well at work, and the M-CaReM-AI method's three principles for using it.

Where AI earns its place.

GAI 011 What It Does Well at Work

Generative AI earns its place at work on first drafts, summaries, searching long text, filling templates and transforming text from one form to another. The M-CaReM-AI method names these as its uses: faster drafting, help to search and summarize, and simple text transformation. Each produces a draft for a person to check, not a finished decision.

The principle the whole course rests on.

GAI 012 AI Proposes; Humans Decide

The method's first principle is that AI proposes and humans decide, and a person stays responsible for checking each proposal against policy, data, workflow and risk. An analyst might confirm most of a suggested field list, reject one item and add the one it missed. Unit six builds the review patterns around this principle.

The second principle.

GAI 013 Context, Not Scratch

The second principle is that AI works from context, not from scratch: the more grounded material you supply, the more useful the output, as when you paste the actual policy paragraph instead of asking what such policies usually say. Without your material, it falls back on what is typical, not on what is true for you.

The third principle.

GAI 014 Mark the Draft

The third principle is that AI is explicit, not invisible: AI-generated content is treated as a first draft and marked as one, like a document noting that a section was drafted with AI and reviewed by its owner. The method also forbids letting AI quietly change approved material; changes go through human review.

What comes before the tool.

GAI 015 The Work Comes First

The method says it stands on its own without AI: AI speeds the work, but none of its core concepts depends on it. Its advice when a tool is weak is to strengthen the human structures first, because they set how much value AI can add. AI amplifies a working process; it does not replace one.

Four entries that put the unit to work: choosing a task, reading an answer, and the working pattern. The opening question is answered here.

Four questions before you start.

GAI 016 Is This a Good Task for AI?

Four questions decide whether a task fits generative AI and how much checking it needs. First, name the output: drafts, summaries, template filling and text transformation are strong cases, and a final decision is never the tool's. Second, check the shape: short, bounded requests are safer than open-ended ones. Third, check the domain, and plan for expert review where it is specialist. Fourth, check what you would paste, and follow your organization's policy for anything personal or confidential. Passing all four sets how much checking to plan, not whether to check.

How to read what comes back.

GAI 017 Reading an AI Answer

Reading a generated answer means looking past its fluency. First, find the claims: the facts, numbers, names and citations it depends on. Second, trace each one to a source, your own material or an outside check. Third, look for inconsistency with your input or with itself, which NIST also counts as confabulation. Fourth, test the reasoning by checking the conclusion directly. The method names the failures to watch: invented fields, confident but wrong rules, and drifting definitions.

The working pattern this course is built on.

GAI 018 Draft, Check, Decide

The course's basic working pattern has three stages, built on the method's three principles. Draft: give the tool grounded context, say who the result is for, and ask for a first draft in the format you need. Check: read the draft for its claims, trace each one, and fix what is wrong or missing. Decide: a person, not the tool, accepts the result, records that AI drafted it, and takes responsibility for it. That answers this unit's question: you cannot take the tool's word because fluency is not truth, so you check, and a person decides. Skipping the middle stage turns a fast draft into a fast error.

One task, worked from start to finish.

GAI 019 One Task, Start to Finish

A worked example, constructed for this course, follows a team lead summarizing a twelve-page travel policy for her staff. The task suits AI and holds no personal data, so she uses her company's approved tool. She pastes the full policy, names her readers, and asks for the five rules they most need. Checking each claim against the policy, she finds one wrong: the draft says receipts are needed above fifty dollars, while the policy says twenty-five. She also spots a missing approval step for international travel. She corrects both, marks the summary as drafted with AI and checked, and sends it under her own name. The tool saved her the writing, not the reading.

Look at what you can now do. You can define generative AI as NIST does, and say why products built on one foundation model share its weaknesses. You can explain why fluent output can be false, name confabulation, and say where it concentrates: long, open, specialist requests. You can state the method's three principles: AI proposes and humans decide, AI works from context, and AI output is a marked draft. You can decide whether a task suits AI, and read an answer for its claims. And you can run draft, check and decide on a real task. Unit Two names the risks.