Data Management

This is Unit Eight of Computer Science: data and databases. The unit opens by separating three words people use interchangeably. Data are raw facts. Information is data that has been processed into meaning. And a database is a collection of related data, which is a third thing again. Most of the unit is built around one model — the relational one, proposed in nineteen seventy and grounded in set theory — and then around the honest observation that the guarantees it offers are purchased rather than given. There is a pair of entries in here that look like contradictions and are not. One says to remove redundancy. The next says to add it back deliberately. Both are correct, and the procedure between them shows why by counting. The unit closes on arrangements for holding an organisation's data at scale — including one that decides the structure before any data arrives, and one that deliberately decides nothing at all. By the end you will be able to do five things. One: say what separates data from information. Two: name the key hierarchy and what a foreign key does. Three: say what the four transaction guarantees promise, and cost. Four: say why normalising and denormalising are both right. Five: say what separates a warehouse from a lake.

Three entries setting terms. The distinction in the first one carries through the whole unit — raw facts are not the same thing as meaning, and a great deal of work sits between them.

Three words.

CS 0411 Data, Information, Database

Data are raw facts, information is data processed into meaning, and a database is a collection of related data. Three separate things behind three words often treated as one.

Managing it.

CS 0412 Data Management

Controlling access through policy, reducing error, and keeping data available. Management is about the conditions around data rather than the data itself.

The software.

CS 0413 The Database Management System

The package used to define, create, use and maintain a database with appropriate controls. It sits between every application and the stored data — the same mediating shape as Unit Six.

Four entries on a system's self-knowledge. A database holds a description of its own structure, and three separate tests of whether what it holds is any good — separate, because a record can pass one and fail another.

The structure.

CS 0414 The Data Model

An abstract model carrying the concepts that describe a database's structure and the operations available on it. The model decides what questions can be asked.

The catalogue.

CS 0415 The Data Dictionary

The catalogue holding metadata — information describing content, format and structure. A database that describes itself is a database other software can work with.

Three tests.

CS 0416 Data Quality's Three Tests

Quality decomposes into accuracy, completeness and consistency — three separately checkable properties. A record can be accurate and incomplete, or complete and inconsistent with another record.

Four kinds of work.

CS 0417 The Four Sublanguages

Creating structures, changing contents, asking questions, and controlling access — four sublanguages, each a different kind of authority over the same data.

Four entries on the model most databases still use. It came out of set theory in nineteen seventy, and its key idea is that the way to connect two tables is to put a value from one inside the other.

The model.

CS 0420 The Relational Model

Proposed in nineteen seventy at IBM Research, basing data on the relation — a construction taken from set theory rather than invented for the purpose.

Identifying a row.

CS 0421 Keys: Super, Candidate, Primary

Attributes that together identify a row uniquely form a superkey; the minimal ones are candidate keys; one candidate is chosen as primary. A narrowing hierarchy, each step removing what is not needed.

Connecting tables.

CS 0422 The Foreign Key

An attribute referencing a primary key in a different relation — the mechanism by which separate tables become connected rather than merely stored together.

The language.

CS 0423 SQL

The declarative language for managing and querying relational data — declarative meaning you state what you want rather than how to get it.

Three entries, and the error entry at the end is the one to hold onto. The guarantees a database offers are real, and they are not free — somebody is paying for them in processing, and that somebody is the system you are running.

The unit of work.

CS 0424 The Transaction

A single logical unit of work accessing and possibly modifying a database through reads and writes. What counts as one unit is a decision, not a fact.

Four properties.

CS 0425 ACID

Atomicity executes the unit whole or not at all, consistency admits only valid states, isolation keeps concurrent transactions from seeing each other's partial work, and durability survives failure.

And the error.

CS 0426 "ACID Costs Nothing"

The guarantees are bought. Full isolation in particular requires significant additional processing — so a system offering weaker guarantees is not necessarily worse-built, it made a different purchase.

Two entries that read as opposites. One removes repeated data. The other puts repeated data back on purpose. They are answers to different questions, and knowing which question you are answering is the whole skill.

Removing it.

CS 0427 Normalization

Structuring a relational database to reduce repeated data and improve integrity, through a defined sequence of forms each stricter than the last.

Putting it back.

CS 0428 Denormalization

Deliberately adding redundant data — extra attributes, extra tables, copies — to reduce query running time. It undoes the previous entry knowingly, in exchange for speed.

One procedure, and it settles the previous argument with arithmetic. Splitting one combined table into two linked tables has a countable effect on how much data is stored — so the trade stops being a matter of taste.

Watch one table become two.

CS 0445 Splitting the Table

The redundancy savings are countable — a combined table split into two linked by a foreign key replaces repeated values with single stored ones. The benefit is a number rather than a principle.

Four entries on databases that are not tables. Each of these picks a storage model matched to the shape of the data — and the last one is the graph structure from Unit Three, turned into a product.

Not tables.

CS 0431 The Nonrelational Database

Storing data without rows and columns, using a storage model chosen to suit the data rather than fitting the data into a general one.

The simplest.

CS 0432 The Key-Value Store

A database built on an associative array, storing key-value pairs and retrieving by key. The hashing idea from Unit Three, offered as a database.

The next.

CS 0433 The Document Store

Data kept in structured document formats, with the document's name as key and its contents as value. Structure inside each record, without a structure imposed across records.

And the general one.

CS 0434 The Graph Database

Data as a network of related nodes in any relationship shape, built for analysing the connections themselves rather than the records at the ends of them.

Five entries on holding an organisation's data. The interesting axis is when structure gets decided — before the data arrives, or not at all — and one of these five holds no data whatsoever.

The first.

CS 0437 The Data Warehouse

Centralising an organisation's data in a single repository, processed and given a structure decided before it arrives. The structure is a commitment made in advance.

How it gets fed.

CS 0438 Extraction, Transformation, Loading

Extraction pulls from source systems, transformation reshapes what was pulled, and loading places it. The transformation step is where the advance commitment gets enforced.

Scaled down.

CS 0439 The Data Mart

A smaller warehouse serving one homogeneous group — narrower content, and better query performance because of the narrowness.

Holding nothing.

CS 0440 The Virtual Data Mart

One access point over underlying stores, holding no physical data and pulling only at query time. A view rather than a copy.

And deciding nothing.

CS 0441 The Data Lake

Structured and unstructured data at any scale, stored raw with no schema defined first. The opposite commitment from the warehouse — every decision deferred to whoever reads it later.

Two entries closing the material. Data past the scale traditional systems were built for, and then a pointer — because everything in this unit ends up feeding something that is taught in a different course entirely.

Past the scale.

CS 0442 Big Data

High volumes of structured, semi-structured and unstructured data at scales traditional databases were not built to handle. The definition is relative to the tooling, and it moves.

And where it goes.

CS 0443 Learning Tools, Pointed Elsewhere

Data management feeds machine learning pipelines, with lakes and warehouses supplying training data — while the learning methods themselves are taught elsewhere.

One capstone, and its structure is worth noticing. A single purchase gets accounted for twice — once as a transaction the operational database has to guarantee, and once as a row that ends up somewhere analytical.

The capstone. One purchase.

CS 0449 One Purchase, Fully Accounted

One purchase exercising the whole unit twice — as a transaction with guarantees to keep, and then again as data moving into the analytical side. Both halves, on the same event.

Five things you can now do. You can separate data from information — raw facts against processed meaning — and say that a database is a third thing, a collection of related data. You can name the key hierarchy from superkey down to primary, and say that a foreign key is what turns separate tables into connected ones. You can say what the four transaction guarantees promise, and that they are purchased — full isolation in particular costs significant processing. You can say why normalising and denormalising are both correct: one reduces repeated data, the other restores it deliberately for speed, and the savings on either side are countable. And you can say what separates a warehouse from a lake — one commits to a structure before the data arrives, and the other defers every decision.