Generative AI at Work / Unit 8 GAI 189 worked example, 60 to 90 seconds
Reading a Model Card Critically
The idea
A model card is a claim made by the developer, and reading it critically means checking it against your own use, using the sections Mitchell and colleagues defined.
In practice
Step one: read the intended and out-of-scope uses first, and stop if your use is out of scope. Step two: read the caveats and ethical considerations before any performance figure. Step three: check whether the factors include the groups, languages and conditions in your own setting. Step four: compare the evaluation data with your real material, asking whether it is anything like your work. Step five: note what the card does not report, such as results for combinations of groups, and treat each gap as untested.
Where people slip
A good model card is evidence of good practice, not of fitness for your task. NIST's warning about benchmarks applies: test on your own cases before relying on the reported numbers.
Your place on the trail
Every lesson rests on earlier ones and opens later ones. Tap any to jump.
Something wrong in this lesson? Flag it · Sources for Generative AI at Work