Assessment and Evaluation

This is Unit Twelve, part two, on assessment: grading, standardized tests and accountability, reading scores with statistics, and the issues standardized tests raise. Part one closed on the choices behind a grade. Part two turns to standardized tests and the numbers they report. Carry one idea through this part: a score means only what its reference group and its intended use allow. At the final entry, the part shows one test valid for one purpose and not for others. By the end you will be able to interpret a score as norm-referenced or criterion-referenced. To describe a set of scores by center and spread. To convert among z scores, T scores, stanines and percentiles. To explain NCLB's accountability design and what replaced it. And to explain why validity depends on the use of a score.

Two entries on grading against a standard or against classmates, and on explaining assessment to parents.

Absolute and relative grading.

EDP 1130 Absolute and Relative Grading

Absolute, or criterion-referenced, grading compares each student with a fixed standard; relative, or norm-referenced, grading compares students with one another, as in grading on the curve. With absolute grading every student can earn an A; on a curve only a set share can, and relative grading discourages cooperation, since one student's gain is another's loss.

Explaining assessment to parents.

EDP 1131 Communicating Assessment to Parents

Teachers explain to parents the purpose, technique and criteria of assessments, through newsletters and district portals, using the same task-focused, strategy-focused feedback they give students, as a newsletter explains what a new writing rubric rewards and shows a sample paragraph. Wise feedback applies to parents too where trust may be an issue.

Four entries on standardized tests: two ways to read a score, stakes, uses, and three kinds of test.

Two ways to read a score.

EDP 1132 Norm- and Criterion-Referenced Scores

Standardized tests are developed by teams and administered and scored uniformly, and their scores are read in one of two ways. A criterion-referenced interpretation compares performance with a standard, telling what a student can do, such as reading grade-level text with ninety percent accuracy. A norm-referenced interpretation compares performance with a norm group, usually as a percentile rank, such as scoring higher than seventy percent of fourth graders nationally. A teacher planning next week's lessons needs criterion-referenced information; a district ranking applicants for limited places needs norm-referenced information. A percentile rank is not a percentage correct.

High stakes and low stakes.

EDP 1133 High-Stakes and Low-Stakes Tests

High-stakes tests carry serious consequences, such as graduation, licensure or school accountability, like a graduation exam; low-stakes tests carry little consequence for the student, like an ungraded field test, so students may not try hard and scores may understate what they know. Test anxiety runs higher when stakes are high.

What standardized tests are used for.

EDP 1134 Uses of Standardized Tests

Standardized tests serve comparison in a wider context, diagnosis, selection and placement, planning through data-based decisions, and accountability; aptitude and achievement scores together help diagnose a learning disability, and admission tests help select students. Classroom grades and standardized scores can disagree in either direction, each telling something the other misses.

Three kinds of standardized test.

EDP 1135 Achievement, Aptitude and Diagnostic Tests

Achievement tests measure what has been learned; diagnostic tests pinpoint specific strengths and weaknesses, such as fluent decoding with weak comprehension; aptitude tests predict general achievement from verbal, quantitative and problem-solving skills. Intelligence tests are now often called cognitive ability tests, stressing developed rather than innate ability.

Four entries on accountability and comparison: federal testing law, subgroups, growth models, and international tests.

From NCLB to ESSA.

EDP 1136 From NCLB to ESSA

The No Child Left Behind Act of two thousand two set federal testing and accountability policy until the Every Student Succeeds Act, signed by President Obama on December tenth, twenty fifteen, reauthorized the Elementary and Secondary Education Act. Much writing on high-stakes testing describes NCLB, so treat it as the two thousand two to twenty fifteen policy.

Adequate yearly progress, and subgroups.

EDP 1137 AYP and Disaggregation

Under NCLB, students were tested yearly in grades three to eight and once in high school, results were disaggregated by subgroup, and every group was to be proficient by the twenty thirteen to twenty fourteen school year, a goal of one hundred percent. A school could meet targets overall yet miss adequate yearly progress through one subgroup.

Status and growth.

EDP 1138 Status and Growth Models

A status model judges a school by the share of students proficient at one time; a growth, or value-added, model judges it by how much students improve from year to year, so a school starting far behind may show strong growth while its status stays low. State cut-offs differ, which is why NAEP serves as a common yardstick.

International comparisons.

EDP 1139 PISA and TIMSS

International comparisons include TIMSS, the Trends in International Mathematics and Science Study, testing grades four and eight, and PISA, the Program for International Student Assessment, testing fifteen-year-olds in reading, mathematical and scientific literacy. Rankings change between rounds, so any ranking must carry its year.

Eight entries on reading scores: distributions, center and spread, the normal curve, standard scores, grade equivalents and scaled scores.

Frequency distributions.

EDP 1140 Frequency Distributions

A frequency distribution lists each score, or range of scores, with the number of students who earned it, as five students scored eight, seven scored nine and two scored ten; a histogram draws the same information as bars, so the shape of a set of scores shows at a glance. Organizing comes before summarizing.

Mean, median and mode.

EDP 1141 Mean, Median and Mode

The mean is the arithmetic average; the median is the middle score when scores are ordered; the mode is the most frequent score, and a distribution with two modes is bimodal. For two, three, three, four and eight, the mean is four, the median three and the mode three, since the mean is pulled toward outliers.

Range, variance and standard deviation.

EDP 1142 Range, Variance and Standard Deviation

Variability describes how spread out scores are: the range is the highest score minus the lowest, so scores from fifty-five to ninety-five have a range of forty; the variance is the average squared distance from the mean; and the standard deviation, its square root, gives the typical distance. The same mean can hide different spreads.

Describing a set of scores.

EDP 1143 Describing a Set of Scores

Describing a set of scores takes a measure of center and a measure of spread. The mean adds the scores and divides by their number; the median is the middle score; the mode is the most frequent; the range is the highest minus the lowest. Seven quiz scores, twelve, fourteen, fifteen, fifteen, sixteen, eighteen and twenty-two, add to one hundred twelve, so the mean is sixteen, the median fifteen, the mode fifteen, and the range ten. Replace the twenty-two with fifty, and the mean rises to twenty while the median stays fifteen. With outliers, report the median, and always pair center with spread.

The normal distribution.

EDP 1144 The Normal Distribution

In a normal distribution, symmetric and bell-shaped, the mean, median and mode are equal; about thirty-four percent of scores fall between the mean and one standard deviation, fourteen percent between one and two, and two percent beyond two, on each side, so about sixty-eight percent of IQ scores fall between eighty-five and one hundred fifteen.

Standard scores and percentiles.

EDP 1145 Standard Scores and Percentiles

Standard scores restate a raw score by its distance from the mean in standard deviations. A z score has a mean of zero and a standard deviation of one. A T score has a mean of fifty and a standard deviation of ten, so T equals fifty plus ten times z. A stanine has a mean of five and a standard deviation of two, running from one to nine. A student scoring sixty on a test with a mean of fifty and a standard deviation of ten has a z of one, a T of sixty, a stanine of seven, and a percentile near the eighty-fourth. A z of two is about the ninety-eighth percentile, not the eighty-fourth.

Grade-equivalent scores.

EDP 1146 Grade-Equivalent Scores

A grade-equivalent score estimates, from a trend line, the grade level whose typical student would earn a given raw score, and such scores are often misinterpreted: a fourth grader scoring six point oh on a fourth-grade test did very well on fourth-grade work, not sixth-grade work. It is not a placement recommendation.

Scaled scores.

EDP 1147 Scaled Scores

A scaled score converts a raw score, the number correct, onto a reporting scale that means the same across test forms. On CLEP exams, each correct answer earns one point, wrong or skipped answers lose nothing, and a scaled score of fifty is equivalent to a course grade of C.

Five entries on the issues standardized tests raise, ending with the payoff: why validity depends on use.

Bias in standardized tests.

EDP 1148 Bias in Standardized Tests

Bias in standardized tests can be everyday, scores reflecting unequal opportunity, or technical: item content and format, and predictive accuracy, where one study found slight over-prediction for Black and Latino students and under-prediction for women. A claim that easy items favor one group has been challenged and remains contested.

Stereotype threat, contested.

EDP 1149 Stereotype Threat, Contested

Stereotype threat, described by Steele and Aronson in nineteen ninety-five, is the idea that awareness of a negative stereotype about one's group can lower test performance. A later meta-analysis found a small average effect and several signs of publication bias, so teach the finding and the debate over its size together.

Teaching to the test.

EDP 1150 Teaching to the Test

High-stakes testing can narrow the curriculum toward tested subjects and invite excessive test preparation, as a school cuts science and art to add reading drills, even though background knowledge, built by a broad curriculum, drives reading comprehension. Reading tests measure knowledge of the world as well as reading skill.

Cheating.

EDP 1151 Cheating

High-stakes testing has brought cheating by students and by adults, including unethical practices such as changing answers, coaching on actual test items and giving extra time; erasure analyses flag classrooms with unusually many wrong-to-right answer changes. Ethical test preparation teaches the content, not the items.

The payoff: validity depends on use.

EDP 1152 Validity Depends on Use

Here the part's idea is decided. Validity belongs to the interpretations and uses of scores, not to the test, which is why one test can support one decision and not another. A unit test with strong content evidence shows what students learned, not later success; an admission test with strong predictive evidence supports selection, not a list of mastered skills. Norm-referenced scores rank; criterion-referenced scores describe. A status model and a growth model can judge the same school oppositely. Every score carries measurement error, so read it as a band. Take a district using one reading test three ways: to place students in reading groups, it checks the scores against observed reading, which is concurrent evidence; to judge teachers, it would need a growth model and far more evidence; to choose skills to teach next, it needs criterion-referenced information a percentile cannot give. Ask valid for what use, and for which students.

Look at what you can now do. You can interpret a score as criterion-referenced, what a student can do, or norm-referenced, where the student ranks. You can describe a set of scores by center and spread, choosing the median when outliers pull the mean. You can convert among z scores, T scores, stanines and percentiles through the z score. You can explain NCLB's design of yearly tests, disaggregated subgroups and adequate yearly progress, replaced by ESSA in twenty fifteen. And you can explain why validity depends on the use of a score. Unit Thirteen, the capstone, draws the course together.