Norm-referenced and Criterion-referenced
Interpretation of Test Results

Results from tests or assessments can be used to make different types of decisions. The specific decision we want to make based on a test result will determine how we interpret the test results.

Sometimes we want to compare students with their peers or rank-order them. When we use tests to make this type of decision we say we are making a norm-referenced or normative interpretation of test results.

Some tests are used to evaluate a person's mastery of a subject, usually by using cut-scores to define proficiency or pass/fail decisions. When tests are used to decide if a student meets a certain pre-defined standard or criteria, we say we are making a criterion-referenced interpretation of test results.

In this interactive module we will use an explorable example to learn the basic concepts and uses of both norm-referenced and criterion-referenced interpretations of test results.

Mary Masters Math

Mary is a 9th grade student at ABC High School and she just took the final Math I test.

The results from this test will be used to:

• Determine if 9th grade students mastered Math I concepts or need remediation before moving on to Math II.
• The minimum score needed to move on to Math II without remediation is 35.


• Give prizes to the top performing 9th grade students in Math I at ABC High.
• To get a prize a student must rank in the top 10 percent of all 9th grade students in Math I at ABC High.

How can we use Mary's test score to decide if she can move on to Math II without remediation? How can we use that same score to decide if she will receive a prize for being a top performer in comparison to her schoolmates?
Let's find out!

Some details about our example:

• The test contains 50 questions and each question is worth 1 point.
• Each 9th grade class has a total of 20 students.
• The high school, ABC High, has five 9th grade classes with a total of 100 ninth grade students.
• ABC High has records from all students who took this test for the past 10 years.

Raw Score and Scaled Score

Mr. Smith, Mary's math teacher, computes the Math I test score by counting the number of correct answers, with no penalty for incorrect responses. We call the number correct a raw score because it is based solely on the number of correctly answered items on the test.

A raw score might be difficult to interpret and compare since tests can have different numbers of questions and total numbers of points. A simple way to make sense of a raw score is to put it on a percent scale. When we transform the raw score into percent correct we produce a scaled score.

If a student got questions right out of 50 on the test, the student would score % correct on the test.

In our example, let's assume Mary got correct questions (raw score) or % correct (scaled score) on the test.

Does this score tell you if she won any of the Math prizes?

Mary's score:

No! We don't know if Mary won a prize because her raw and percent correct scores tell us nothing about how her score compares to her schoolmates.

Norm-referenced Scores and Interpretations

In order to know how one student's score compares to other students who took the same test we must have a norm-referenced score.

A norm-referenced score typically indicates the examinee's relative position in a group of test takers that we want to compare the examinee's score with. Once we have information about this group we are able to answer the questions above.

A basic ranking in terms of raw or scaled scores can start to give us a better picture of how Mary's performance compares to that of her classmates.

There are 20 students in Mary's class, so to place in the top 10% she would need to be one of the top 2 students in her class. Looking at the sorted chart above we can see that Mary did not rank in the top 10% of her class.

Can Mary still win a Math I prize?

To answer this question we first need to define Mary's norm group. A norm group is any group we wish to make comparisons against.

In order to quickly understand how a particular score compares to other scores in a norm group we usually use a type of norm-referenced score called percentile rank. If a raw score of 15 correspond to a percentile of 30, it means that 30% of the norm group had raw scores lower than 15. Similarly, when we say that a student's score is at the 80th percentile we are saying that the student's performance on the test was equal to or better than 80% of the students in the norm group.

Percentile ranks are different from percent correct scores! The percentile rank indicates a student's relative standing in a group. The percent correct score indicates what percent of the total possible points a student got on a test.

Quiz: We know that the results of the Math I test will be used to give prizes to students who scored at the top 10% of 9th grade students in Math I at ABC High. What would be the most appropriate norm group for this kind of decision?

1. Mary's 9th grade class
2. All ABC High's 9th grade students in the current school year
3. All ABC High's students
4. All current and past ABC High's 9th grade students who have taken this test

Yes! Mary won a Math I prize since she placed in the top 10% of all 9th graders at ABC High even though she was not in the top 10% of her class.

Can you tell if she will be able to move on to Math II without remediation?

Criterion-referenced Scores and Interpretations

Placement, mastery or pass/fail decisions are usually made by comparing the student's performance on a test against a fixed set of predetermined criteria or learning standards.

Test performance is summarized as the student's test score, and the predetermined criteria is usually defined as a minimum cut-score that the student must obtain to demonstrate proficiency in the subject matter.

When we compare test scores against a cut-score to make placement, mastery or pass/fail decisions we are making a criterion-referenced interpretation.

Combining Norm- and Criterion-referenced Interpretations

Generally speaking, it is not the test but the interpretation of its results that is norm-referenced or criterion-referenced. A test might have been built to provide criterion-referenced interpretation of a student's mastery of a subject but after many administrations it could be used to make a norm-referenced interpretation, that is, to compare the student's performance to that of other test-takers.

Additionally, we can, and often do, interpret a single test score both in terms of norm- and criterion-referencing.

Activity 4: Now we will look at some sample questions that use both criterion-referenced and norm-referenced interpretation of test scores to apply the knowledge you have gained in this module.

1. Select a question you want to answer from the list under "Decision Type".
2. Select the score type that would be most appropriate for this decision from the list under "Score Type".
3. Select the norm group (use none if not needed) that would be most appropriate for the chosen question.
4. Click "Submit" to update the chart below and see how the score type and norm group selected impacted Mary and her classmates in terms of the decision you want to make and find the answer to the question you selected.

Does the question selected require a norm-referenced, criterion-referenced or combined interpretation of the test results?

Summary

• Norm-referenced interpretation: when we interpret a person's score by comparing his or her score to those of other individuals who have taked the same test (norm group).


• Criterion-referenced interpretation: when we interpret a person's performance by comparing it to some predefined standard or criterion of proficiency.

In our example, we could say that we made a norm-referenced interpretation of Mary's test results when:

• we used her percentile rank to decide if she would win a Math I prize.
• we said that students who had scores at the 80th percentile in their class are eligible to enroll in Calculus.

We made a criterion-referenced interpretation of Mary's test results when:

• we decided that she would move on to Math II if she obtained at least 35 points or 70% correct on her Math I test.
• we changed the rules for the Math prize and defined that all students who scored 45 points on the test would receive the prize.

We made a combined norm- and criterion-referenced interpretation of Mary's test results when:

• we used her percentile rank (norm) and the predefined cut-score (criterion) to decide if she would be able to enroll in Math II when there were only a limited number of spots available.