Test Norms
Norms
Norms have been defined as the standard or average performance of a
group of pupils in a test.
It is essential to keep in mind that there is difference between norm and
standard.
Norms indicate the actual achievement of students at standardized
level, while standard indicate the desired level of performance.
Norms are averages or values determined by actual measurement of a
group of persons who are the representatives of specific population.
While standard possesses a desirable objective, which can be less or
more than the obtained norm.
Test norms
Test norms consist of data that make it possible to determine the
relative standing of an individual who has taken a test.
By itself, a subject’s raw score (e.g., the number of answers that agree
with the scoring key) has little meaning.
Almost always, a test score must be interpreted as indicating the
subject’s position relative to others in some group.
Norms provide a basis for comparing the individual with a group.
To prepare norms, we administer a test on a large population in order to
change the scores into percentiles or standard scores
Cont..
Norm refers to behavior that is usual, average, normal, standard,
expected, or typical.
In a psychometric context, norms are the test performance data of a
particular group of testtakers that are designed for use as a reference
when evaluating or interpreting individual test scores.
A normative sample is that group of people whose performance on a
particular test is analyzed for reference in evaluating the performance of
individual testtakers.
Members of the normative sample will all be typical with respect to
some characteristic(s) of the people for whom the particular test was
designed.
Sampling to Develop Norms
The process of administering a test to a representative sample of test-
takers for the purpose of establishing norms is referred to as
standardization or test standardization.
A test is said to be standardized when it has clearly specified procedures
for administration and scoring, typically including normative data.
To understand how norms are derived, an understanding of sampling is
necessary.
Sampling
Sampling is a technique of selecting individual members or a subset of
the population to make statistical inferences from them and estimate
characteristics of the whole population.
This population is the complete universe or set of individuals with at
least one common, observable characteristic. The common observable
characteristic(s) could be just about anything.
Two types of sampling
Probability sampling
Non-probability sampling
Types of Sampling
Probability (random) sampling, start with a complete sampling
frame of all eligible individuals from which sample is selected. In this
way, all eligible individuals have a chance of being chosen for the
sample, and results will be more generalizable. Probability sampling
methods tend to be more time-consuming and expensive than non-
probability sampling. It is mainly used in quantitative research
Non-probability (non-random) sampling, does not start with a
complete sampling frame, so some individuals have no chance of being
selected. Consequently, the effect of sampling error cannot be
estimated and there is a significant risk of ending up with a non-
representative sample which produces non-generalizable results.
However, non-probability sampling methods tend to be cheaper and
more convenient, and they are useful for exploratory research and
hypothesis generation.
Relativity Of Norms
Relativity of norms is a concept regarding normative scores and how
they differ when compared between different groups.
For example, a student takes a standardized test which is scored on a
normative scale (like the SAT). They score in the 80th percentile
nationally (which means they did better than 80% of all students who
took the test nationwide), 85th percentile locally (which means they did
better than 85% of the students who took the test in their school
system) and 55th percentile of applicants to a prestigious university
(they did better than 55% of the students that applied to that school).
These are all normative scores but they differ between the groups.
Cont..
Relativity of norms is a concept of determining which group is most
relevant to what information you want.
If the information most important is how the student compared to others
in their area, then the local scores would be used. If the student wants
to attend the prestigious university, then those scores would be more
important.
This concept just addresses how normative scores can be different and
how the same score can be viewed differently depending on which
sample is looked at.
Steps in developing Norms
1. Defining the target population
Normative group -based on intention of test .
2. Selecting the sample from the target population
True representative sample.
Cross sectional
Large sample
Random sampling
3. Standardizing the conditions
Test administration must be standard, valid
Percentiles
Percentiles are expressed in terms of the percentage of persons in the
standardization sample.
A percentile indicate the individual relative position in the
standardization sample.
Percentiles are different from percentage scores; the percentage scores
are raw scores, which expressed in terms of percentage of correct items
and percentiles are derived scores, which expressed in terms of
percentage of a person.
Lower the percentile the poorer the persons standing.
Standard Score Norms
Standard scores express the individual’s distance from the mean in
terms of the standard deviation of the distribution.
Standard score is a derived score which has a fix mean and fixed
standard deviation.
There are several types of standard scores such as z score(also known
as sigma scores), T score, Sten score, Stanine score etc.
Norm-Referenced
Versus Criterion-
Referenced Testing
Norm-referenced Testing
Norm-referenced is a type of test that assesses the test taker’s ability
and performance against other test takers.
It could also include a group of test takers against another group of test
takers.
The test’s content covers a broad area of topics that the test takers are
expected to know and the difficulty of the content varies.
This test must also be administered in a standardized format.
Norm-referenced test helps determine the position of the test taker in a
predefined population.
Examples of norm-referenced tests include SAT etc.
Cont..
Norm-referenced tests are specifically designed to rank test takers on a
“bell curve,” or a distribution of scores that resembles, when graphed,
the outline of a bell—i.e., a small percentage of students performing
well, most performing average, and a small percentage performing
poorly.
Norm-referenced refers to standardized tests that are designed to
compare and rank test takers in relation to one another.
Norm-referenced tests report whether test takers performed better or
worse than a hypothetical average student, which is determined by
comparing scores against the performance results of a statistically
selected group of test takers, typically of the same age or grade level,
who have already taken the exam.
Criterion-Referenced testing
We may define a criterion as a standard on which a judgment or
decision may be based.
Criterion-referenced testing and assessment may be defined as a
method of evaluation and a way of deriving meaning from test scores by
evaluating an individual’s score with reference to a set standard.
Most found in education and occupational settings.
Cont..
Some examples:
To be eligible for a high-school diploma, students must demonstrate at
least a sixth-grade reading level.
To earn the privilege of driving an automobile, would-be drivers must
take a road test and demonstrate their driving skill to the satisfaction of
a state-appointed examiner.
To be licensed as a psychologist, the applicant must achieve a score that
meets or exceeds the score mandated by the state on the licensing test.
Cont..
The criterion in criterion-referenced assessments typically derives from
the values or standards of an individual or organization.
For example, in order to earn a black belt in karate, students must
demonstrate a black-belt level of proficiency in karate and meet related
criteria such as those related to self-discipline and focus.
Each student is evaluated individually to see if all of these criteria are
met.
Regardless of the level of performance of all the testtakers, only
students who meet all the criteria will leave the training room with a
brand-new black belt
Cont..
The focus in the criterion-referenced approach is on how scores relate to
a particular content area or domain, the approach has also been referred
to as domain- or content-referenced testing and assessment.
Because criterion-referenced tests are frequently used to gauge
achievement or mastery, they are sometimes referred to as mastery
tests. The criterion-referenced approach has enjoyed widespread
acceptance in the field of computer-assisted education programs. In
such programs, mastery of segments of materials is assessed before the
program user can proceed to the next level
Cont..
“Has this flight trainee mastered the material she needs to be an airline
pilot?”
This is the type of question that an airline personnel office might seek to
address with a mastery test on a flight simulator.
If a standard, or criterion, for passing a hypothetical “Airline Pilot Test”
(APT) has been set at 85% correct, then trainees who score 84% correct
or less will not pass. It matters not whether they scored 84% or 42%.
Conversely, trainees who score 85% or better on the test will pass
whether they scored 85% or 100%. All who score 85% or better are said
to have mastered the skills and knowledge necessary to be an airline
pilot.
Cont..
Another airline might find it useful to set up three categories of findings
based on criterion-referenced interpretation of test scores:
85% or better correct = pass
75% to 84% correct = retest after a two-month refresher course
74% or less = fail
Criticism
Critics of the criterion-referenced approach argue that if it is strictly
followed, potentially important information about an individual’s
performance relative to other testtakers is lost.
Another criticism is that although this approach may have value with
respect to the assessment of mastery of basic knowledge, skills, or both,
it has little or no meaningful application at the upper end of the
knowledge/skill continuum. Thus, the approach is clearly meaningful in
evaluating whether pupils have mastered basic reading, writing, and
arithmetic. But how useful is it in evaluating doctoral-level writing or
math?
Norm-referenced and criterion-referenced are two of many ways that
test data may be viewed and interpreted. However, these terms are
not mutually exclusive, and the use of one approach with a set of test
data does not necessarily preclude the use of the other approach for
another application.
In a sense, all testing is ultimately normative, even if the scores are as
seemingly criterion-referenced as pass–fail. This is so because even in a
pass–fail score there is an inherent acknowledgment of a continuum of
abilities. At some point in that continuum, a dichotomizing cutoff point
has been applied.
Difference between Norm
reference and Criterion reference
testing
The area of focus regarding test results
In norm-referenced interpretations of test data, a usual area of focus is
how an individual performed relative to other people who took the test.
In criterion-referenced interpretations of test data, a usual area of focus
is the testtaker’s performance: what the testtaker can or cannot do;
what the testtaker has or has not learned; whether the testtaker does or
does not meet specified criteria for inclusion in some group, access to
certain privileges, and so forth.
Cont..
The difference is actually in the scores—and some tests can
provide both criterion-referenced results and norm-referenced results!
Norm-referenced assessments work similarly: An individual student’s
percentile rank describes their performance in comparison to the
performance of students in the norm group, but does not indicate
whether or not they met or exceed a specific standard or criterion.
Relative Score (Norm-referenced)
Absolute Score (Criterion-referenced)
THANKYOU!
