Inside The Black Box: Raising Standards Through Classroom Assessment

Inside the Black Box: Raising Standards Through Classroom Assessment by Paul Black and Dylan Wiliam
Illustration © 1998 by A. J. Garces
Inside the Black Box: Raising Standards Through

Classroom Assessment
By Paul Black and Dylan Wiliam
Firm evidence shows that formative assessment is an essential component of classroom work and that its development can raise
standards of achievement, Mr. Black and Mr. Wiliam point out. Indeed, they know of no other way of raising standards for
which such a strong prima facie case can be made.
RAISING the standards of learning that are more effectively. What is missing from the
achieved through schooling is an important efforts alluded to above is any direct help with
national priority. In recent years, governments this task. This fact was recognized in the TIMSS
throughout the world have been more and more video study: ʺA focus on standards and
vigorous in making changes in pursuit of this accountability that ignores the processes of
aim. National, state, and district standards; teaching and learning in classrooms will not
target setting; enhanced programs for the provide the direction that teachers need in their
external testing of studentsʹ performance; quest to improve.ʺ1
surveys such as NAEP (National Assessment of
In terms of systems engineering, present policies
Educational Progress) and TIMSS (Third
in the U.S. and in many other countries seem to
International Mathematics and Science Study);
treat the classroom as a black box. Certain
initiatives to improve school planning and
inputs from the outside ‐‐ pupils, teachers, other
management; and more frequent and thorough
resources, management rules and requirements,
inspection are all means toward the same end.
parental anxieties, standards, tests with high
But the sum of all these reforms has not added
stakes, and so on ‐‐ are fed into the box. Some
up to an effective policy because something is
outputs are supposed to follow: pupils who are
missing.
more knowledgeable and competent, better test
Learning is driven by what teachers and pupils results, teachers who are reasonably satisfied,
do in classrooms. Teachers have to manage and so on. But what is happening inside the
complicated and demanding situations, box? How can anyone be sure that a particular
channeling the personal, emotional, and social set of new inputs will produce better outputs if
pressures of a group of 30 or more youngsters in we donʹt at least study what happens inside?
order to help them learn immediately and And why is it that most of the reform initiatives
become better learners in the future. Standards mentioned in the first paragraph are not aimed
can be raised only if teachers can tackle this task
© Phi Delta Kappa, October, 1998 Page 1

at giving direct help and support to the work of Is there evidence that improving

teachers in classrooms? formative assessment raises standards?
The answer usually given is that it is up to Is there evidence that there is room for
teachers: they have to make the inside work
improvement?
better. This answer is not good enough, for two
reasons. First, it is at least possible that some Is there evidence about how to improve
changes in the inputs may be counterproductive formative assessment?
and make it harder for teachers to raise
standards. Second, it seems strange, even unfair, In setting out to answer these questions, we
to leave the most difficult piece of the standards‐ have conducted an extensive survey of the
raising puzzle entirely to teachers. If there are research literature. We have checked through
ways in which policy makers and others can many books and through the past nine yearsʹ
give direct help and support to the everyday worth of issues of more than 160 journals, and
classroom task of achieving better learning, then we have studied earlier reviews of research. This
surely these ways ought to be pursued process yielded about 580 articles or chapters to
vigorously. study. We prepared a lengthy review, using
material from 250 of these sources, that has been
This article is about the inside of the black box.
published in a special issue of the journal
We focus on one aspect of teaching: formative
Assessment in Education, together with comments
assessment. But we will show that this feature is
on our work by leading educational experts
at the heart of effective teaching. from Australia, Switzerland, Hong Kong,
Lesotho, and the U.S.3
The Argument
We start from the self‐evident proposition that The conclusion we have reached from our
teaching and learning must be interactive. research review is that the answer to each of the
Teachers need to know about their pupilsʹ three questions above is clearly yes. In the three
progress and difficulties with learning so that main sections below, we outline the nature and
they can adapt their own work to meet pupilsʹ force of the evidence that justifies this
needs ‐‐ needs that are often unpredictable and conclusion. However, because we are presenting
that vary from one pupil to another. Teachers a summary here, our text will appear strong on
can find out what they need to know in a variety assertions and weak on the details of their
of ways, including observation and discussion justification. We maintain that these assertions
in the classroom and the reading of pupilsʹ are backed by evidence and that this backing is
written work. set out in full detail in the lengthy review on
which this article is founded.
We use the general term assessment to refer to
all those activities undertaken by teachers ‐‐ and We believe that the three sections below
by their students in assessing themselves ‐‐ that establish a strong case that governments, their
provide information to be used as feedback to agencies, school authorities, and the teaching
modify teaching and learning activities. Such profession should study very carefully whether
assessment becomes formative assessment when they are seriously interested in raising standards
the evidence is actually used to adapt the in education. However, we also acknowledge
teaching to meet student needs.2 widespread evidence that fundamental change
in education can be achieved only slowly ‐‐
There is nothing new about any of this. All through programs of professional development
teachers make assessments in every class they that build on existing good practice. Thus we do
teach. But there are three important questions not conclude that formative assessment is yet
about this process that we seek to answer: another ʺmagic bulletʺ for education. The issues

involved are too complex and too closely linked innovation would record the same
to both the difficulties of classroom practice and achievement as a pupil in the top 35% of
the beliefs that drive public policy. In a final those not so involved.
section, we confront this complexity and try to • An effect size gain of 0.7 in the recent
sketch out a strategy for acting on our evidence. international comparative studies in
mathematics5 would have raised the
Does Improving Formative score of a nation in the middle of the
Assessment Raise Standards? pack of 41 countries (e.g., the U.S.) to
one of the top five.
A research review published in 1986,
concentrating primarily on classroom
Many of these studies arrive at another
assessment work for children with mild
important conclusion: that improved formative
handicaps, surveyed a large number of
assessment helps low achievers more than other
innovations, from which 23 were selected.4
students and so reduces the range of
Those chosen satisfied the condition that
achievement while raising achievement overall.
quantitative evidence of learning gains was
A notable recent example is a study devoted
obtained, both for those involved in the
entirely to low‐achieving students and students
innovation and for a similar group not so
with learning disabilities, which shows that
involved. Since then, many more papers have
frequent assessment feedback helps both groups
been published describing similarly careful
enhance their learning.6 Any gains for such
quantitative experiments. Our own review has
pupils could be particularly important.
selected at least 20 more studies. (The number
Furthermore, pupils who come to see
depends on how rigorous a set of selection
themselves as unable to learn usually cease to
criteria are applied.) All these studies show that
take school seriously. Many become disruptive;
innovations that include strengthening the
others resort to truancy. Such young people are
practice of formative assessment produce
likely to be alienated from society and to become
significant and often substantial learning gains.
the sources and the victims of serious social
These studies range over age groups from 5‐
problems.
year‐olds to university undergraduates, across
several school subjects, and over several Thus it seems clear that very significant learning
countries. gains lie within our grasp. The fact that such
gains have been achieved by a variety of
For research purposes, learning gains of this
methods that have, as a common feature,
type are measured by comparing the average
enhanced formative assessment suggests that
improvements in the test scores of pupils
this feature accounts, at least in part, for the
involved in an innovation with the range of
successes. However, it does not follow that it
scores that are found for typical groups of pupils
would be an easy matter to achieve such gains
on these same tests. The ratio of the former
on a wide scale in normal classrooms. Many of
divided by the latter is known as the effect size.
the reports we have studied raise a number of
Typical effect sizes of the formative assessment
other issues.
experiments were between 0.4 and 0.7. These
effect sizes are larger than most of those found
• All such work involves new ways to
for educational interventions. The following
enhance feedback between those taught
examples illustrate some practical consequences
and the teacher, ways that will require
of such large gains.
significant changes in classroom
practice.
• An effect size of 0.4 would mean that
• Underlying the various approaches are
the average pupil involved in an
assumptions about what makes for

effective learning ‐‐ in particular the literature,ʺ according to a review of
assumption that students have to be assessment practices in U.S. schools.10
actively involved. • The most important difficulties with
• For assessment to function formatively, assessment revolve around three issues.
the results have to be used to adjust The first issue is effective learning.
teaching and learning; thus a significant • The tests used by teachers encourage
aspect of any program will be the ways rote and superficial learning even when
in which teachers make these teachers say they want to develop
adjustments. understanding; many teachers seem
• The ways in which assessment can affect unaware of the inconsistency.
the motivation and self‐esteem of pupils • The questions and other methods
and the benefits of engaging pupils in teachers use are not shared with other
self‐assessment deserve careful teachers in the same school, and they
attention. are not critically reviewed in relation to
what they actually assess.
Is There Room for Improvement? • For primary teachers particularly, there
is a tendency to emphasize quantity and
A poverty of practice. There is a wealth of
presentation of work and to neglect its
research evidence that the everyday practice of
quality in relation to learning.
assessment in classrooms is beset with problems
and shortcomings, as the following selected
The second issue is negative impact.
quotations indicate.
• The giving of marks and the grading
• ʺMarking is usually conscientious but
function are overemphasized, while the
often fails to offer guidance on how
giving of useful advice and the learning
work can be improved. In a significant
function are underemphasized.
minority of cases, marking reinforces
• Approaches are used in which pupils
underachievement and
are compared with one another, the
underexpectation by being too generous
prime purpose of which seems to them
or unfocused. Information about pupil
to be competition rather than personal
performance received by the teacher is
improvement; in consequence,
insufficiently used to inform subsequent
assessment feedback teaches low‐
work,ʺ according to a United Kingdom
achieving pupils that they lack ʺability,ʺ
inspection report on secondary schools.7
causing them to come to believe that
• ʺWhy is the extent and nature of
they are not able to learn.
formative assessment in science so
impoverished?ʺ asked a research study
The third issue is the managerial role of
on secondary science teachers in the
assessments.
United Kingdom.8
• ʺIndeed they pay lip service to
• Teachersʹ feedback to pupils seems to
[formative assessment] but consider that
serve social and managerial functions,
its practice is unrealistic in the present
often at the expense of the learning
educational context,ʺ reported a study of
function.
Canadian secondary teachers.9
• Teachers are often able to predict pupilsʹ
• ʺThe assessment practices outlined results on external tests because their
above are not common, even though own tests imitate them, but at the same
these kinds of approaches are now
widely promoted in the professional

time teachers know too little about their However, it is also clear that the political

pupilsʹ learning needs. commitment to external testing in order to
• The collection of marks to fill in records promote competition had a central priority,
is given higher priority than the analysis while the commitment to formative assessment
of pupilsʹ work to discern learning was marginal. As researchers the world over
needs; furthermore, some teachers pay have found, high‐stakes external tests always
no attention to the assessment records of dominate teaching and assessment. However,
their pupilsʹ previous teachers. they give teachers poor models for formative
assessment because of their limited function of
Of course, not all these descriptions apply to all providing overall summaries of achievement
classrooms. Indeed, there are many schools and rather than helpful diagnosis. Given this fact, it
classrooms to which they do not apply at all. is hardly surprising that numerous research
Nevertheless, these general conclusions have studies of the implementation of the education
been drawn by researchers who have collected reforms in the United Kingdom have found that
evidence ‐‐ through observation, interviews, and formative assessment is ʺseriously in need of
questionnaires ‐‐ from schools in several development.ʺ13 With hindsight, we can see
countries, including the U.S. that the failure to perceive the need for
substantial support for formative assessment
An empty commitment. The development of and to take responsibility for developing such
national assessment policy in England and
support was a serious error.
Wales over the last decade illustrates the
obstacles that stand in the way of developing In the U.S. similar pressures have been felt from
policy support for formative assessment. The political movements characterized by a distrust
recommendations of a government task force in of teachers and a belief that external testing will,
198811 and all subsequent statements of on its own, improve learning. Such fractured
government policy have emphasized the relationships between policy makers and the
importance of formative assessment by teachers. teaching profession are not inevitable ‐‐ indeed,
However, the body charged with carrying out many countries with enviable educational
government policy on assessment had no achievements seem to manage well with policies
strategy either to study or to develop the that show greater respect and support for
formative assessment of teachers and did no teachers. While the situation in the U.S. is far
more than devote a tiny fraction of its resources more diverse than that in England and Wales,
to such work.12 Most of the available resources the effects of high‐stakes state‐mandated testing
and most of the public and political attention are very similar to those of the external tests in
were focused on national external tests. While the United Kingdom. Moreover, the traditional
teachersʹ contributions to these ʺsummative reliance on multiple‐choice testing in the U.S. ‐‐
assessmentsʺ have been given some formal not shared in the United Kingdom ‐‐ has
status, hardly any attention has been paid to exacerbated the negative effects of such policies
their contributions through formative on the quality of classroom learning.
assessment. Moreover, the problems of the
relationship between teachersʹ formative and How Can We Improve Formative
summative roles have received no attention. Assessment?
It is possible that many of the commitments The self‐esteem of pupils. A report of schools in
were stated in the belief that formative Switzerland states that ʺa number of pupils . . .
assessment was not problematic, that it already are content to ʹget by.ʹ . . . Every teacher who
happened all the time and needed no more than wants to practice formative assessment must
formal acknowledgment of its existence.

reconstruct the teaching contracts so as to he or she can do to improve, and should avoid
counteract the habits acquired by his pupils.ʺ14 comparisons with other pupils.
The ultimate user of assessment information Self‐assessment by pupils. Many successful
that is elicited in order to improve learning is innovations have developed self‐ and peer‐
the pupil. There are negative and positive assessment by pupils as ways of enhancing
aspects of this fact. The negative aspect is formative assessment, and such work has
illustrated by the preceding quotation. When the achieved some success with pupils from age 5
classroom culture focuses on rewards, ʺgold upward. This link of formative assessment to
stars,ʺ grades, or class ranking, then pupils look self‐assessment is not an accident; indeed, it is
for ways to obtain the best marks rather than to inevitable.
improve their learning. One reported
To explain this last statement, we should first
consequence is that, when they have any choice,
note that the main problem that those who are
pupils avoid difficult tasks. They also spend
developing self‐assessments encounter is not a
time and energy looking for clues to the ʺright
problem of reliability and trustworthiness.
answer.ʺ Indeed, many become reluctant to ask
Pupils are generally honest and reliable in
questions out of a fear of failure. Pupils who
assessing both themselves and one another; they
encounter difficulties are led to believe that they
can even be too hard on themselves. The main
lack ability, and this belief leads them to
problem is that pupils can assess themselves
attribute their difficulties to a defect in
only when they have a sufficiently clear picture
themselves about which they cannot do a great
of the targets that their learning is meant to
deal. Thus they avoid investing effort in
attain. Surprisingly, and sadly, many pupils do
learning that can lead only to disappointment,
not have such a picture, and they appear to have
and they try to build up their self‐esteem in
become accustomed to receiving classroom
other ways.
teaching as an arbitrary sequence of exercises
The positive aspect of studentsʹ being the with no overarching rationale. To overcome this
primary users of the information gleaned from pattern of passive reception requires hard and
formative assessments is that negative outcomes sustained work. When pupils do acquire such an
‐‐ such as an obsessive focus on competition and overview, they then become more committed
the attendant fear of failure on the part of low and more effective as learners. Moreover, their
achievers ‐‐ are not inevitable. What is needed is own assessments become an object of discussion
a culture of success, backed by a belief that all with their teachers and with one another, and
pupils can achieve. In this regard, formative this discussion further promotes the reflection
assessment can be a powerful weapon if it is on oneʹs own thinking that is essential to good
communicated in the right way. While learning.
formative assessment can help all pupils, it
Thus self‐assessment by pupils, far from being a
yields particularly good results with low
luxury, is in fact an essential component of
achievers by concentrating on specific problems
formative assessment. When anyone is trying to
with their work and giving them a clear
learn, feedback about the effort has three
understanding of what is wrong and how to put
elements: recognition of the desired goal,
it right. Pupils can accept and work with such
evidence about present position, and some
messages, provided that they are not clouded by
understanding of a way to close the gap
overtones about ability, competition, and
between the two.15 All three must be
comparison with others. In summary, the
understood to some degree by anyone before he
message can be stated as follows: feedback to
or she can take action to improve learning.
any pupil should be about the particular
qualities of his or her work, with advice on what

Such an argument is consistent with more improving understanding. Dialogue with the
general ideas established by research into the teacher provides the opportunity for the teacher
way people learn. New understandings are not to respond to and reorient a pupilʹs thinking.
simply swallowed and stored in isolation; they However, there are clearly recorded examples of
have to be assimilated in relation to preexisting such discussions in which teachers have, quite
ideas. The new and the old may be inconsistent unconsciously, responded in ways that would
or even in conflict, and the disparities must be inhibit the future learning of a pupil. What the
resolved by thoughtful actions on the part of the examples have in common is that the teacher is
learner. Realizing that there are new goals for looking for a particular response and lacks the
the learning is an essential part of this process of flexibility or the confidence to deal with the
assimilation. Thus we conclude: if formative unexpected. So the teacher tries to direct the
assessment is to be productive, pupils should be pupil toward giving the expected answer. In
trained in self‐assessment so that they can manipulating the dialogue in this way, the
understand the main purposes of their learning teacher seals off any unusual, often thoughtful
and thereby grasp what they need to do to but unorthodox, attempts by pupils to work out
achieve. their own answers. Over time the pupils get the
message: they are not required to think out their
The evolution of effective teaching. The research
own answers. The object of the exercise is to
studies referred to above show very clearly that
work out ‐‐ or guess ‐‐ what answer the teacher
effective programs of formative assessment
expects to see or hear.
involve far more than the addition of a few
observations and tests to an existing program. A particular feature of the talk between teacher
They require careful scrutiny of all the main and pupils is the asking of questions by the
components of a teaching plan. Indeed, it is clear teacher. This natural and direct way of checking
that instruction and formative assessment are on learning is often unproductive. One common
indivisible. problem is that, following a question, teachers
do not wait long enough to allow pupils to think
To begin at the beginning, the choice of tasks for
out their answers. When a teacher answers his
classroom work and homework is important.
or her own question after only two or three
Tasks have to be justified in terms of the
seconds and when a minute of silence is not
learning aims that they serve, and they can work
tolerable, there is no possibility that a pupil can
well only if opportunities for pupils to
think out what to say.
communicate their evolving understanding are
built into the planning. Discussion, observation There are then two consequences. One is that,
of activities, and marking of written work can all because the only questions that can produce
be used to provide those opportunities, but it is answers in such a short time are questions of
then important to look at or listen carefully to fact, these predominate. The other is that pupils
the talk, the writing, and the actions through donʹt even try to think out a response. Because
which pupils develop and display the state of they know that the answer, followed by another
their understanding. Thus we maintain that question, will come along in a few seconds,
opportunities for pupils to express their there is no point in trying. It is also generally the
understanding should be designed into any case that only a few pupils in a class answer the
piece of teaching, for this will initiate the teacherʹs questions. The rest then leave it to
interaction through which formative assessment these few, knowing that they cannot respond as
aids learning. quickly and being unwilling to risk making
mistakes in public. So the teacher, by lowering
Discussions in which pupils are led to talk about
the level of questions and by accepting answers
their understanding in their own ways are
from a few, can keep the lesson going but is
important aids to increasing knowledge and

actually out of touch with the understanding of been shown to improve learning when it gives

most of the class. The question/answer dialogue each pupil specific guidance on strengths and
becomes a ritual, one in which thoughtful weaknesses, preferably without any overall
involvement suffers. marks. Thus the way in which test results are
reported to pupils so that they can identify their
There are several ways to break this particular
own strengths and weaknesses is critical. Pupils
cycle. They involve giving pupils time to
must be given the means and opportunities to
respond; asking them to discuss their thinking
work with evidence of their difficulties. For
in pairs or in small groups, so that a respondent
formative purposes, a test at the end of a unit or
is speaking on behalf of others; giving pupils a
teaching module is pointless; it is too late to
choice between different possible answers and
work with the results. We conclude that the
asking them to vote on the options; asking all of
feedback on tests, seatwork, and homework
them to write down an answer and then reading
should give each pupil guidance on how to
out a selected few; and so on. What is essential is
improve, and each pupil must be given help and
that any dialogue should evoke thoughtful
an opportunity to work on the improvement.
reflection in which all pupils can be encouraged
to take part, for only then can the formative All these points make clear that there is no one
process start to work. In short, the dialogue simple way to improve formative assessment.
between pupils and a teacher should be What is common to them is that a teacherʹs
thoughtful, reflective, focused to evoke and approach should start by being realistic and
explore understanding, and conducted so that confronting the question ʺDo I really know
all pupils have an opportunity to think and to enough about the understanding of my pupils to
express their ideas. be able to help each of them?ʺ
Tests given in class and tests and other exercises Much of the work teachers must do to make
assigned for homework are also important good use of formative assessment can give rise
means of promoting feedback. A good test can to difficulties. Some pupils will resist attempts
be an occasion for learning. It is better to have to change accustomed routines, for any such
frequent short tests than infrequent long ones. change is uncomfortable, and emphasis on the
Any new learning should first be tested within challenge to think for yourself (and not just to
about a week of a first encounter, but more work harder) can be threatening to many. Pupils
frequent tests are counterproductive. The cannot be expected to believe in the value of
quality of the test items ‐‐ that is, their relevance changes for their learning before they have
to the main learning aims and their clear experienced the benefits of such changes.
communication to the pupil ‐‐ requires scrutiny Moreover, many of the initiatives that are
as well. Good questions are hard to generate, needed take more class time, particularly when
and teachers should collaborate and draw on a central purpose is to change the outlook on
outside sources to collect such questions. learning and the working methods of pupils.
Thus teachers have to take risks in the belief that
Given questions of good quality, it is essential to
such investment of time will yield rewards in
ensure the quality of the feedback. Research
the future, while ʺdeliveryʺ and ʺcoverageʺ with
studies have shown that, if pupils are given only
poor understanding are pointless and can even
marks or grades, they do not benefit from the
be harmful.
feedback. The worst scenario is one in which
some pupils who get low marks this time also Teachers must deal with two basic issues that
got low marks last time and come to expect to are the source of many of the problems
get low marks next time. This cycle of repeated associated with changing to a system of
failure becomes part of a shared belief between formative assessment. The first is the nature of
such students and their teacher. Feedback has each teacherʹs beliefs about learning. If the

teacher assumes that knowledge is to be assessment have to be called into question. The
transmitted and learned, that understanding promotion of testing as an important component
will develop later, and that clarity of exposition for establishing a competitive market in
accompanied by rewards for patient reception education can be very harmful. The more recent
are the essentials of good teaching, then shifting of emphasis toward setting targets for
formative assessment is hardly necessary. all, with assessment providing a touchstone to
However, most teachers accept the wealth of help check pupilsʹ attainments, is a more mature
evidence that this transmission model does not position. However, we would argue that there is
work, even when judged by its own criteria, and a need now to move further, to focus on the inside of
so are willing to make a commitment to teaching the ʺblack boxʺ and so to explore the potential of
through interaction. Formative assessment is an assessment to raise standards directly as an integral
essential component of such instruction. We do part of each pupilʹs learning work.
not mean to imply that individualized, one‐on‐
It follows from this view that several changes
one teaching is the only solution; rather we
are needed. First, policy ought to start with a
mean that what is needed is a classroom culture
recognition that the prime locus for raising
of questioning and deep thinking, in which
standards is the classroom, so that the
pupils learn from shared discussions with
overarching priority has to be the promotion
teachers and peers. What emerges very clearly
and support of change within the classroom.
here is the indivisibility of instruction and
Attempts to raise standards by reforming the
formative assessment practices.
inputs to and measuring the outputs from the
The other issue that can create problems for black box of the classroom can be helpful, but
teachers who wish to adopt an interactive model they are not adequate on their own. Indeed,
of teaching and learning relates to the beliefs their helpfulness can be judged only in light of
teachers hold about the potential of all their pupils for their effects in classrooms.
learning. To sharpen the contrast by overstating
The evidence we have presented here
it, there is on the one hand the ʺfixed I.Q.ʺ view ‐
establishes that a clearly productive way to start
‐ a belief that each pupil has a fixed, inherited
implementing a classroom‐focused policy would
intelligence that cannot be altered much by
be to improve formative assessment. This same
schooling. On the other hand, there is the
evidence also establishes that in doing so we
ʺuntapped potentialʺ view ‐‐ a belief that starts
would not be concentrating on some minor
from the assumption that so‐called ability is a
aspect of the business of teaching and learning.
complex of skills that can be learned. Here, we
Rather, we would be concentrating on several
argue for the underlying belief that all pupils
essential elements: the quality of teacher/pupil
can learn more effectively if one can clear away,
interactions, the stimulus and help for pupils to
by sensitive handling, the obstacles to learning,
take active responsibility for their own learning,
be they cognitive failures never diagnosed or
the particular help needed to move pupils out of
damage to personal confidence or a combination
the trap of ʺlow achievement,ʺ and the
of the two. Clearly the truth lies between these
development of the habits necessary for all
two extremes, but the evidence is that ways of
students to become lifelong learners.
managing formative assessment that work with the
Improvements in formative assessment, which
assumptions of ʺuntapped potentialʺ do help all
are within the reach of all teachers, can
pupils to learn and can give particular help to those
contribute substantially to raising standards in
who have previously struggled.
all these ways.
Policy and Practice Four steps to implementation. If we accept the
Changing the policy perspective. The assumptions argument outlined above, what needs to be
that drive national and state policies for done? The proposals outlined below do not

follow directly from our analysis of assessment that leave the task of translating them into
research. They are consistent with its main everyday practice entirely up to the
findings, but they also call on more general teachers. Their classroom lives are too busy
sources for guidance.16 and too fragile for all but an outstanding
few to undertake such work. What teachers
At one extreme, one might call for more research need is a variety of living examples of
to find out how best to carry out such work; at implementation, as practiced by teachers
the other, one might call for an immediate and with whom they can identify and from
large‐scale program, with new guidelines that whom they can derive the confidence that
all teachers should put into practice. Neither of they can do better. They need to see
these alternatives is sensible: while the first is examples of what doing better means in
unnecessary because enough is known from the practice.
results of research, the second would be
unjustified because not enough is known about So changing teachersʹ practice cannot begin
classroom practicalities in the context of any one with an extensive program of training for
countryʹs schools. all; that could be justified only if it could be
claimed that we have enough ʺtrainersʺ who
Thus the improvement of formative assessment know what to do, which is certainly not the
cannot be a simple matter. There is no quick fix case. The essential first step is to set up a
that can alter existing practice by promising small number of local groups of schools ‐‐
rapid rewards. On the contrary, if the some primary, some secondary, some inner‐
substantial rewards promised by the research city, some from outer suburbs, some rural ‐‐
evidence are to be secured, each teacher must with each school committed both to a
find his or her own ways of incorporating the school‐based development of formative
lessons and ideas set out above into his or her assessment and to collaboration with other
own patterns of classroom work and into the schools in its local group. In such a process,
cultural norms and expectations of a particular the teachers in their classrooms will be
school community.17 This process is a relatively working out the answers to many of the
slow one and takes place through sustained practical questions that the evidence
programs of professional development and presented here cannot answer. They will be
support. This fact does not weaken the message reformulating the issues, perhaps in relation
here; indeed, it should be seen as a sign of its to fundamental insights and certainly in
authenticity, for lasting and fundamental terms that make sense to their peers in other
improvements in teaching and learning must classrooms. It is also essential to carry out
take place in this way. A recent international such development in a range of subject
study of innovation and change in education, areas, for the research in mathematics
encompassing 23 projects in 13 member education is significantly different from that
countries of the Organisation for Economic Co‐ in language, which is different again from
operation and Development, has arrived at that in the creative arts.
exactly the same conclusion with regard to
effective policies for change.18 Such arguments The schools involved would need extra
support in order to give their teachers time
lead us to propose a four‐point scheme for
to plan the initiative in light of existing
teacher development.
evidence, to reflect on their experience as it
1. Learning from development. Teachers will not develops, and to offer advice about training
take up ideas that sound attractive, no others in the future. In addition, there
matter how extensive the research base, if would be a need for external evaluators to
the ideas are presented as general principles help the teachers with their development

work and to collect evidence of its problem. Rather, it is deeply social and
effectiveness. Video studies of classroom personal.19
work would be essential for disseminating
findings to others. The chief negative influence here is that of
short external tests. Such tests can dominate
2. Dissemination. This dimension of the teachersʹ work, and, insofar as they
implementation would be in low gear at the encourage drilling to produce right answers
outset ‐‐ offering schools no more than to short, out‐of‐context questions, they can
general encouragement and explanation of lead teachers to act against their own better
some of the relevant evidence that they judgment about the best ways to develop
might consider in light of their existing the learning of their pupils. This is not to
practices. Dissemination efforts would argue that all such tests are unhelpful.
become more active as results and resources Indeed, they have an important role to play
became available from the development in securing public confidence in the
program. Then strategies for wider accountability of schools. For the immediate
dissemination ‐‐ for example, earmarking future, what is needed in any development
funds for inservice training programs ‐‐ program for formative assessment is to
would have to be pursued. study the interactions between these
external tests and formative assessments to
We must emphasize that this process will
see how the models of assessment that
inevitably be a slow one. To repeat what we
external tests can provide could be made
said above, if the substantial rewards promised
more helpful.
by the evidence are to be secured, each teacher
must find his or her own ways of incorporating All teachers have to undertake some
the lessons and ideas that are set out above into summative assessment. They must report to
his or her own patterns of classroom work. Even parents and produce end‐of‐year reports as
with optimum training and support, such a classes are due to move on to new teachers.
process will take time. However, the task of assessing pupils
summatively for external purposes is clearly
3. Reducing obstacles. All features in the
different from the task of assessing ongoing
education system that actually obstruct the
work to monitor and improve progress.
development of effective formative
Some argue that these two roles are so
assessment should be examined to see how
different that they should be kept apart. We
their negative effects can be reduced.
do not see how this can be done, given that
Consider the conclusions from a study of
teachers must have some share of
teachers of English in U.S. secondary
responsibility for the former and must take
schools.
the leading responsibility for the latter.20
Most of the teachers in this study were However, teachers clearly face difficult
caught in conflicts among belief systems and problems in reconciling their formative and
institutional structures, agendas, and values. summative roles, and confusion in teachersʹ
The point of friction among these conflicts minds between these roles can impede the
was assessment, which was associated with improvement of practice.
very powerful feelings of being
overwhelmed, and of insecurity, guilt, The arguments here could be taken much
frustration, and anger. . . . This study further to make the case that teachers
suggests that assessment, as it occurs in should play a far greater role in contributing
schools, is far from a merely technical to summative assessments for
accountability. One strong reason for giving

teachers a greater role is that they have Directions for future research could include

access to the performance of their pupils in a a study of the ways in which teachers
variety of contexts and over extended understand and deal with the relationship
periods of time. between their formative and summative
roles or a comparative study of the
This is an important advantage because
predictive validity of teachersʹ summative
sampling pupilsʹ achievement by means of
assessments versus external test results.
short exercises taken under the conditions of
Many more questions could be formulated,
formal testing is fraught with dangers. It is
and it is important for future development
now clear that performance in any task
that some of these problems be tackled by
varies with the context in which it is
basic research. At the same time,
presented. Thus some pupils who seem
experienced researchers would also have a
incompetent in tackling a problem under
vital role to play in the evaluation of the
test conditions can look quite different in the
development programs we have proposed.
more realistic conditions of an everyday
encounter with an equivalent problem.
Are We Serious About Raising
Indeed, the conditions under which formal
tests are taken threaten validity because Standards?
they are quite unlike those of everyday The findings summarized above and the
performance. An outstanding example here program we have outlined have implications for
is that collaborative work is very important a variety of responsible agencies. However, it is
in everyday life but is forbidden by current the responsibility of governments to take the
norms of formal testing.21 These points open lead. It would be premature and out of order for
up wider arguments about assessment us to try to consider the relative roles in such an
systems as a whole ‐‐ arguments that are effort, although success would clearly depend
beyond the scope of this article. on cooperation among government agencies,
academic researchers, and school‐based
4. Research. It is not difficult to set out a list of educators.
questions that would justify further research
in this area. Although there are many and The main plank of our argument is that
varied reports of successful innovations, standards can be raised only by changes that are
they generally fail to give clear accounts of put into direct effect by teachers and pupils in
one or another of the important details. For classrooms. There is a body of firm evidence that
example, they are often silent about the formative assessment is an essential component
actual classroom methods used, the of classroom work and that its development can
motivation and experience of the teachers, raise standards of achievement. We know of no
the nature of the tests used as measures of other way of raising standards for which such a
success, or the outlooks and expectations of strong prima facie case can be made. Our plea is
the pupils involved. that national and state policy makers will grasp
this opportunity and take the lead in this
However, while there is ample justification direction.
for proceeding with carefully formulated
projects, we do not suggest that everyone
else should wait for their conclusions. 1.
James W. Stigler and James Hiebert, "Understanding
Enough is known to provide a basis for
and Improving Classroom Mathematics Instruction:
active development work, and some of the An Overview of the TIMSS Video Study," Phi Delta
most important questions can be answered Kappan, September 1997, pp. 19-20.
only through a program of practical
implementation.

2.
There is no internationally agreed-upon term here. Range: Findings and Interpretations from a National
"Classroom evaluation," "classroom assessment," Evaluation Study in England," International Journal of
"internal assessment," "instructional assessment," and Science Education, vol. 17, 1995, pp. 481-92.
"student assessment" have been used by different 14.
authors, and some of these terms have different Phillipe Perrenoud, "Towards a Pragmatic Approach
meanings in different texts. to Formative Evaluation," in Penelope Weston, ed.,
Assessment of Pupils' Achievement: Motivation and School
3.
Paul Black and Dylan Wiliam, "Assessment and Success (Amsterdam: Swets and Zeitlinger, 1991), p. 92.
Classroom Learning," Assessment in Education, March 15.
1998, pp. 7-74. D. Royce Sadler, "Formative Assessment and the
Design of Instructional Systems," Instructional Science,
4.
Lynn S. Fuchs and Douglas Fuchs, "Effects of vol. 18, 1989, pp. 119-44.
Systematic Formative Evaluation: A Meta-Analysis," 16.
Exceptional Children, vol. 53, 1986, pp. 199-208. Paul J. Black and J. Myron Atkin, Changing the
Subject: Innovations in Science, Mathematics, and
5.
See Albert E. Beaton et al., Mathematics Achievement Technology Education (London: Routledge for the
in the Middle School Years (Boston: Boston College, Organisation for Economic Co-operation and
1996). Development, 1996); and Michael G. Fullan, with
6.
Suzanne Stiegelbauer, The New Meaning of Educational
Lynn S. Fuchs et al., "Effects of Task-Focused Goals Change (London: Cassell, 1991).
on Low-Achieving Students with and Without
17.
Learning Disabilities," American Educational Research See Stigler and Hiebert, pp. 19-20.
Journal, vol. 34, 1997, pp. 513-43. 18.
Black and Atkin, op. cit.
7.
OFSTED (Office for Standards in Education), 19.
Peter Johnston et al., "Assessment of Teaching and
Subjects and Standards: Issues for School Development
Arising from OFSTED Inspection Findings 1994-5: Key Learning in Literature-Based Classrooms," Teaching
Stages 3 and 4 and Post-16 (London: Her Majesty's and Teacher Education, vol. 11, 1995, p. 359.
Stationery Office, 1996), p. 40. 20.
Dylan Wiliam and Paul Black, "Meanings and
8.
Nicholas Daws and Birendra Singh, "Formative Consequences: A Basis for Distinguishing Formative
Assessment: To What Extent Is Its Potential to and Summative Functions of Assessment," British
Enhance Pupils' Science Being Realized?," School Educational Research Journal, vol. 22, 1996, pp. 537-
Science Review, vol. 77, 1996, p. 99. 48.
21.
9.
Clement Dassa, Jesús Vazquez-Abad, and Djavid Ajar, These points are developed in some detail in Sam
"Formative Assessment in a Classroom Setting: From Wineburg, "T. S. Eliot, Collaboration, and the
Practice to Computer Innovations," Alberta Journal of Quandaries of Assessment in a Rapidly Changing
Educational Research, vol. 39, 1993, p. 116. World," Phi Delta Kappan, September 1997, pp. 59-
65.
10.
D. Monty Neill, "Transforming Student
Assessment," Phi Delta Kappan, September 1997, pp. PAUL BLACK is professor emeritus in the School of
35-36. Education, King's College, London, where DYLAN
11.
WILIAM is head of school and professor of educational
Task Group on Assessment and Testing: A Report assessment.
(London: Department of Education and Science and
the Welsh Office, 1988).
Copyright 1998 Phi Delta Kappa International
12.
Richard Daugherty, National Curriculum Assessment:
A Review of Policy, 1987-1994 (London: Falmer Press,
1995).
13.
Terry A. Russell, Anne Qualter, and Linda
McGuigan, "Reflections on the Implementation of
National Curriculum Science Policy for the 5-14 Age

Inside The Black Box: Raising Standards Through Classroom Assessment

Uploaded by

Inside The Black Box: Raising Standards Through Classroom Assessment

Uploaded by

Inside the Black Box: Raising Standards Through Classroom Assessment by Paul Black and Dylan Wiliam

Illustration © 1998 by A. J. Garces