Teaching, Learning and Assessing Statistical Problem Solving
John Marriott, Neville Davies and Liz Gibson
Nottingham Trent University, UK
Journal of Statistics Education Volume 17, Number 1 (2009), www.amstat.org/publications/jse/v17n1/marriott.html
Copyright © 2009 by John Marriott, Neville Davies and Liz Gibson, all rights reserved. This text may be freely shared among
individuals, but it may not be republished in any medium without express written consent from the authors and advance notification of
the editor.
Key Words: Problem solving; Teaching; Learning; Assessment.
Abstract
In this paper we report the results from a major UK government-funded project, started in 2005, to review statistics and handling data
within the school mathematics curriculum for students up to age 16. As a result of a survey of teachers we developed new teaching
materials that explicitly use a problem-solving approach for the teaching and learning of statistics through real contexts. We also
report the development of a corresponding assessment regime and how this works in the classroom.
Controversially, in September 2006 the UK government announced that coursework was to be dropped for mathematics exams sat by
16-year-olds. A consequence of this decision is that areas of the curriculum previously only assessed via this method will no longer be
assessed. These include the stages of design, collection of data, analysis and reporting which are essential components of a statistical
investigation. The mechanism outlined here could provide some new and useful ways of coupling new teaching methods with learning
and doing assessment – in short, they could go some way towards making up for the educational loss of not doing coursework. Also,
our findings have implications for teaching, learning and assessing statistics for students of the subject at all ages.
1. Introduction
The state of mathematics education in the United Kingdom has been an issue of concern for some years. Following a commission by
the UK government, Smith (2004) published a report into post-14 mathematics provision. His report made wide-ranging
recommendations for improving mathematics education in schools in England. Based on one of the recommendations, in 2005 the
Qualifications and Curriculum Authority (QCA) commissioned the Royal Statistical Society Centre for Statistical Education
(RSSCSE) to review the position of the teaching of statistics and handling data (S&HD) in the curriculum. (The three authors of this
paper comprised the RSSCSE/QCA review team.) Indeed, Smith had suggested that S&HD education might be improved by teaching
it through other subjects, such as science and geography, rather than from its current position in the mathematics curriculum.
In autumn 2005 the RSSCSE/QCA Review carried out a national survey of heads of mathematics, geography and science to determine
their views, needs and capabilities within the S&HD area. We were surprised to find that teachers of science and geography appeared
more confident than those of mathematics when teaching school students to understand and interact with statistical concepts and ideas
- skills that are fundamental for getting students to develop their statistical literacy.
For statutory curriculum reasons at this time it was not possible for the RSSCSE/QCA Review project to recommend the teaching of
S&HD solely through other subjects and so the Smith suggestion cited in the first paragraph could not be implemented. However,
examining the approach to the teaching and learning of S&HD within the science, geography and other curricular it was noted from
the survey that the key differences in teaching were in the application of S&HD to real contexts and problems arising from within
each subject. This corresponds with the growing body of opinion that has been suggesting that statistics is best taught through problem
solving. These research findings led the project team to develop a set of learning and teaching resources for use in mathematics
lessons by mathematics teachers that draw on real problems in real contexts. Alongside the teaching materials a new assessment
regime was developed that is designed specifically to test the effectiveness of teaching and learning in a problem solving setting. In
this paper we describe the strategy we used to produce the resources and assessment. The full report can be obtained by emailing the
first named author ([email protected]).
In the next section we review the way teachers of statistics have gradually proposed changing the way statistics is taught to make it
more relevant, with some making the case for using a problem solving approach entirely. As part of the RSSCSE/QCA Review we
devised a portfolio of problems through which some topics in statistics can be taught and we report the development of resources to
support this approach in section 2. In section 3 we describe one of the resources that we produced in some detail and also briefly
describe how they were trialed in schools. In section 4 we report the development of methods for assessing this approach and present
an analysis of the results obtained from trialing the new assessment. In section 5 we present some conclusions and suggest that our
findings could be used for the development of teaching, learning and assessment of statistics at all levels.
2. Learning and Teaching Through Problem Solving
A simple paradigm for solving problems using statistics is summarised in the English National Curriculum using four activities:
specify the problem and plan; collect data from a variety of suitable sources; process and represent the data; and interpret and discuss
the results. The activities are cyclic (illustrated in Figure 1) because it may be necessary to refine the initial approach to solving a
problem and repeat the process over again. Throughout this paper we refer to this four stage cycle as the (statistical) problem solving
approach (PSA). It is also known as ‘the handling data cycle’. There are variations of this cycle that are currently being used – for
example Wild and Pfannkuch (1999) proposed a five-stage procedure that is closely allied with the cycle illustrated in Figure 1.
Figure 1. The statistical problem solving approach
It was from the early to mid 1990s that literature began to emerge which was explicit in advocating the use of the PSA (see, for
example, Chatfield 1995 and 2002) for the teaching of statistics. Stuart (1995 and 2003) discusses the dominance of mathematical
thinking in statistics education and suggests the PSA as being a good counter-measure to this. Garfield (1995) and Garfield and Ben-
Zvi (2007) summarise educational research views on statistical learning which suggest that teaching statistics through solving
problems is considered to improve students’ skills, particularly as they interact with real data, see also Cobb (1992) and Cobb and
Moore (1997).
More recently Franklin and Mewborn (2006) reported that the American Statistical Association has endorsed the reports from the
Guidelines for Assessment and Instruction in Statistics Education (GAISE) project which advocate the active learning of statistics
using real data and a problem solving approach. Rossman et al (2006) found that using the problem solving approach in the teaching
of statistics is of great benefit to both teachers and learners, a view also supported by Groth (2006).
The PSA has been included in the English National Curriculum since 2000. However, the survey of heads of mathematics carried out
in 2005 by the RSSCSE/QCA Review project suggested that the good intentions implied by this curriculum specification were never
really carried through into the taught, learned or assessed versions of the same curriculum. The Review found compelling evidence
that even heads of mathematics departments in secondary schools were not confident about teaching the PSA.
Following the decision to abolish coursework in September 2006, in October the UK curriculum development organisation
Mathematics in Education and Industry (MEI, 2006) produced an evaluation of the role of this method of assessment in mathematics.
It provides evidence that the intended outcome of coursework was not achieved. The full report ‘Coursework in Mathematics’ is
available from their website. It notes that many factors contributed to the negative opinion that teachers had formed of the handling
data part of the coursework, with the pressure of time being one of the most important. The RSSCSE/QCA Review found that many
teachers did not fully understand the importance of the PSA, or indeed how it worked in practice. In addition to this the demands and
nature of assessment needed for the coursework meant that the full cyclical nature of the approach was neither taught nor assessed. It
appears that for many teachers the PSA will not have formed a key part of statistics courses they will have studied or attended. The
RSSCSE/QCA Review project came to an early conclusion that there was a pressing need to both help teachers develop professionally
in this area and also to produce materials for use in the classroom.
Carrying out the problem solving cycle depicted in Figure 1, using real data in real-world problem contexts, requires a number of
different cognitive skills. Thus, in devising a teaching, learning materials and assessment regime that can be used to grade students’
problem solving skills, these cognitive skills need to be identified. Different forms of learning also need to be recognized. Bloom et al.
(1956) published a taxonomy of educational objectives which was later revised by Anderson and Krathwol (2001). The six categories
of Anderson’s revised taxonomy are: remembering; understanding; applying; analysing; evaluating; and creating. These categories are
considered to be a hierarchy of skills – although there is some educational debate as to whether they are also progressive.
In considering the development of the teaching, learning and assessment resources that use the PSA we completed a mapping from the
handling data specification within the English national curriculum for mathematics onto the categories of the revised taxonomy. Table
1 presents this mapping and shows that each stage in the cycle demands the use of at least four levels of the taxonomy. For example,
even at the first stage of the PSA, the only category that is not used to any extent is ‘evaluating’.
Anderson and Krathwohl (2001) also introduced a second attribute/category which they refer to as the knowledge dimension. The
categories of this dimension, representing the outcomes of the thinking process, are factual, conceptual, procedural and metacognitive.
The classification in Table 1 was used, together with a two way table that combines the cognitive process dimension with the
knowledge dimension, to produce a mapping of the learning objectives of the statistical PSA onto the resulting two way classification.
This table is not reproduced here, but can be viewed at www.rsscse.org.uk/qca/doc/PSAtwowaymap.pdf. This process then naturally
identified our starting point for the development of both the teaching and learning materials and the assessment and its associated
questions.
Table 1: Revised Taxonomy and the Problem Solving Approach
Stage of problem Descriptor Level in taxonomy
solving approach (from the QCA specification) (from the revised taxonomy)
1. Specify the problem formulate questions in terms of the Remember: recognising; recalling
and plan data needed, and consider what Understand: interpreting;
inferences can be drawn from the exemplifying; explaining
data; decide what data to collect Apply: executing
(including sample size and data
Analyse: differentiating;
format) and what statistical analysis
organising
is needed
Create: planning; producing;
generating
2. Collect data collect data from a variety of Remember: recognising; recalling
suitable sources, including Understand: classifying;
experiments and surveys, and comparing
primary and secondary sources Apply: executing; implementing
Analyse: organising
Create: planning
3. Process and turn the raw data into usable Remember: recognising; recalling
represent the data information that gives insight into Understand: interpreting;
the problem exemplifying; classifying;
summarising
Apply: executing; implementing
Analyse: differentiating;
organising
4. Interpret and answer the initial question by Remember: recalling
discuss the data drawing conclusions from the data Understand: interpreting;
exemplifying; summarising;
inferring; comparing; explaining
Analyse: differentiating;
organising; attributing
Evaluate: checking; critiquing
Create: generating; producing
From January to September 2006 the RSSCSE/QCA Review project produced eight exemplar resources for teaching statistical topics
through the PSA. These were designed, written, trialed and refined using input from practicing teachers in secondary schools in
England. The resources are freely available from the web site www.rsscse.org.uk/qca. These materials are designed to support teachers
in delivering statistical concepts in a holistic manner. A simple and appealing version of the handling data cycle diagram was used
(see Figure 2) which is reinforced and repeated throughout the delivery and presentation of the materials.
These resources differ from what has previously been used in schools in two important respects. The first, and arguably the most
important, part of the PSA occurs at the planning stage of the cycle. These resources lead students into a detailed discussion of the
problem and come to a decision that data can assist them in addressing the problem and attempt to seek a solution. The students then
decide what data they think would be the most helpful and they discuss and arrive at a decision as to how best these data might be
collected. This experience draws the students into the problem and establishes their ownership of the procedures that are to follow
and, therefore, of the solutions that are eventually formulated. The second important feature of the materials is that there is regular
reinforcement of the cycle involved in the PSA with constant reminders of the current stage of the process as the investigation
proceeds. These two features serve to establish and reinforce the statistical PSA as a natural and powerful evidence-based and logical
approach to solving problems.
Figure 2. Schematic diagram of the problem solving approach
The teaching materials and supporting documentation are designed for formative and summative assessment. Teachers are provided
with notes which they can draw on as much or as little as they need to. These contain suggestions for discussion and so support
teachers in their use of questioning for formative assessment. There are also worksheets which allow teachers to get quick insights into
learners’ understanding throughout the lesson. Summative approaches to assessing the materials enabled us to judge the effectiveness
of the materials in terms of the learning in the classroom.
3. New Resources for Teaching and Learning - an Example
In order to fully illustrate the manner in which the PSA is used in the resources we have developed, we describe in detail one of them,
called "How safe is your area?" On the website for this resource (www.rsscse.org.uk/qca/Resource1.htm) the teacher can find a
teacher’s overview of the resource, access to both PowerPoint and over head projector (OHP) versions of the resource and a pupil
worksheet and pupil questionnaire. We discuss the PowerPoint version in this section and PowerPoint notes that advise the teacher on
the use of the presentation are also available to download. These duplicate in large part the notes provided within the PowerPoint file
itself.
3.1. The resource
The presentation opens with the slide in Figure 3 and a statement of the lesson objectives
Objectives
Children should learn:
● to relate individual statistical techniques to a wider problem;
● to think analytically about a statistical problem;
● to apply a variety of techniques to solve a problem.
The four stages of the PSA are clearly marked at the top of each slide and are colour coded throughout the presentation to provide
visual cues to the pupils. Additional visual cues are provided by associating different icons with each stage in the cycle. During any
stage this icon is displayed together with an enlarged colour coded box to allow pupils to associate their current activities with the
stage of the cycle they are currently working on. These points are illustrated for the PLAN stage in Figure 3 and Figure 4.
Figure 3 Introductory slide in "How safe is your area?" Figure 4 Initial slide for the PLAN stage
As the first slide for each stage of the PSA appears, the associated notes for the teacher show the objectives. For the PLAN stage these
are
Objectives
Children should learn:
● the context of the problem;
● to ask questions about presented information;
● to hypothesise.
The teacher is then advised to begin with a quick discussion of the media and reporting of crime – some example questions are
provided:
● What was the most recent crime that made the headlines?
● What kinds of crimes tend to make the headlines?
● Why do they make headlines?
These questions are presented on a slide that is then followed by slides that enable pupils to learn the context of the problem by
discussing questions that relate to the incidence of crime in different regions, the perceived relative ‘importance’ of different types of
crime and how they are recorded by the authorities.
Figure 5 Different approaches to the problem
The class could address the "How safe is your area?" problem by investigating the perception of crime. This will require them to
conduct their own survey (primary data), or they could study actual crime figures (secondary data) from government websites to
answer the same question (with less processing of data at the initial stage). Once the appropriate choice has been made (see Figure 5)
