An Introduction to

EEM424 Design of Experiments

Mohd Nazri Mahmud

 AGENDA

• Course Outcomes
• Programme Outcomes
• Assessments
• Reference book
• Course contents
 Course Outcomes

CO1:
To be able to describe the procedure for planning experiments
based on modern experimental designs and to be able to
contrast the procedure with the traditional one-factor-at-a-time
design
CO2:
To be able to analyse data and draw conclusions from modern
experimental designs for comparative experiments
CO3:
To be able to analyse data and draw conclusions from modern
experimental designs involving two or more factors
 Programme Outcomes

PO3:
Design/Development of Solutions
Ability to design solution for complex engineering problems and
design systems, components or processes that meet specified
needs with appropriate consideration for public health and
safety, cultural, societal and environmental considerations.

PO4:
Investigation
Ability to conduct investigations of complex problems using
research-based knowledge and research methods including
design of experiments, analysis and interpretation of data,
synthesis of information to provide valid conclusions.
 Assessment

1. Assignment
– 4 assignments
– 20%
2. Test
– 2 tests
– 10%
3. Quiz
– 4 quizzes
– 10%
4. Final exam
– 4 questions
– 60%
Reference Book

• Design and Analysis of

Experiments
• Eighth edition
• Douglas C. Montgomery
CONTENTS
CONTENTS
CONTENTS
CONTENT
 Learning approach

• Active Learning – Problem-based

 What is an experiment?

• An experiment is a test or series of test in which

– purposeful changes are made to one or more variables of a
process or a system and
– the effect of those changes are observed, recorded and
analysed to achieve some pre-defined objectives.
 Why experiment?

• Investigators perform experiments in virtually all fields of

inquiry, usually to
– discover something about a particular process or system
– understand cause-and-effect relationships in a system
– identify the reasons for changes that may be observed in the
output response when we change the input variables
– to determine which input variables are responsible for the
observed changes in the response
– develop a model relating the response to the important input
variables and to use this model for process or system
improvement or other decision-making.
• Experimentation plays an important role in technology
commercialization and product realization activities
– new product design and formulation, manufacturing process
development, and process improvement.
 What is a designed experiment?

• Designed experiment refers to

– the statistical approach of planning an experiment
– so that experimental data could be produced, collected, and
analyzed systematically by statistical methods
– to generate results that could support valid and objective
conclusions
• When the problem involves data that are subject to
experimental errors, statistical methods are the only
objective approach to analysis.
• Thus, there are two aspects to any experimental
problem: the design of the experiment and the
statistical analysis of the data.
 Why designing experiment?

• Experimentation is a vital part of the scientific (or engineering)

method.
– most problems in science and engineering require observation of the
system at work and experimentation to elucidate information about why
and how it works.
– Well-designed experiments can often lead to a model of system
performance, called empirical models.
– These empirical models can be manipulated by a scientist or an engineer
• Experiments are used widely in the engineering world
– Process characterization & optimization
– Evaluation of material properties
– Product design & development
– Component & system tolerance determination
• “All experiments are designed experiments, some are poorly
designed, some are well-designed”
 Why designing experiment?

• The application of experimental design techniques

early in process development can result in
– 1. Improved process yields
– 2. Reduced variability and closer conformance to nominal or
target requirements
– 3. Reduced development time
– 4. Reduced overall costs.
 Why designing experiment?

• Experimental design methods are also of fundamental

importance in engineering design activities, where new products
are developed and existing ones improved.
• Some applications of experimental design in engineering design
include
– 1. Evaluation and comparison of basic design configurations
– 2. Evaluation of material alternatives
– 3. Selection of design parameters so that the product will work well under
a wide variety of field conditions, that is, so that the product is robust
– 4. Determination of key product design parameters that impact product
performance
– 5. Formulation of new products.
• The use of experimental design in product realization can result
in products that are easier to manufacture and that have
enhanced field performance and reliability, lower product cost,
and shorter product design and development time.
 Objectives of experiment

• Some typical objectives of experiment:

– To identify factors that are influential to the effects of interest
– Which factors are influential to the effects of interest (Factor
Screening or characterisation problem) or Screening
experiments
– To find the settings or levels of the influential factors that
would result in the desired effects- Optimisation problem
– To find the settings or levels of the influential factors so that
the desired effects are less sensitive to the variation in the
uncontrollable factors - Robustness problem
Some real applications
Some real applications
 Engineering Experiments

• Reduce time to
design/develop new products
& processes
• Improve performance of
existing processes
• Improve reliability and
performance of products
• Achieve product & process
robustness
• Evaluation of materials,
design alternatives, setting
component & system
tolerances, etc.
Airplane problem
 Experimental Strategies
• There are at least three strategies of experimentation:
– “Best-guess” experiments
– One-factor-at-a-time (OFAT) experiments
– Statistically designed experiments a.k.a factorial experiment
 Experimental Strategies
• “Best-guess” experiments
– select an arbitrary combination of assumed influential
factors, test them, and see what happens.
– frequently used in practice by engineers and scientists
– often works reasonably well, too, because the experimenters
often have a great deal of technical or theoretical knowledge
of the system they are studying, as well as considerable
practical experience.
– Indefinite guessing of factor combinations without guarantee
of the ‘best’
– Premature stop
 Experimental Strategies
• One-factor-at-a-time (OFAT) experiments
– selecting a starting point, or baseline set of levels, for each factor,
and then successively varying each factor over its range with the
other factors held constant at the baseline level.
– After all tests are performed, a series of graphs are usually
constructed showing how the response variable is affected by
varying each factor with all other factors held constant.
– select the optimal combination
– Used extensively
– fails to consider any possible interaction between the factors
– An interaction is the failure of one factor to produce the same
effect on the response at different levels of another factor.
– Interactions between factors are very common, and if they occur,
the one-factor-at-a-time strategy will usually produce poor results.
 Experimental Strategies
• Statistically designed experiments a.k.a factorial
experiment
– Factors are varied together, instead of one at a time.
– for studying the joint effects of two or more factors
 Basic principles of experimental
design
• Randomization
• Replication
 Basic principles of experimental
design
• Randomization
– Running the trials in an experiment in random order
– both the allocation of the experimental material and the order in
which the individual runs of the experiment are to be performed
are randomly determined
– Statistical methods require that the observations (or errors) be (or
assumed to be) independently distributed random variables.
– Randomization usually makes this assumption valid
– also assist in “averaging out” the effects of extraneous factors that
may be present.
– Computer software programs are widely used to present the runs
in the experimental design in random order using a random
number generator.
 Basic principles of experimental
design
• Replication
– an independent repeat run of each factor combination.
– Replication has two important properties
• allows the experimenter to obtain an estimate of the experimental
error.
• permits the experimenter to obtain a more precise estimate of the
sample mean
– There is an important distinction between replication and
repeated measurements
 Basic concepts and
terminologies
Run – one observation of an experiment
Noise – fluctuations in the observation of the individual runs
Sampling – taking a sample from a population for a study to draw conclusions
about that population
Random samples – any of the member of the population has an equal probability
of being chosen as a sample

Sample mean – a measure of the central tendency

Suppose y1, y2,….,yn represents a sample, the sample mean, y

n
 yi
i 1
y
n
 Basic concepts and
terminologies
Sample variance – a measure of the dispersion of the sample

 y 
n 2
i y
i 1
s 
2
n 1
Sample standard deviation – a measure of the dispersion of the sample

s2
 Planning, Conducting &
Analyzing an Experiment

1. Recognition of & statement of problem

2. Choice of factors, levels, and ranges
3. Selection of the response variable(s)
4. Choice of design
5. Conducting the experiment
6. Statistical analysis
7. Drawing conclusions, recommendations

DOX 6E Montgomery 33
 Problem-based Learning

• Planning a paper aeroplane experiment

1. Recognition of & statement of problem

2. Choice of factors, levels, and ranges
3. Selection of the response variable(s)
4. Ready to choose an experimental design
• Step 1: Specify the problem that needs
experimentation clearly
• A clear unambiguous statement of the problem
• What is your objective of carrying the experiment?
• Prepare a list of specific problems/questions that need answer
• Several reasons could be
– To identify factors that are influential to the effects of interest
» Which factors are influential to the effects of interest (Factor
Screening problem)
» Screening experiments
– To find the settings or levels of the influential factors that would result in
the desired effects- Optimisation problem
– To find the settings or levels of the influential factors so that the desired
effects are less sensitive to the variation in the uncontrollable factors -
Robustness problem
– To compare between two / more of different settings
• Step 2: Choose factors, levels and ranges.
– factors can be classified as either potential design factors or
nuisance factors.
– The potential design factors –
• those factors that the experimenter may wish to vary in
the experiment.
• further classification of them is helpful
– design factors, held-constant factors, and allowed-to-vary factors.
• Step 2: Choose factors, levels and ranges.
– The design factors
• the factors actually selected for study in the experiment.
– Held-constant factors
• variables that may exert some effect on the response, but for
purposes of the present experiment these factors are not of interest,
so they will be held at a specific level.
– allowed-to-vary factors
• the experimental units or the “materials” to which the design factors
are applied are usually nonhomogeneous
• we often ignore this unit-to-unit variability and rely on randomization
to balance out any material or experimental unit effect.
• We often assume that the effects of held-constant factors and
allowed-to vary factors are relatively small.
– Nuisance factors
• may have large effects that must be accounted for, yet we
may not be interested in them in the context of the present
experiment.
• often classified as controllable, uncontrollable, or noise
factors.
• A controllable nuisance factor is one whose levels may be
set by the experimenter.
– E.g. select different batches of raw material or different days of
the week when conducting the experiment.
– The blocking principle is often useful in dealing with controllable
nuisance factors.
• uncontrollable, but can be measured
– an analysis procedure called the analysis of
covariance can often be used to compensate for its
effect.
– probably can be measured and treated as a covariate
• Noise factor
– factor that varies naturally and uncontrollably in the process, but can be
controlled for purposes of an experiment
– objective is usually to find the settings of the controllable design factors that
minimize the variability transmitted from the noise factors.
– noise is usually called experimental error or simply error. It is a statistical
error, meaning that it arises from variation that is uncontrolled and generally
unavoidable.
– noise factors are often functions of environmental conditions such as
temperature or relative humidity.
– They may be properties of raw materials that vary from batch to batch or over
time in the process.
– They may be process variables that are difficult to control or to keep at
specified targets.
– In some cases, they may involve the way the consumer handles or uses the
product.
– Noise variables may often be controlled at the research or development
level, but they cannot be controlled at the production or product use level.
 Choose ranges and levels

– must choose the ranges over which these factors will be

varied and the specific levels at which runs will be made.
– Thought must also be given to how these factors are to be
controlled at the desired values and how they are to be
measured
• Step 3: Specify the response variable(s)
– It is usually critically important to identify issues related to defining
the responses of interest and how they are to be measured
– the experimenter should be certain that the chosen variable(s)
really provides useful information about the process/system under
study.
– Most often, the average or standard deviation (or both) of the
measured characteristic will be the response variable.
– Multiple responses are not unusual.
– The experimenters must decide how each response will be
measured, and address issues such as how will any measurement
system be calibrated and how this calibration will be maintained
during the experiment.
• Now you are ready to apply the steps to plan for our first
experimentation next week
 Next week:
A problem-based learning activity
• Planning for a simple comparative experiment
 Thank You
Transforming Higher Education For A Sustainable Tomorrow
Additional notes/slides
We lead