RES 320

Social Research –
Methodological Thinking
Week 13
Department of Psychology
Faculty of Humanities

Professor Eugene L Davids

Unit 11
Inferential statistics
Recap: Inferential statistics
• In inferential statistics, researchers attempt to go beyond
summarising their data as done in descriptive statistics

• Inferential statistics are largely concerned with

• estimation
• estimate the value of population parameters

• hypothesis testing
• test hypotheses about population parameters

• Parameter is a numerical characteristic of a population

• In inferential statistics,
• researchers use sample data to make generalisations about
populations

• if you calculate a numerical index such as a mean or a correlation

coefficient on sample data, it is called a statistic

• if you were able to calculate a numerical index (such as a mean or a

correlation coefficient) using the data from an entire population, it would
be called a population parameter

• the goal of inferential statistics is to understand population parameters

• Researchers must use sample data to understand populations,
however, because it is rarely feasible to collect data from
everyone in the population of interest
• random sampling is assumed in inferential statistics, because these
samples follow the laws of probability and allow researchers to make
warranted claims about population parameters

• Different symbols to represent statistics and parameters

Recap: Statistics and parameters
Recap: Hypothesis testing
• Hypothesis testing is the branch of inferential statistics concerned with
how well sample data support a null hypothesis and when the null
hypothesis can be rejected
• in hypothesis testing the researcher states the null and alternative hypotheses and
then uses the data to determine what decision should be made about these
hypotheses

• Null hypothesis is a statement about a population parameter

• typically, it states that there is no relationship between the independent and
dependent variables
• Alternative hypothesis is the logical opposite of the null hypothesis
• stating that there is a relationship between the independent and dependent
variables
Logic of hypothesis testing
• Step 1
• Assume that an effect is not present or that there is no true
relationship
• between your independent and dependent variables,
• and then you determine whether the data warrant rejection of the null
hypothesis

• Do males or females have higher mean starting salaries in the

populations of male and female recent college graduates
• Null hypothesis: H0: μM = μF
• Alternative hypothesis: H1: μM ≠ μF
• Step 2
• Now you must decide what alpha level you want to use
• also called the level of significance
• set by the researcher at some small value (typically .05),
• it is the point at which the researcher would conclude to reject the null
hypothesis
• set by the researcher before analysing the data

• If you set alpha at .05, you will incorrectly reject the null
hypothesis only 5% of the time or less
• You will only conclude 5% of the time that there is a relationship in the
population, when there really is not a relationship
• this error a “Type I” error
• Step 3
• Next, you input the data into a statistical programme such as
SPSS and run the appropriate statistical test

• When the means for two groups of people are being compared,
the most common statistical test is the independent samples t-
test
• probability value (or p value), a value between 0 and 1
• a very small p value provides the evidence you need to reject the null
hypothesis
• point at which you will decide to reject the null hypothesis will be
determined by the alpha level
• Remember: if the p value is less than (or equal to) the alpha level,
then reject the null hypothesis and tentatively accept the
alternative hypothesis

• When researchers reject the null hypothesis, they are warranted

in claiming that their research finding is statistically significant
Application
• Now we can complete our test of the null hypothesis about the
starting salaries of male and female recent college graduates.
• We are using the independent samples t test.
• Following convention, we set our alpha level at .05. We entered the
data into the statistical package SPSS, ran the independent samples
t test, and found that the value on the t distribution for our test
statistic was 2.18. More importantly, the p value was equal to .04.
• p value is less than .05 = reject null hypothesis, and accept alternative
hypothesis
• Means = Statistically significant difference between male and female
starting salaries
Application
• In a study investigating whether students who participate in
extracurricular activities have higher average exam scores than
those who do not, an independent samples t-test was conducted.
• The results showed that students who participate in
extracurricular activities (M = 3.4, SD = 0.5) had a higher average
exam score compared to those who did not participate (M = 3.1,
SD = 0.6), with t(58) = 2.45, p = 0.018. With an alpha level of 0.05,
students need to determine if the observed difference in average
exam score is statistically significant based on the p-value
provided.
• Determine the H0 and H1

• Determine whether you will reject or accept the null hypothesis

• What do the results mean

• Determine the H0 and H1
• H0: There is no difference in average exam scores between students who
participate in extracurricular activities and those who do not
• H1: There is a difference in average exame scores between the two groups

• Determine whether you will reject or accept the null hypothesis

• Reject H0
• p value = .018 and alpha level = .05

• What do the results mean

• the difference in average exam scores between students who participate
in extracurricular activities and those who do not is statistically
significant
• this suggests that participation in extracurricular activities is associated
with higher average scores, leading us to reject the null hypothesis
• In addition to knowing statistical significance
• We are also interested in the practical significance
• also called clinical significance
• is a subjective but carefully considered decision made by the researcher
about whether the difference between the means or the observed
relationship is “big enough to matter” for practical decisions
• e.g., to continue the line of research, to make policy decisions, or to make
clinical recommendations
• To aid in making judgments about practical significance,
researchers frequently use what are called effect size indicators
• effect size indicator is a measure of the magnitude or strength of a
relationship
• how big an effect or difference between means is present

• Many different effect size indicators, such as Cohen’s d, eta

squared (η2), omega squared (ω2), and the amount of variance
explained by one or more independent variables
Directional alternative hypotheses
• Sometimes a researcher will state an alternative hypothesis in a
directional form rather than in a non-directional form during
statistical testing
• to test the hypothesis that one population mean is greater than (or less
than) another

• Non-directional alternative hypothesis is an alternative

hypothesis that includes a “not equal to” sign ≠
• Directional alternative hypothesis contains either a greater
than sign (>) or a less than sign (<)
• Alternative hypothesis: H1: μTraining > μNo Training
Hypothesis-testing errors
• Because samples rather than complete populations are used in
inferential statistics, hypothesis testing can sometimes provide
the wrong answer
Type I and Type II errors
• Imagine a psychology study testing whether a new cognitive-
behavioural therapy (CBT) programme is more effective than traditional
therapy in reducing symptoms of anxiety. A Type I error would occur if
researchers conclude that the new CBT programme is more
effective when, in reality, there is no true difference between the
two therapies; they have incorrectly rejected a true null hypothesis.
This might lead to the programme being widely adopted based on false
efficacy. Conversely, a Type II error would happen if researchers
conclude that there is no difference between the therapies when, in
fact, the new CBT programme is actually more effective. In this case,
they fail to reject the null hypothesis and might overlook a genuinely
beneficial therapy for anxiety, potentially depriving patients of an
effective treatment option.
• Type I error
• False positive

• Type II error
• False negative
Hypothesis testing in practice
• For each statistical test that you run, you will always apply the
logic of hypothesis testing (Table 15.3)
t Test for correlation coefficients
• Correlation coefficients show the strength and direction of the
relationship between two quantitative variables

• t test for correlation coefficients is the statistical test used to

determine whether an observed correlation coefficient is
statistically significant
One-way analysis of variance
• One-way analysis of variance
• also called one-way ANOVA
• is used to compare two or more group means for statistical significance
• used when you have one quantitative dependent variable and one
categorical independent or predictor variable

• Two-way ANOVA is used when you have two categorical

independent variables, three-way ANOVA is used when you
have three categorical independent variables, and so forth
Post hoc tests in analysis of variance
• One-way ANOVA, if the categorical variable has only two levels,
then a statistically significant outcome is interpretable; just look
to see which mean is larger and conclude that mean is
significantly larger than the other mean

• Three or more means, however, a statistically significant

outcome for the one-way ANOVA
• must be followed up with post hoc tests to determine which of the
means are significantly different
• Cannot do independent samples t-test for multiple pairs, as it
would inflate the probability of making a Type I error
• Use post hoc test to control for this – by adjusting (correcting) p values
• Popular post hoc test include:
• Newman-Keuls test
• Sidak test
• Bonferroni test
Analysis of Covariance
• Analysis of covariance
• also called ANCOVA
• is used when you have a quantitative dependent variable and a mixture
of categorical and quantitative independent variables

• ANCOVA is the statistical test used for experimental designs that

have pre-tests and more than one group, such as the pretest-
posttest control-group design and the non-equivalent
comparison-group design
One-way repeated measures analysis of
variance
• One-way repeated measures analysis of variance
• also called one-way repeated measures ANOVA
• is used when you have one quantitative dependent variable and one
within-participants independent variable

• With a within-participants independent variable,

• the same participants are measured more than once
• this is the appropriate analytical procedure for a repeated measures
design as well as for the one-group pretest-posttest designs
t Test for regression coefficients
• t test for regression coefficients
• is used to test the significance of the regression coefficients obtained in
regression analysis

• Previously, we looked at
• simple regression, the researcher analyses the relationship between
one quantitative dependent variable and one quantitative
independent or predictor variable
• multiple regression, the researcher analyses the relationship between
one quantitative dependent variable and two or more quantitative
independent or predictor variables
Chi-square test for contingency tables
• chi-square test for contingency tables
• is used to determine whether a relationship observed in a contingency
table is statistically significant

• We used the effect size indicator called Cramer’s V

• to determine the strength of the relationship
• the size of this effect size indicator can be interpreted as the size of a
correlation coefficient
Summary
• 6 steps in hypothesis testing (Table 15.3)

• Some statistical tests

• t-Test for correlation/regression coefficients
• ANOVA
• Post hoc tests
• ANCOVA
• One-way repeated ANOVA
• Chi-squared test for contingency tables
 Thank You
Next Lecture
Revision Lecture Prof Eugene L Davids

Student Evaluation: Room 11-30 (Humanities Building)

Please keep an eye out for this
email and complete Consultation: Tuesdays 9h00 – 11h00
(by prior email arrangement)
Exam
• Units 7 – 11 Email: eugene.davids@up.ac.za
• 12 November, 7h30
• 90 minutes