ICT615 Information Technology Research Methods
Topic 5
Experimental design
Question 01
What hypotheses are being tested? What are the independent and dependent
variables of interest?
The level of performance targets will influence people’s perceived
competence, such that people provided with overly challenging targets
will have lower levels of perceived competence than people provided with
moderate performance targets.
Independent variable: Target
Dependent variable: performance
What population do you think the authors would like to generalise their results to?
The author targets the Business community. And the sample of this community author take
business student.
Have the subjects been randomly selected from the population of interest? If not,
how were they selected.
No, it is very difficult to select the sample from population. And it’s depended on the
scenario.
What is the experimental treatment?
According to this hypotheses and variables the lab environment is used.
Was a control group used?
No there is no control group is used in this scenario.
How were the variables operationalised (measured)?
We used crowdsourcing to measure the variables.
What did the experiment show? Were the hypotheses supported?
Yes, the hypothesis is supported.
Question 02
What hypothesis would you test?
Teaching users a technique to remember passwords will result in users creating stronger
passwords compared to users who do not receive the technique.
Selecting relevant variables – what variables will need to be measured? How will
they be measured?
Independent variable: teaching to remember passwords.
Dependent variable: password strength
Dependent variable will be measured by password strength meter or password strength or
maybe password complexity.
Specify levels of treatment – what treatments are applied in this study?
The treatment group will receive instruction on using a password manager tool, which
generate complex and difficult password combination. But on the other hand, control group
will not receive any instruction.
Control of experimental environment – how will you ensure consistency?
The experiment will be conducted in a controlled environment where the same computer
hardware and software will be used. The participants will be instructed to use the same
website to create password.
Choose the experimental design – what sort of experimental design will be used?
How many groups? What size? Is there a control group?
In this case we use “True” lab experiment. So, we have two groups (treatment and control).
The sample size will be based on power analysis, and the participants will be randomly
assigned to each group.
Select and assign subjects – how will subjects be selected? What population are they
representative of?
Participants will be selected from general population, with having no prior knowledge of the
password manager tool. The participants should be representing their group of population in
term of computer knowledge and age.
Pilot test, revise, test – what sort of piloting will be done?
Pilot test will be conducted to ensure that the experimental design and
procedures are feasible and practical. The pilot test will also help to refine the
measurement instruments and instructions given to participants. Based on the
result of test, the experimental design may be revised before the actual study is
conducted.
