Scribd
Upload
61 views

Steps of Test Development

The document outlines the steps involved in test development, including test conceptualization, construction, tryout, analysis, and revision. Key steps include generating test items from theory and data collection, developing response scales, writing clear items without issues like double-barreled questions, conducting pilot testing and expert reviews, administering tests for exploratory factor analysis and confirmatory factor analysis, and assessing reliability and validity through internal consistency, test-retest reliability, and convergent and discriminant validity.

Uploaded by

Daksh Aneja
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
61 views

Steps of Test Development

The document outlines the steps involved in test development, including test conceptualization, construction, tryout, analysis, and revision. Key steps include generating test items from theory and data collection, developing response scales, writing clear items without issues like double-barreled questions, conducting pilot testing and expert reviews, administering tests for exploratory factor analysis and confirmatory factor analysis, and assessing reliability and validity through internal consistency, test-retest reliability, and convergent and discriminant validity.

Uploaded by

Daksh Aneja
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 7

STEPS OF TEST DEVELOPMENT

Test
conceptualization

Test construction

Test tryout

Analysis

Revision

1
TEST CONCEPTUALIZATION

Some preliminary questions

 What is the test designed to measure?  What is the ideal format of the test?

 What is the objective of the test?  Should more than one form of the test be developed?

 Is there a need for this test?  What special training will be required of test users for
administering or interpreting the test?
 Who will use this test?
 What types of responses will be required of testtakers?
 Who will take this test?
 Who benefits from an administration of this test?
 What content will the test cover?
 Is there any potential for harm as the result of an administration of
 How will the test be administered?
this test?

 How will meaning be attributed to scores on this test?

2
TEST CONSTRUCTION

 Item generation

 From theory

 From ground – primary data collection/FGDs/ expert interviews

 No. of items?

3
RESPONSE SCALES

 Summative scales

 Likert scale

 Paired comparisons

 Comparative scaling

 Categorical scaling

 Guttman scale

 All respondents who agree with the stronger statements will also agree with milder statements

4
TEST CONSTRUCTION
Item writing – good & bad items

 Exceptionally lengthy items X

 CoP scale example

 Reading difficulty levels

 Multiple negatives X

 ‘I am not in favor of corporations stopping funding for anti-nuclear groups’ OR

 ‘I favor private corporations’ support of groups advocating a nuclear ban.’?

Item writing – good & bad items

 Double-barreled questions X

 ‘I like working in teams because I like to meet people’

 ‘I like working and brainstorming in groups’

 Ambiguous sentences X

 Positively and negatively worded items (acquiescence bias)

 ‘I feel that I have a number of good qualities’ (self-esteem)

 ‘I certainly feel useless at times’ (self esteem)

5
TEST CONSTRUCTION
• Item pool
1. Pilot testing • Expert rating/peer reviews/ testtaker reviews – iterations
• Item selection – Version 1

• Sample (1:4 to 1:10)


2. Test administration

• Analysis of factor loadings, factor structure, some items dropped


3. EFA (Exploratory Factor Analysis) • Version 2

• Understanding the factor structure


4. CFA (Confirmatory Factor
Analysis)

• Internal consistency reliability/ test-retest


5. Reliability and validity assessments • Convergent. Discriminant validity

6
SUMMARY OF STEPS

1. Intercorrelations between variables 3. No. of factor to be extracted

1. Adequate intercorrelations should exist between variables (several >0.3) 1. With Eigen Values >=1.

2. KMO MSA must be >0.5, .08 or higher is great; Bartlett’s test should be 2. According to Scree plot – till the graph starts becoming a straight line
significant
3. Decision on 1 or 2 should be based on literature about the concept.
3. Check diagonal of anti-image correlation matrix (gives KMO for individual
variables. Must be >0.5. Remove any item which does not meet this 4. Factor loadings
criterion)
1. Minimum acceptable 0.3 to 0.4 – depends on sample size (see the table in
provided text)
2. Extraction method – Principal Component Analysis with
2. Remove the variables with less than the minimum acceptable loadings
Varimax rotation
3. If a variable is cross-loading on 2 factors:
1. Check communalities of variables – remove those <0.5 starting with the
smallest value. 1. If the cross loading difference in <0.2 – better to remove the item

2. Run the analysis each time after removing an item as intercorrelations and 2. If cross-loading difference is >0.2, decide on the basis of the higher
communalities will change each time. factor loading + where does the variable fit better conceptually between
2 factors + what does the literature say, and take a decision

You might also like