Teamwork in Multi-Person Systems: A Review and Analysis
Teamwork in Multi-Person Systems: A Review and Analysis
8, 1052± 1075
As the scope and complexity of modern task demands exceed the capability of
individuals to perform, teams are emerging to shoulder the burgeoning
requirements. Accordingly, researchers have striven to understand and enhance
human performance in team settings. The purpose of this review is to summarize
that research, from the theoretical underpinnings that drive it, to the
identi® cation of team-level elements of success, to the methodologies and
instruments that capture and measure those characteristics. Further speci® ed are
three important avenues to creating successful teams: team selection, task design
and team training. In other words, one can select the right people, provide them
with a task engineered for superior performance and train them in the appropriate
skills to accomplish that task. Under task design, new technologies and
automation are examined that both support and impede team functioning.
Finally, throughout are provided critical remarks about what is known about
teamwork and what is needed to be known to move the science and practice of
team performance forward. The paper concludes with the identi® cation of team
issues that require further investigation.
Ergonomics ISSN 0014-013 9 print/ISSN 1366-584 7 online Ó 2000 Taylor & Francis Ltd
http://www.tandf.co.uk/journals
Teamwork in multi-person systems 1053
strategies to help respond to change (Dyer 1984, Modrick 1986, Morgan et al. 1986,
Salas and Cannon-Bowers 1997). To understand eVective team performance or
`teamwork’, one must understand how groups of individuals function to produce
eVectual synchronized output, rather than just summed or aggregated responses
(Steiner 1972, Hackman and Morris 1975, Nieva et al. 1978, Meister 1985,
Fleishman and Zaccaro 1992). But, what precisely is `teamwork’ ?
2. Theoretical developments
The ® rst serious attempts to study team processes began in the 1950s and 1960s, with
a focus largely on military teams and team processes that enabled them to function
more eVectively under conditions of extreme time pressure, high stress, ambiguous
and incomplete information, and severe consequences for actions taken. Much of the
impetus for team research over the years has been tied to team failures, particularly
those associated with high visibility (e.g. aircraft accidents, military accidents) (Ilgen
1999).
Social psychological approach: pertaining to the social and psychological implications of team
members’ relationships/interactions with one another
Interaction Processes (Hackman and Morris 1975)
Normative Model of Group EVectiveness (Hackman 1983, 1987)
Ecological approach: relating to the spacing and interdependence of team members and
institutions, or the relationships of team members with their organizational or working
environment
Group and Organizational Boundaries (Sundstrom and Altman 1989, Sundstrom et al.
1990)
Human resource approach: focusing on the utilization of human capabilities and talents
Human Resource Management (Shea and Guzzo 1987)
Lifecycle approach: pertaining to changes within a team that result from its maturation or
evolution over a lifecycle
Time and Transition Model (Gersick 1985, 1988)
Team Evolution and Maturation Model (TEAM) (Morgan et al. 1986, Morgan et al. 1994)
¯ exibility (Prince and Salas 1993), potency (Guzzo et al. 1994), cohesion (Mullen and
Copper 1994), performing self-correction (McIntyre and Salas 1995), using closed-
loop communication (McIntyre and Salas 1995), exhibiting assertiveness (Smith-
Jentsch et al. 1996b), predicting each others’ behaviour (Volpe et al. 1996), and four
speci® c skill dimensions (information exchange, communication, supporting
behaviours, and team initiative/leadership) (Smith-Jentsch et al. 1998a,b). Recently,
Cannon-Bowers et al. (1995) and Salas and Cannon-Bowers (2000) summarized
teamwork dimensions into three primary categories: cognitions, skills and attitudes.
Cognitions (or knowledge) include cue strategy associations, task speci® c team-mate
characteristics, shared task models, team mission, objectives, norms, and resources,
task sequencing, accurate task models and problem models, team role interaction
patterns, teamwork skills, boundary spanning roles, and team orientation.
Behaviours (or skills) consist of adaptability, shared situational awareness, mutual
Teamwork in multi-person systems 1055
Process
· Cognitive processes · Information exchange
· Position-speci® c taskwork skills · Communication
· Supporting behavior
Outcome
· Team leadership
increase a team’s functioning. Before one can plan methods for improving team
performance, however, one must ® rst evaluate those factors (input and process
variables) that impinge on it. Table 3, adapted from the taxonomies of Morgeson et
al. (1997) and Meister (1985), enumerates the most signi® cant of these.
Con® guration of table 3 factors serves to in¯ uence, positively or negatively, the
manner in which a team performs. With these determinants in mind, the ® rst
intervention Ð team selection Ð is presented.
the following types of teams: command and control teams, production teams,
customer service teams, professional/technical decision-making teams, and executive
teams. What types of traits or skills are needed for each Ð functional ¯ exibility or
highly specialized skills? Ability to deal with people and handle personal dynamics?
Ability to deal with poorly de® ned problems? Once a determination is made with
respect to basic abilities, team staYng requires that one goes further and address how
selection choices may in¯ uence performance outcomes of the team as a whole. There
is a need, therefore, to understand not only individual eVectiveness, but also team
eVectiveness as well, and to evaluate probable interactions between group-and
individual-level factors. This is where the theories and models of team eVectiveness
presented earlier provide some basis for action.
Success depends not only on knowledge, skills and attitudes (KSA) required for
individual task performance, but also on those traits of individual team members
that facilitate team interaction and functioning (e.g. learning ability, initiative, risk-
taking propensities, adaptability, tolerance for stress, etc.). These characteristics are
critical for teams that require more coordination, such as command and control
teams and traditional assembly line (production) teams. They are less important in
collaborative tasks where team interaction is less restricted. Similarly, service teams
would look on the ability to adapt team tasks to environmental demands as a
desirable attribute. Leadership qualities in¯ uence the performance of all teams,
whether they are exhibited by formally appointed leaders, or by non-appointed,
emerging leaders (Klimoski and Jones 1995).
4.1.2. Team size: Team selection involves establishing appropriate team size. This
becomes problematic when the task has not been performed before, or when it is
arti® cially constrained by such factors as leader preferences, available resources, or
the number of people free to participate. If too few people are chosen, undue stress
will be placed on team members; on the other hand, if too many are chosen,
resources are wasted (Klimoski and Jones 1995). Size is largely ® xed by the nature of
the tasks to be performed. It limits the manner in which the team can be organized
and how it can interact. Empirical evidence suggests that group productivity is not a
linear function of group size, but that a negatively accelerated function generally
exists between the two, that is, larger teams tend to be detrimental to eVectiveness,
typically as a result of heightened coordination needs (Kidd 1961, George et al. 1963,
Steiner 1972, Gladstein 1984, Sundstrom et al. 1990). As a general rule, teams should
be staVed to the smallest number needed to do the work (Hackman 1987, Sundstrom
et al. 1990).
4.1.3. Team composition: Not only size, but aspects of team composition must also
be determined. Speci® cally, team composition may vary along several dimensions:
(1) attributes such as age, gender, race, intelligence, aptitude, training, experience,
personality, etc., (2) distribution of these attributes within the team and (3) the
stability of team composition over time. Among the issues to be raised are: (1) the
eVects of team homogeneity/heterogeneity, compatibility and turnover, (2) the
degree of redundancy that exists between team functions and (3) the extent to which
team output is related to individual performances. With respect to the latter, the
more individual performance determines team output, the less important become the
team processes, and the less necessary it becomes to provide special team training
(Meister 1985).
1060 C. R. Paris et al.
4.1.4. Team stability: Team stability was mentioned above. Because individual skill
is a major determinant of team performance, the eVects of adding untrained
personnel as replacements are likely to be adverse. While adding skilled replacements
may show a positive eVect, that eVect tends to be smaller than the negative impact
resulting from fewer skilled replacements. As a general rule, there is little disruption
of team performance from turnover, as long as only one team member is replaced at
a time and that replacement is as skilled as the person he replaces. Disruption is
increased if additional team members are replaced (Naylor and Briggs 1965, Meister
1985). Teams composed of 40% or more untrained individuals demonstrate
declining performance (Morgan et al. 1978), and the greatest eVect on performance
is likely to come from changes in key or central positions (Ziller 1963, Trow 1964).
eVectiveness. These authors proposed speci® c advice for task design. First,
management should make jobs motivating by encouraging autonomy, wide
participation in team decisions, a variety of task assignments, and interdependence
between team members. Management should create a supportive context for the
team in terms of training, resources, information, and encouragement, and should
monitor and encourage positive team processes. Some of the key design targets that
have been investigated to date are discussed below.
reduce workload and may compromise SA, it may not be desirable to implement full
automation in team tasks, even if that is technically possible. Endsley (1997)
recommends that intermediate levels of automation may be preferable for certain
types of tasks or functions to keep SA at a higher level and allow human operators to
perform critical functions.
Automation also makes possible interactive training, which may now be
provided through distributed simulations linked via local or wide area networks.
Salas and Cannon-Bowers (1997) caution, however, that such simulations need to
link speci® c task requirements to team training needs to create eVective learning
environments (explanation of event-based approaches to training in Dwyer et al.
1997). Training simulators may also be embedded in operational equipment so that
they can provide instructional opportunities when operators are not busy performing
their tasks. Sophisticated systems, capable of capturing multiple forms of
performance data (eye movements, communications, keypress actions), diagnosing
performance, and providing on-line feedback, now exist (e.g. Zachary and Cannon-
Bowers 1997, Zachary et al. 1997a,b).
Besides their training value, computer systems may expedite decision-making in
team settings. Decision-making at the team level multiplies those limitations inherent
in human decision-making at the individual level. Decision support systems (DSS) can
aid the decision-maker by reducing problem complexity and associated memory load,
and by enhancing SA. An example of current work in this area may be found in the
TADMUS (Tactical Decision-Making Under Stress) DSS being designed for Naval
command and control environments. This DSS organizes and presents tactical data in
a form that is consistent with experts’ usage, and guides decision-makers through the
huge amount of tactical data which they need to process (see Morrison et al. 1998, for a
description of this system). In business settings, DSS (called group decision support
systems, or GDSS) support tasks with pooled interdependencies that require
discussion and communication. They are typically utilized in decision rooms, local
decision networks, teleconferencing, and remote decision-making. While most
evidence supports the use of decision support in group settings, i.e. decision quality
is increased, some research suggests that decision aids may isolate operators from each
other’s SA and decision-making processes, thereby reducing redundancy and human
input, and increasing the possibility of errors (Mosier and Shitka 1996). Bowers et al.
(1996) suggest that team decision-making imposes additional requirements on
decision-makers due to the distributed nature of the information, and that making
eVective use of that information requires that one develops techniques to integrate
what is necessary and vital to the decision-making task.
For better or for ill, the requirements for teams to operate with automated
systems will likely increase with time. Clearly, that technology can guarantee neither
greater eVectiveness nor eYciency in the way teams perform. Given that automation
eVects are both compounded and confounded at the team level, it becomes
imperative that engineering eVorts to optimize team performance account for their
potential impact. Taking the time to address this, as well as the impact of other task
design variables, may enable us to ameliorate many foreseeable de® ciencies in the
way teams are likely to conduct their tasks.
4.3.1. What to train: Until recently, team performance has been considered to be a
function of the average skill level of the individuals who comprise the team. While
individual skills are unquestionably necessary to team success, they are not suYcient
(Salas et al. 1992). Process losses, resulting from interruptions in the ¯ ow of
communication or coordination behaviours, are likely to threaten performance if
teamwork skills are not adequately developed (Steiner 1972). One can maximize
training eYciency by combining individual and team skills training into a single
training design and by allocating the appropriate amount of individual skills training
relative to team skills training. Additionally, one can tailor the sequence in which
each type of training is be presented (i.e. train individual skills ® rst), as well as the
time that should be allowed to elapse between individual and team skills training
(Salas et al. 1992). Finally, the skills that are targeted for training should meet
speci® c criteria, namely: (1) they have been empirically demonstrated to have an
eVect on team success, (2) they are diYcult to learn, (3) they require more than
simple repetition for development, (4) they require practice to prevent their loss and
(5) they may be infrequently required, but are essential for survival (Dyer 1984).
Cannon-Bowers et al. (1995) proposed a framework that contributes some useful
insights into selecting the appropriate competencies for training. The competencies
may be speci® c to a particular task or team, or they may be generic in that they apply
across teams or tasks. To ascertain what competencies must be trained, one would
begin by asking certain questions: Are the members of this team the same individuals
over time? Is the membership fairly stable? Does the team perform the same or
similar tasks over time? Are the tasks fairly stable? The answers to these questions
will align the required competencies into one of four categories. Table 4 provides an
explanation of each competency type, as well as the four possible alignments of the
various competency types.
This framework establishes the idea that team membership and task character-
istics prescribe the type of competencies that will lead to its success. Skills targeted
for training will be focused on the ability to work with speci® c or diVerent
individuals, or to work on speci® c or diVerent types of tasks. Once the appropriate
competencies and focus for training have been identi® ed, then it becomes important
to determine how best to train those skills.
4.3.2. What is team training?: In spite of signi® cant progress regarding what to
train, many present-day assumptions regarding how to train have no basis in
research. Team training is more than just team building. Clearly the data imply that
Teamwork in multi-person systems 1065
Competency alignments
Task speci® c/team speci® c: Needed when team membership is stable and the number of tasks
is small. Examples: sports teams and some air crews/combat teams
Task speci® c/team generic: Needed when team members perform a speci® c team task, but do
not work consistently with the same teammates. Examples: ® re® ghting team, air crews, or
medical teams
Task generic/team speci® c: Needed when team membership is stable, but the tasks vary.
Examples: self-managed work teams or quality circles
Task generic/team generic: Needed where team members work on a variety of teams, as well
as on a variety of tasks. Examples: task forces or project teams
it is not enough to train individual responsibilities and simply hope that the team
members will magically ® gure out how to operate as a team (Stout et al. 1994,
McIntyre and Salas 1995, Salas and Cannon-Bowers 2000).
Training strategies should be grounded in theory and sound instructional
principles. Salas and his colleagues have undertaken to formulate and empirically
validate principles, guidelines, and speci® cations for team training (Swezey and Salas
1992a,b, Salas and Cannon-Bowers 1997, 2000). Team training is essentially `a set of
tools and methods that, in combination with required competencies and training
objectives, form an instructional strategy’ (Salas and Cannon-Bowers 2000: 5).
Cognitive, behavioural, and aVective competencies necessary for eVective teamwork
drive the training objectives. Those objectives combine with available resources
(tools and methods) to shape the development of speci® c instructional strategies.
Training tools include, but are not limited to, team task analysis, task simulation and
exercises, and performance measurement and feedback. Methods for delivery may be
information-based , demonstration-based , or practice-based, and may include
lectures, video or multimedia presentations, demonstrations, guided practice, and
role-playing (Salas and Cannon-Bowers 1997, 2000). The goal is to design and
develop from these tools and methods speci® c instructional strategies for in¯ uencing
team processes and outcomes. Representative instructional strategies that have been
1066 C. R. Paris et al.
formulated in recent years are presented in table 5 (note that each of these strategies
has demonstrated its eVectiveness in the areas indicated).
4.3.2.1. Train the part or the whole?: Particularly problematic in the develop-
ment of training strategies is the selection of `part’ versus `whole’ training methods.
In part training, team members learn speci® c components of the task individually
and sequentially, and then gradually integrate those skills until the entire task is
mastered. In whole training, they are exposed to the entire task during all segments
of the training procedure. Instructors must decide whether to (1) provide individual
skills training that includes a focus on the interactive requirements of the team task
(individual-whole training), (2) provide individual skills training that ignores the
team context (individual-part training), (3) provide team skills training with speci® c
focus on each individual’s assigned subtask (team-part training) or (4) provide team
skills training within the context of the team and its coordination and communica-
tion requirements (team-whole training). If high complexity and high organization
characterize the team task, then the best strategy may be to combine the individual-
whole and the team-whole approaches. Unfortunately, no standard exists for
sequencing these two training types or for the establishing the appropriate ratio of
each. It is hoped that future research will shed more light on when individual/team
and part/whole training paradigms are most bene® cial, and will demonstrate how
task characteristics aVect these relationships (Salas et al. 1992).
sessions, but increases its scope to cover many aspects during later sessions. It
increases in speci® city with training, such that it reinforces gross aspects of
performance during early phases of team training, then more speci® c aspects later on
to `® ne tune’ performance. Finally, it is sequenced so that individual feedback is
given in initial training sessions, while feedback regarding overall performance is
provided in later phases of training. As the number of interdependencies among
team members increases, so does the importance of team feedback (Salas et al. 1992).
4.3.2.3. What are the instructional strategies?: Certain teamwork skills have
presented considerable challenge to the development of instructional strategies. One
such skill is the ability to learn on a continuous basis. Since learning is meant to be a
continuous process, teams should be trained in techniques that stimulate continuous
team growth. One of the strategies illustrated earlier Ð `Guided Team Self-
Correction’ Ð has successfully structured this type of training and made signi® cant
progress in meeting this particular challenge (Blickensderfer et al. 1997a,b, Smith-
Jentsch et al. 1998b).
Progress has been slower in other areas, notably in instructional development for
two very complex teamwork skills Ð shared SA and decision-making. If one accepts
the idea that teams are information processing units, a notion advanced by Hinsz et
al. (1997), then it becomes apparent that member interactions can enhance or
degrade a group’s ability to attend to, encode, store, retrieve, feedback, and
ultimately learn from crucial task information. These interactions become the
foundation for training in the areas of SA and decision-making.
SA relies on both individual and team mental models (Cannon-Bowers et al.
1993, Duncan et al. 1996, Stout et al. 1996). Each team member perceives one or
more parts of the complex environment, and these perceptions must be integrated
with those of fellow team-mates. The objective is for the team to share a common
picture of the environment. To accomplish this, team members must voice
communications that promote collective awareness of the surrounding environment,
both internal and external to the team, and make timely and accurate reports of
deviations from the norm or potential problems. Team leaders can continually
update team members during times of stress to keep them abreast of rapidly
changing priorities and performance objectives. Training that supports these
abilities, as well as the development of shared perceptions of critical task and
interpersonal dimensions will improve a team’s SA ability.
Decision-makers must ® rst recognize the problem and the need for action.
Depending on their task environment, teams rely on controlled, analytic decision-
making (e.g. problem-solving teams in an industrial environment) or on more
automatic perceptual-cognitiv e processes (e.g. tactical decision-making teams, law
enforcement or ® re-® ghting teams, emergency medical teams). The latter form of
decision-making, called `naturalistic decision-making’ (Zsambok 1997) or `recogni-
tion-primed decision-making’ (Klein 1989, 1993), is prevalent in environments where
performance is both resource and data-limited. In those environments, decision-
makers must maintain large amounts of information in memory under conditions of
high workload and stress, and their decisions may be skewed by the lack of complete,
error-free, unambiguous data. Because the situation itself either determines or
constrains the response options, decision-makers in these environments typically
make up to 95% of all decisions without considering alternatives (Klein 1989,
Kaempf et al. 1996). If the situation appears similar to one that the decision-maker
1068 C. R. Paris et al.
has previously experienced, the pattern will be recognized and the course of action is
usually immediately obvious. Due to notable team failures in these types of
environments (e.g. the 1988 Vincennes incident, in which the decision was made to
launch a defensive missile against an Iranian airbus carrying 290 passengers, whose
radar signal only moments before was misclassi® ed as that of a hostile attacking
® ghter aircraft), eVorts are currently underway to improve decision-making in these
types of environments.
Because this type of decision-making relies heavily on the ability of decision-
makers to recognize important features/patterns within contexts, naturalistic
approaches would promote the viability of training perceptual processes (Orasanu
1995). In fact, this approach parallels what researchers have discovered concerning
the development of expertise. In short, novices tend to use more controlled, analytic
approaches to reach their decisions. As they accumulate experience, they begin to
switch between controlled and more automatic, perceptual-cognitiv e processes. By
the time they become experts, they almost exclusively utilize the automatic
perceptual processes (Klein and HoVman 1993). Training for team members who
must perform in naturalistic settings might focus on key novice/expert diVerences
that have been identi® ed, namely: (1) recognizing patterns, (2) making ® ne
perceptual discriminations, (3) recognizing typicality and detecting anomalies, (4)
mentally simulating future states and past states, (5) improvising and (6) adapting to
events (Shanteau 1988, Klein and HoVman 1993, Means et al. 1993).
An earlier referenced training strategy, `Critical Thinking Training’, represents a
less common form of naturalistic decision-making called `explanation-base d
reasoning’ or story-generatio n (Cohen et al. 1998). In story-generation , the
decision-maker attempts to evaluate the discrepancies between expectations and
what actually happens by creating a reasonable hypothesis or story to explain it. As
with recognition-primed decision-making, however, this type of reasoning is not
exhaustive in the analytic tradition, but is fairly short and concise. It is usually
implemented when pattern recognition fails or when time permits.
These two processes Ð feature-matching and story-generatio n Ð have been
demonstrated to account for the majority of situation assessments and subsequent
decisions made by decision-makers in environments which preclude the use of more
controlled, analytic types of decision-making. In fact, Kaempf et al. (1993)
demonstrated that in a sample of 183 decisions made by the Navy’s command-
level decision-makers in the Combat Information Centre of an Aegis cruiser, 87% of
the situation assessments were derived through feature matching and the remaining
13% were derived through story generation.
6. Concluding remarks
The progression of this concept has been traced from its inception 50 years ago to
current thinking. It has been learnt that teamwork is the seamless integration of
speci® c cognitive, behavioural and aVective competencies that allow team members
to adapt and optimize their performance. Researchers have made great strides in
de® ning teams, in ascertaining how they mature and develop, and in diVerentiating
their performance from that of individuals and groups. They have identi® ed
competencies that enable teams to meet their goals, and have developed eVective
techniques for capturing, measuring and teaching those skills. In terms of training, a
considerable repertoire has been acquired from which to draw principles, guidelines
and speci® cations to maximize success. Finally, researchers have considered ways to
1070 C. R. Paris et al.
select better teams, to design better team tasks and to adapt automation to team
settings.
Acknowledgements
The views expressed herein are those of the authors and do not re¯ ect the oYcial
position of the organization to which they are aYliated.
References
ADAMS , M. J., TENNEY, Y. J. and PEW , R. W. 1995, Situation awareness and the cognitive
management of complex systems, Human Factors, 37, 85 ± 104.
ADELMAN, L. ZIRK, D. A., LEHNER, P. E., MOFFETT, R. J. and HALL, R. 1986, Distributed
tactical decision-making: Conceptual framework and empirical results, IEEE Transac-
tions on Systems, Man, and Cybernetics, SMC-16, 794 ± 805.
ANNETT, J. 1997, Analysing team skills, in R. Flin, E. Salas, M. Strub and L. Martin (eds),
Decision-making Under Stress: Emerging Themes and Applications (Aldershot: Ashgate),
315 ± 325.
BAKER , D. P. and SALAS, E. 1992, Principles for measuring teamwork skills, Human Factors, 34,
469 ± 475.
BANDURA , A. 1986, Social Foundations of Thought and Action (Englewood CliVs: Prentice-
Hall).
BASS, B. M. 1980, Individual capability, team performance, and team productivity, in E. A.
Fleischman and M. D. Dunnette (eds), Human Performance and Productivity (Hillsdale:
Lawrence Erlbaum), 179 ± 232.
BLICKENSDERFER , E. L., CANNON -BOWERS , J. A. and SALAS , E. 1997a, Fostering shared mental
models through team self-correction: theoretical bases and propositions, in M.
Beyerlein, D. Johnson and S. Beyerlein (eds), Advances in Interdisciplinary Studies in
Work Team, vol. 4 (Greenwich: JAI Press).
BLICKENSDERFER , E. L., CANNON -BOWERS, J. A. and SALAS, E. 1997b, Training teams to self-
correct: an empirical investigation. Paper presented at the 12th Annual Conference of
the Society for Industrial and Organizational Psychology, St Louis, MO.
BLICKENSDERFER , E. L., CANNON -BOWERS , J. A. and SALAS , E. 1998, Cross-training and team
performance, in J. A. Cannon-Bowers and E. Salas (eds), Making Decisions Under
Stress: Implications for Individual and Team Training (Washington, DC: American
Psychological Association Press), 299 ± 311.
BOWERS, C. A., DEATON , J., OSER, R., PRINCE, C. and KOLB , M. 1993, The impact of automation
on crew communication and performance, in Proceedings of the Seventh International
Symposium on Aviation Psychology (Columbus: Ohio State University), 758 ± 761.
BOWERS, C. A., OSER, R. L., SALAS , E. and CANNON-BOWERS , J. A. 1996, Team performance in
automated systems, in R. Parasuraman and M. Mouloua (eds), Automation and Human
Performance: Theory and Applications (Mahwah: Lawrence Erlbaum), 243 ± 263.
BOWERS, C. A., URBAN, J. M. and MORGAN , B. B. JR 1992, The Study of Crew Coordination and
Performance in Hierarchical Team Decision Making. Report no. TR-92-01 (Orlando:
University of Central Florida, Team Performance Laboratory).
CAMPION, M. A., MEDSKER, G. J. and HIGGS, A. C. 1993, Relations between work group
characteristics and eVectiveness: implications for designing eVective work groups,
Personnel Psychology, 46, 823 ± 850.
CAMPION, M. A., PAPPER, E. M. and MEDSKER , G. J. 1996, Relations between work team
characteristics and eVectiveness: a replication and extension, Personnel Psychology, 49,
429 ± 452.
CANNON -BOWERS, J. A. and SALAS, E. (eds) 1998, Making Decisions Under Stress: Implications for
Individual and Team Training (Washington, DC: American Psychological Association).
CANNON -BOWERS, J. A., SALAS , E. and CONVERSE, S. A. 1993, Shared mental models in expert
team decision-making, in N. J. Castellan, Jr (ed.), Individual and Group Decision Making
(Hillsdale: Lawrence Erlbaum), 221 ± 246.
CANNON -BOWERS, J. A., SALAS, E. and GROSSMAN , J. D. 1991, Improving tactical decision-
making under stress: research directions and applied implications. Paper presented at
the International Applied Military Psychology Symposium, Stockholm.
Teamwork in multi-person systems 1071
CANNON -BOWERS , J. A., TANNENBAUM , S. I., SALAS , E. and VOLPE, C. E. 1995, De® ning team
competencies: Implications for training requirements and strategies, in R. Guzzo and E.
Salas (eds), Team EVectiveness and Decision Making in Organizations (San Francisco:
Jossey Bass).
COHEN, M. S., FREEMAN, J. T. and THOMPSON , B. 1998, Critical thinking skills in tactical
decision-making: a model and a training strategy, in J. A. Cannon-Bowers and E. Salas
(eds), Making Decisions Under Stress: Implications for Individual and Team Training
(Washington, DC: American Psychological Association Press), 155 ± 189.
DICKINSON, T. L. 1969, The eVects of work interaction and its interplay with task organization
on team and member performance. Doctoral dissertation, Ohio State University,
Dissertation Abstracts International, 30, 1937 ± 1938.
DIETERLY, D. L. 1988, Team performance requirements, in S. Gael (ed.), The Job Analysis
Handbook for Business, Industry, and Government, vol. 1 (New York: Wiley), 486 ± 492.
DRISKELL, J. E. and JOHNSTON, J. H. 1998, Stress exposure training, in J. A. Cannon-Bowers
and E. Salas (eds), Making Decisions Under Stress: Implications for Individual and Team
Training (Washington, DC: American Psychological Association Press), 191 ± 217.
DRISKELL, J. E. and SALAS , E. 1992, Collective behaviour and team performance, Human
Factors, 34, 277 ± 288.
DUNCAN, P. C., ROUSE, W. B., JOHNSTON, J. H., CANNON -BOWERS, J. A., SALAS , E. and BURNS, J.
J. 1996, Training teams working in complex systems: a mental model-based approach,
in W. B. Rouse (ed.), Human/Technology Interaction in Complex Systems, vol. 8
(Greenwich: JAI Press), 173 ± 231.
DWYER, D. J., FOWLKES, J. E., OSER, R. L., SALAS, E. and LANE , N. E. 1997, Team performance
measurement in distributed environments: the TARGETs methodology, in M. T.
Brannick, E. Salas and C. Prince (eds), Assessment and Measurement of Team
Performance: Theory, Research, and Application (Hillsdale: Lawrence Erlbaum), 137 ±
154.
DYER , D. J. 1984, Team research and team training: a state-of-the-art review, in F. A. Muckler
(ed.), Human Factors Review: 1984 (Santa Monica: Human Factors Society), 285 ± 323.
ENDSLEY , M. R. 1997, Level of automation: integrating humans and automated systems, in
Proceedings of the Human Factors and Ergonomic s Society 41st Annual Meeting (Santa
Monica: Human Factors Society).
ENTIN, E., SERFATY, D. and DECKERT, J. C. 1994, Team Adaptation and Coordination Training.
Technical report, NAWCTSD Contract N61339-91-C-0142 .
FLEISHMAN, E. A. and ZACCARO, S. J. 1992, Toward a taxonomy of team performance
functions, in R. W. Swezey and E. Salas (eds), Teams: Their Training and Performance
(Norwood: Ablex), 31 ± 56.
GENTNER, D. and STEVENS, A. L. 1983, Mental Models (Hillsdale: Lawrence Erlbaum).
GEORGE, C. E., HOAK , G. and BOUTWELL , J. 1963, Pilot Studies of Team EVectiveness. Research
memorandum no. 28, AD-627, 217 (US Army Infantry Human Research Unit, Human
Resources Research OYce, Ft Benning, GA).
GERSICK, C. J. G. 1988, Time and transition in work teams: towards a new model of group
development, Academy of Management Review, 31, 941.
GLADSTEIN, D. L. 1984, Groups in context: A model of task group eVectiveness, Administrative
Science Quarterly, 29, 499 ± 517.
GOODMAN , P. S., RAVLIN, E. and SCHMINKE, M. 1987, Understanding groups in organizations,
in L. L. Cummings and B. M. Staw (eds), Research in Organizational Behaviour, vol. 9
(Greenwich: JAI Press), 121 ± 173.
GREGORICH, S. E., HELMREICH, R. L. and WILHELM, J. A. 1990, The structure of cockpit
management attitudes, Journal of Applied Psychology, 75, 682 ± 690.
GUZZO, R. A. and SALAS, E. (eds) 1995, Team EVectiveness and Decision Making in
Organizations (San Francisco: Jossey Bass).
GUZZO, R. A., YOST, P. R., CAMPBELL, R. J. and SHEA, J. P. 1994, Potency in groups:
articulating the construct, British Journal of Social Psychology, 32, 87 ± 106.
HACKMAN, J. R. 1983, A Normative Model of Work Team EVectiveness. Technical report no. 2
(New Haven: Yale University).
HACKMAN, J. R. 1987, The design of work teams, in J. W. Lorsch (ed.), Handbook of
Organizational Behaviour (Englewood CliVs: Prentice-Hall), 315 ± 342.
1072 C. R. Paris et al.
HACKMAN , J. R. (ed.) 1990, Groups That Work (And Those That Do not): Creating Conditions
for EVective Teamwork (San Francisco: Jossey-Bass).
HACKMAN , J. R. and MORRIS , C. G. 1975, Group tasks, group interaction process, and group
performance eVectiveness: a review and proposed integration, in L. Berkowitz (ed.),
Advances in Experimental Social Psychology, vol. 8 (New York: Academic Press), 45 ±
99.
HINSZ, V. B., TINDALE , R. S. and VOLLRATH , D. A. 1997, The emerging conceptualisation of
groups as information processors, Psychological Bulletin, 121, 43 ± 64.
ILGEN , D. R. 1999, Teams embedded in organizations: some implications, American
Psychologist, 54, 129 ± 139.
JENTSCH, F., BOWERS, C., REDSHAW , B., BERGEN, H., HENNING, J. and HOLMES, B. 1995,
DiVerential EVects of Automation on Team Performance, Workload, and Communica-
tions. Technical report (Orlando: Naval Air Warfare Centre Training Systems Division,
University of Central Florida).
JOHNSTON , J. H., SMITH-JENTSCH, K. A. and CANNON -BOWERS, J. A. 1997, Performance
measurement tools for enhancing team decision-making, in M. T. Brannick, E. Salas
and C. Prince (eds), Team Performance Assessment and Measurement: Theory, Methods,
and Applications (Hillsdale: Lawrence Erlbaum).
KAEMP F, G. L., KLEIN, G., THORDSEN, M. L. and WOLF, S. 1996, Decision-making in complex
Naval command-and-control environments, Human Factors, 38, 220 ± 231.
KAEMP F, G. L., WOLF, S. and MILLER, T. E. 1993, Decision-making in the Aegis combat
information center, in Proceedings of the Human Factors and Ergonomics Society 37th
Annual Meeting (Santa Monica, CA), 1107 ± 1111.
KIDD, J. S. 1961, A comparison of one-, two-, and three-man work units under various
conditions of workload, Journal of Applied Psychology, 45, 195 ± 200.
KLEIN, G. A. 1989, Recognition-primed decisions, in W. Rouse (ed.), Advances in Man ±
Machine Systems Research, vol. 5 (Greenwich: JAI Press), 47 ± 92.
KLEIN, G. A. 1993, A recognition primed decision (RPD) model of rapid decision-making, in
G. A. Klein, J. Orasanu, R. Calderwood and C. E. Zsambok (eds), Decision Making in
Action: Models and Methods (Norwood: Ablex), 138 ± 147.
KLEIN, G. A. and HOFFMAN, R. 1993, Seeing the invisible: perceptual ± cognitive aspects of
expertise, in M. Rabinowitz (ed.), Cognitive Science Foundations of Instruction
(Hillsdale: Lawrence Erlbaum) 203 ± 226.
KLEINMAN, D. L. and SERFATY, D. 1989, Team performance assessment in distributed decision-
making, in Proceedings of the Symposium on Interactive Networked Simulation for
Training (Orlando), 22 ± 27.
KLIMOSKI, R. and JONES , R. G. 1995, StaYng for eVective group decision-making: key issues in
matching people and teams, in R. A. Guzzo, E. Salas and Associates (eds), Team
EVectiveness and Decision Making in Organizations (San Francisco: Jossey-Bass), 291 ±
332.
KOLODNY , H. F. and KIGGUNDU , M. N. 1980, Towards the development of a sociotechnical
systems model in woodlands mechanical harvesting, Human Relations, 33, 623 ± 645.
MCINTYRE, R. M. and SALAS , E. 1995, Measuring and managing for team performance: emerging
principles from complex environments, in R. Guzzo and E. Salas (eds), Team EVectiveness
and Decision Making in Organizations (San Francisco: Jossey-Bass), 149 ± 203.
MEANS , B., SALAS, E., CRANDALL , B. and JACOBS, O. 1993, Training decision-makers for the real
world, in G. A. Klein, J. Orasanu, R. Calderwood and C. E. Zsambok (eds), Decision
Making in Action: Models and Methods (Norwood: Ablex).
MEISTER, D. 1985, Behavioural Foundations of System Development. 2nd edn (Malabar: Robert
E. Krieger).
MODRICK, J. A. 1986, Team performance and training, in J. Zeidner (ed.), Human Productivity
Enhancement: Training and Human Factors in Systems Design, vol. 1 (New York:
Praeger), 130 ± 166.
MORGAN , B. B. JR, COATES , G. D., ALLUIZI, E. A. and KIRBY , R. H. 1978, The Team-Training
Load as a Parameter of EVectiveness for Collective Training in Units. ITR-78-14, DTIC
no. AD A063 135 (Norfolk: Old Dominion University).
Teamwork in multi-person systems 1073
MORGAN , B. B. JR, GLICKMAN, A. S., WOODWARD , E. A., BLAIWES, A. S. and SALAS, E. 1986,
Measurement of Team Behaviours in a Navy Environment. NTSC technical report TR-
86-014 (Orlando: Naval Training Systems Centre).
MORGAN , B. B. JR, HERSCHLER, D. A., WIENER, E. L. and SALAS , E. 1993, Implications of
automation technology for aircrew coordination performance, in W. B. Rouse (ed.),
Human/Technology Interaction in Complex Systems, vol. 6 (Greenwich: JAI Press), 105 ±
136.
MORGAN , B. B. JR, SALAS , E. and GLICKMAN, A. S. 1994, An analysis of team evolution and
maturation. Journal of General Psychology, 120, 277 ± 291.
MORGESON , F. P., AIMAN-SMITH, L. D. and CAMPION, M. A. 1997, Implementing work teams:
Recommendations from organizational behaviour and development theories, in M. M.
Beyerlein, D. A. Johnson and S. T. Beyerlein (eds), Advances in Interdisciplinary Studies
of Work Teams, 4, 1 ± 44.
MORRISON , J. G., KELLY, R. T., MOORE , R. A. and HUTCHINS, S. G. 1998, Implications of
decision-making research for decision support and displays, in J. A. Cannon-Bowers
and E. Salas (eds), Making Decisions Under Stress: Implications for Individual and Team
Training (Washington, DC: American Psychological Association Press), 375 ± 406.
MOSIER, K. L. and SKITKA, L. J. 1996, Human decision-making and automated decision aids:
made for each other?, in R. Parasuraman and M. Mouloua (eds), Automation and
Human Performance: Theory and Applications (Mahwah: Lawrence Erlbaum), 201 ± 220.
MULLEN, B. and COPPER, C. 1994, The relation between group cohesiveness and performance:
an integration, Psychological Bulletin, 115, 210 ± 217.
NAYLOR , J. C. and BRIGGS, G. E. 1965, Team-training eVectiveness under various conditions,
Journal of Applied Psychology, 49, 223 ± 229.
NAYLOR , J. C. and DICKINSON, T. L. 1969, Task structure, work structure, and team
performance, Journal of Applied Psychology, 53, 167 ± 177.
NIVEA, F., FLEISHMAN, E. A. and REICK, A. 1978, Team Dimensions: Their Identity, Their
Measurement, and Relationships. Final technical report, Contract DAH19-78-C-0001
(Washington, DC: Advance Resources Research Organization).
ORASANU , J. 1990, Shared mental models and crew performance. Paper presented at the 34th
Annual Meeting of the Human Factors Society, Orlando.
ORASANU , J. 1995, Training for aviation decision-making: the naturalistic decision-making
perspective, in Proceedings of the Human Factors and Ergonomics Society 39th Annual
Meeting (Santa Monica, CA).
ORASANU , J. and SALAS , E. 1993, Team decision-making in complex environments, in G. A.
Klein, J. Orasanu, R. Calderwood and C. Zsambok (eds), Decision Making in Action:
Models and Methods (Norwood: Ablex), 327 ± 345.
OSER, R. L., MCCALLUM, G. A., SALAS, E. and MORGAN , B. JR 1989, Toward a De® nition of
Teamwork: An Analysis of Critical Team Behaviour. NTSC technical report no. 90-009
(Orlando: Naval Training Systems Centre).
PEARCE , J. A. and RAVLIN, E. C. 1987, The design and activation of self-regulating work
groups, Human Relations, 40, 751 ± 782.
PRINCE, C., CHIDESTER, T. R., BOWERS , C. and CANNON -BOWERS, J. 1992, Aircrew coordination:
achieving teamwork in the cockpit, in R. W. Swezey and E. Salas (eds), Teams: Their
Training and Performance (Norwood: Ablex), 329 ± 353.
PRINCE, C. and SALAS , E. 1993, Training and research for teamwork in the military aircrew, in
E. L. Wiener, B. G. Kanki and R. L. Helmreich (eds), Cockpit Resource Management
(Orlando: Academic Press), 337 ± 366.
ROBY, T. B. and LANZETTA, J. T. 1957a, A Replication Study of Work Group Structure and Task
Performance. DTIC no. AD 134 205 (Air Force Personnel and Training Research
Centre, Lackland Air Force Base, TX).
ROBY, T. B. and LANZETTA, J. T. 1957b, Con¯ icting principles in man ± machine system design,
Journal of Applied Psychology, 41, 170 ± 178.
ROUSE , W. B. and MORRIS , N. M. 1986, On looking into the black box: prospects and limits in
the search for mental models, Psychological Bulletin, 100, 349 ± 363.
RUFFELL-SMITH, H. P. 1979, A Simulator Study of the Interaction of Pilot Workload with Errors.
NASA technical report no. TM-78482 (MoVett Field: National Aeronautics and Space
Administration, Ames Research Centre).
1074 C. R. Paris et al.
SWEZEY, R. W., OWENS, J. M. BERGONDY , M. L. and SALAS , E. 1998, Task and training
requirements analysis methodology (TTRAM): an analytic methodology for identifying
potential training uses of simulator networks in teamwork-intensive task environments,
Ergonomics, 41, 1678 ± 1697.
SWEZEY, R. W. and SALAS , E. 1992a, Guidelines for use in team-training development, in R. W.
Swezey and E. Salas (eds), Teams: Their Training and Performance (Norwood: Ablex),
219 ± 245.
SWEZEY, R. W. and SALAS , E. (eds) 1992b, Teams: Their Training and Performance (Norwood:
Ablex).
TANNENBAUM , S. I., BEARD , R. L. and SALAS , E. 1992, Team building and its in¯ uence on team
eVectiveness: an examination of conceptual and empirical developments, in K. Kelley
(ed.), Issues, Theory, and Research in Industrial/Organization Psychology (Amsterdam:
Elsevier), 117 ± 153.
TANNENBAUM , S. I., SMITH-JENTSCH, K. A. and BEHSON , S. J. 1998, Training team leaders to
facilitate team learning and performance, in J. A. Cannon-Bowers and E. Salas (eds),
Making Decisions Under Stress: Implications for Individual and Team Training
(Washington, DC: American Psychological Association Press), 247 ± 270.
THORNTON , C. BRAUN , C., BOWERS, C. A. and MORGAN , B. JR 1992, Automation eVects in the
cockpit: a low-® delity investigation, in Proceedings of the 36th Annual Meeting of the
Human Factors Society (Santa Monica, CA), 30 ± 34.
TROW , D. B. 1964, Teamwork Under Turnover and Succession. Technical report no. 2, OYce of
Naval Research contract no. 3679 (00), project NR 170-331, DTIC no. AD 601 816
(Endicott: Harpur College).
URBAN , J. M., BOWERS, C. A., CANNON -BOWERS , J. A. and SALAS , E. 1995, The importance of
team architecture in understanding team processes, in M. Beyerlein (ed.), Advances in
Interdisciplinary Studies in Work Teams, vol. 2 (Greenwich: JAI Press), 205 ± 228.
VOLP E, C. E., CANNON -BOWERS , J. A., SALAS , E. and SPECTOR, P. 1996, The impact of cross
training on team functioning, Human Factors, 38, 87 ± 100.
WEST, M. A. 1996, Handbook of Work Group Psychology (New York: Wiley).
WIENER, E. L. 1993, Crew coordination and training in the advanced-technology cockpit, in E.
L. Wiener, B. G. Kanki and R. L. Helmreich (eds), Cockpit Resource Management (San
Diego: Academic Press), 199 ± 229.
WIENER, E. L., CHIDESTER, T. R., KANKI , B. G., PALMER, E. A., CURRY , R. E. and GREGORICH, S.
E. 1991, The Impact of Cockpit Automation on Crew Coordination and Communication :
I. Overview, LOFT Evaluations, Error Severity, and Questionnaire Data. Technical
report no. NASA-CR-177587 (MoVett Field: NASA Ames Research Centre).
WIENER, E. L., KANKI, B. J. and HELMREICH, R. L. (eds) 1993, Cockpit Resource Management
(San Diego: Academic).
ZACHARY , W., BILAZARIAN, P., BURNS, J. and CANNON -BOWERS , J. A. 1997a, Advanced embedded
training concepts for shipboard systems (CD-ROM), in Proceedings of the 19th Annual
Interservice/Industry Training, Simulation, and Education Conference (Orlando: National
Training Systems Association), 670 ± 679.
ZACHARY , W. and CANNON -BOWERS , J. A. 1997, Guided practice Ð a new vision for intelligent
embedded training, in Proceedings of the Human Factors and Ergonomics Society 41st
Annual Meeting (Santa Monica, CA).
ZACHARY , W., RYDER, J., HICINBOTHOM, J. and BRACKEN, K. 1997b, The use of executable
cognitive models in simulation-based intelligent embedded training, in Proceedings of
the Human Factors and Ergonomics Society 41st Annual Meeting (Santa Monica, CA).
ZILLER, R. C. 1963, The EVects of Changes in Group Composition on Group Performance. Final
report, grant no. AFOSR 62-95, DTIC no. AD 413 965 (Newark: University of
Delaware).
ZSAMBOK, C. E. 1997, Naturalistic decision-making: where are we now?, in C. E. Zsambok and
G. Klein (eds), Naturalistic Decision Making (Mahwah: Lawrence Erlbaum).