Research in Engineering Design (2023) 34:221–256

https://doi.org/10.1007/s00163-022-00406-y

ORIGINAL PAPER

Research methods in engineering design: a synthesis of recent studies

using a systematic literature review
David Escudero‑Mancebo1,2 · Nieves Fernández‑Villalobos1,3 · Óscar Martín‑Llorente1 ·
Alejandra Martínez‑Monés2

Received: 8 November 2021 / Revised: 26 October 2022 / Accepted: 23 December 2022 / Published online: 16 January 2023
© The Author(s) 2023

Abstract
The relation between scientific research and engineering design is fraught with controversy. While the number of academic
PhD programs on design grows, because the discipline is in its infancy, there is no consolidated method for systematically
approaching the generation of knowledge in this domain. This paper reviews recently published papers from four top-ranked
journals in engineering design to analyse the research methods that are frequently used. The research questions consider
the aim and contributions of the papers, as well as which experimental design and which sources of data are being used.
Frequency tables show the high variety of approaches and aims of the papers, combining both qualitative and quantitative
empirical approaches and analytical methods. Most of the papers focus on methodological concerns or on delving into a
particular aspect of the design process. Data collection methods are also diverse without a clear relation between the type of
method and the objective or strategy of the research. This paper aims to act as a valuable resource for academics, providing
definitions related to research methods and referencing examples, and for researchers, shedding light on some of the trends
and challenges for current research in the domain of engineering design.

Keywords Research methodologies in engineering design · Engineering design and evaluation

1 Introduction potentially different, and evolving in parallel with the devel-

opment of the domain. In young domains such as Engineer-
Doctoral studies have a long tradition in higher education ing Design, the discussion about which research procedures
systems (Bogle 2018). Doctoral studies are highly relevant and paradigms should be employed is still open.
because they are considered as a key for technical devel- Simon (1996), in his book The Science of Design, defined
opment and industrial excellence in developed countries. design as a search for an optimum in a space of alternatives
Normally, a PhD diploma is compulsory for pursuing and that take into account the specifications and restrictions of
it is highly valued for getting involved in research projects a given problem. Hatchuel (2001) highlighted limitations
in companies. The goal of doctoral programs is to provide of Simon’s position discussing that designing cannot be
postgraduates with competences for the generation of knowl- reduced to taking decisions among a bounded set because
edge in a given domain. The means to generate knowledge the number of concepts related to the problem and the possi-
depends on the area, being research methods and techniques ble number of decisions to be taken could be expandable and
uncountable, not only due to human creativity but also to
social interaction. (Subrahmanian et al. 2020) place Simon
* David Escudero‑Mancebo
descuder@infor.uva.es and Hatchuel’s approaches in a historical timeline that
describes different models of how designing is understood,
1
Escuela de Ingenierías Industriales, Universidad de evidencing the challenges for research design as a discipline
Valladolid, Paseo del Cauce Paseo del Cauce 59, that defines a common language that includes the impact of
47011 Valladolid, Spain
context and users in designing, in addition to the problems..
2
Escuela de Ingeniería Informática, Universidad de Valladolid, Probably due to the youth of design as a research discipline,
Paseo de Belén 15, 47011 Valladolid, Spain
or due to its socio-technical nature, it does not yet have a
3
Escuela Técnica Superior de Arquitectura, Universidad de consolidated research methods and techniques. Blessing and
Valladolid, Avenida Salamanca 18, 47014 Valladolid, Spain

13
Vol.:(0123456789)

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

222 Research in Engineering Design (2023) 34:221–256

Chakrabarti (2009) proposed the DRM (Design Research probably becoming a universal standard for rating the qual-
Methodology) motivated by “frustration about the lack of ity of research. Therefore, being aware of the kind of works
a common terminology, benchmarked research method and published in scientific journals related to engineering design
a common research methodology in design”. Through the could be of outstanding importance for scholars who have
analysis of recent research papers, this work has the aim to configure the contents of the courses related to research
to confirm how these visions about research in engineering methodologies in this field, as well as for PhD supervisors
design are projected in current state-of-the-art publications. and students to focalize efforts for being more productive in
Since the work of Blessing and Chakrabarti, there have terms of publications. The analysis of scientific papers about
been some relevant proposals that have shed light on differ- research in engineering design performed presented in this
ent aspects of the global design research landscape. Koski- paper aims to contribute to this aim.
nen et al. (2011) proposed the term ‘constructive design There are many possible ways to analyse, categorise or
research’ and presented alternatives to integrate research classify research works because there are many dimensions
within the practice of design. Joost et al. (2016) used the of analysis. Creswell (2009) presents a classical distinction
term ‘design as research’ in a volume that compiled dis- between (1) quantitative, (2) qualitative and (3) mixed-
courses of experts about questions on design research and methods (combining qualitative and quantitative research
its relationship with other disciplines. Vaughan (2017) pre- methods). For quantitative methods experimental designs,
sented a survey that collected different points of view related non-experimental design are distinguished. For qualitative,
to doctoral education in the opinions of design graduates narrative research, ethnographic research, phenomenologi-
about practice-based research design. Redström (2017) pre- cal research, grounded theory and case study research are
sented an essay about how to develop theory -knowledge- by distinguished. For mixed-methods, sequential, concur-
practice, experimentation and making -design. These works rent and transformative methods are distinguished. Bless-
are a multi-faceted compendium of practical experiences and ing and Chakrabarti (2009) identified the following ones:
visions of experts on how to perform activities related to (1) paradigm, that includes empiricism (Randolph 2003;
research in the domain of design. Although many examples Solomon 2007) and ethno-methodology (Atkinson 1988),
and discussions presented in the cited books focus on the methodologies, theories, views and assumptions (Kothari
topic of research through/by design, rather than on research 2004); (2) aim, research questions and hypotheses; (3)
in engineering design, all of them agree on the relevance nature of the study, including observational vs interventional
of research into the design due to the increasing number of (Thiese 2014), comparative vs non-comparative; (4) units
PhD programs that could benefit from background knowl- of analysis; (5) data collection methods including record-
edge about this topic. In this paper, we present an alternative ings, interview, questionnaires (De Leeuw 2008); (6) role
approach to shed light on the relations between research and of the researcher (Fink 2000); (7) time constraints, duration
design: instead of collecting the personal visions of experts, and continuation of the research process; (8) observed pro-
we summarise and classify research papers on research in cesses including layout drawing, prototype or product; (9)
Engineering Design in terms of aims and contributions, setting referring to laboratory or field research (Paluck and
methods and approaches, data collection techniques, and Cialdini 2014); (10) tasks including type and complexity
research instruments used for the collection of data. To this and nature; (11) number of cases, case size and participants
end, we have carried out a systematic review of the literature (Diggle et al. 2011); (12) object of analysis distinguishing
on research in engineering design. The overarching research objects, companies, projects, documents… (13) coding and
question (RQ) that drives the review is: What is the current analysis, analysis and (14) verification methods (Brewer and
landscape of research methods in engineering design? Crano 2014); or (15) findings, that is, statement models or
Access to doctoral studies normally requires candi- conclusions resulting from the study. Reich and Subrahma-
dates to have a Master’s degree in which they have taken nian (2021) use the PSI framework (Problem, Social and
courses about research methodologies. Doctoral stud- Institutional space) to analyse and categorise research design
ies normally culminate with the defense of a PhD thesis works focussing on dimensions related to the problem being
in which postgraduates have to show their capabilities to addressed concerning (1) disciplinary, (2) structural com-
generate knowledge in a specific field. Submitting a PhD plexity and (3) knowledge availability; dimensions related
thesis that includes activities previously reviewed in sci- with who is included in designing concerning (4) the per-
entific journals is generally considered as a quality war- spective required to formulate the problem, (5) the inclu-
ranty of the research performed by the student. Although sion of participants in the design process and the (6) capa-
publishing journal papers is not the only way to assess the bilities of the design team; and finally dimensions related
excellence of the research work performed in a PhD thesis, with how designing is executed taking into account (7) the
the quasi-exponential increase of scientific publications we ties or connections between actors, (8) the accessibility to
are witnessing (Tenopir and King 2014) indicates that it is knowledge and (9) the institutional complexity (Reich and

13

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Research in Engineering Design (2023) 34:221–256 223

Subrahmanian 2020). The dimensions presented by Bless- As we aimed to high-impact journals reporting research in
ing and Chakrabari have the ambition to classify different engineering design, we focused on the journals indexed in
aspects to be taken into account when research in engineer- SCIE (Science Citation Index Expanded) related to Science
ing design works are tackled. The dimensions proposed by and Technology, discarding the 22 journals indexed in ESCI
Reich and Subrahmian are complementary and arise when (Emerging Sources Citation Index), the 10 indexed in AHCI
they analyse the factors influencing success in engineering (Arts and Humanities Citation Index) and the 5 indexed
design projects. When analysing papers, some of the details in SSCI (Social Sciences Citation Index). Among the 43
related to some of the listed dimensions could be missing in remaining journals indexed in SCIE, 13 of them correspond
the descriptions (timing, success validation etc.) so that we to categories related to Chemistry and Biology (for example
had to devise alternative proposals. Anti-Cancer Drug Design or Molecular System Design &
Our analysis pivots around the division between empiri- Engineering) 11 of them to Computer Science or Electrics
cal qualitative, quantitative research and mixed-methods (for example Design Codes and Cryptography or Computer
proposed by (Creswell 2009). This classification was com- Aid Design); 3 with Mathematics (for example Journal of
plemented with analytical research methods, as specified by Combinatorial Design) and 2 with Building (Architectural
(Adrion 1993), cited by (Glass 1995) (defined in Sect. 2.2). Engineering and Design Management or Structural Design
From this germinal division, data-collection methods, strate- of Tall and Special Building). Closer to engineering design
gies, and contributions of the studies are reported in cross- are the 14 remaining journals: 4 indexed in Mechanics
analysis tables. We aim to identify the main goals and results Journal of Mechanical Design, Mechanics Based Design of
pursued or obtained by researchers (dimensions 2 and 15 of Structures and Machines, Journal of Advanced Mechanical
Blessing and Chakrabarti 2009), the strategies of enquiry Design Systems for Manufacturing and Journal of Strain
and methodologies they follow (dimensions 1, 3, 9, 10 of Analysis for Engineering Design), 4 related to Materi-
Blessing and Chakrabarti 2009), and which data sources and als (Materials & Design, Proceedings of the Institution of
instruments are most (and least) commonly used (rest of Mechanical Engineers, International Journal of Mechan-
dimensions of Blessing and Chakrabarti 2009) in the domain ics and Materials in Design and Road Materials and Pave-
of engineering design. ment Design); and 2 related with vehicle design (Journal
The structure of the document is the following: First, of Ship Production and Design, and International Journal
we present the review method and the categories used to of Vehicle Design). In spite of being closer to the topic of
classify the papers. We then present the quantitative results research in engineering design, we discarded these journals
of the number of papers in each of the categories and the for being too specific. The remaining 4 journals were: (i)
cross relations of the different classes, shedding light on the Design Studies (DS), (ii) the International Journal of Design
relative weight of each of the qualitative and quantitative (IJD), (iii) the Journal of Engineering Design (JED) and (iv)
approaches and the most frequent data-collection methods Research on Engineering Design (RED). Table 1 shows that
used. Next, we discuss the usefulness of the obtained results these journals share the category denominated “Engineering
for academics and professionals interested in research design Multidisciplinary”. In this category, there are 6 journals that
and the paper ends with the conclusions. Complementary have the term “Design” in the title, the four selected plus
material is provided with a brief description of each of the International Journal of Technology and Design Education
analysed papers. (also indexed in SSCI), Artificial Intelligence for Engineer-
ing Design Analysis and Manufacturing (also indexed in
Computer Science) that were discarded for being specialized
2 Method in education and in artificial intelligence with applications
in engineering design, respectively, and therefore, out of the
We follow Kitchenham et al. (2009) as a guideline for per- focus of our research.
forming the systematic review. The nature of the research Each of the selected journals declare in their presenta-
question did not suit a usual search in the databases, as we tion their aims and audience: RED focuses on design theory
were interested in analysing the approaches to research pub- and methodology, DS focuses on design processes, JED
lished in the field of engineering design. For this reason, we focuses on different aspects of the design of engineered
focused on identifying papers published in relevant journals products and systems, and IJD publishes research papers
in the field. The data sources are journal papers in the field in all fields of design. The audience of DS, JEC and IJD is
of engineering design. broader than the one of RED, which focuses on mechanical,
A simple search in the Journal of Citation Reports using civil, architectural, and manufacturing engineering. Overall,
the term “Design” as a key search title criterion, generates the four journals constitute a rich and representative sample
a list of 99 journals indexed in different categories. Only that includes works of diverse nature, applying a variety of
80 are indexed in 2020, the rest of them in previous years.

13

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

 224

13
Table 1  Journals that are the focus of interest in the study. Eng Mult is the category named “Engineering, Multidisciplinary”, Eng Manu the one named “Engineering, Manufacturing” and Eng
Ind the one named “Engineering, Industrial” of the SCIE JCR index. SS Inter is the category “Social Sciences, Interdisciplinary” of the SSCI JCR index. In the cells, A/B figures mean number
of reviewed (A) papers versus total number of papers (B). Special issues are underlined.

Eng Eng Eng Ind SS Inter Impact #TOTAL N/2018 D/2018 J/2019 F/2019 M/2019 A/2019 M/2019 J/2019 J/2019 A/2019 S/2019 O/2019 N/2019
Mult Manu factor
(2020)

Design X X 2.780 17/35 0/0 0/0 1/6 0/0 1/1 0/0 2/4 0/0 8/8 0/0 2/7 0/0 3/9
Studies
Interna- X X X 1.923 17/17 0/0 8/8 0/0 0/0 0/0 5/5 0/0 0/0 0/0 4/4 0/0 0/0 0/0
tional
Journal
of
Design
Journal of X 2.588 17/24 1/2 0/0 0/0 1/1 0/0 3/3 0/0 2/2 1/1 5/5 0/0 4/10 0/0
Engi-
neering
Design
Research X X X 2.655 17/28 0/0 0/0 7/8 0/0 0/0 4/8 0/0 0/0 4/7 0/0 0/0 2/5 0/0
in Engi-
neering
Design

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Research in Engineering Design (2023) 34:221–256
Research in Engineering Design (2023) 34:221–256 225

research methods and approaches to different problems in In the rest of this section, we present the categories that
the context of research in engineering design. were identified in the analysis under each topic. Appen-
Sample selection in systematic literature reviews must dix shows complementary information with representative
be structured, comprehensive, and transparent (Hiebl examples of the categories.
2021). To comply with these three requirements, we estab-
lished a recent and limited temporal window and applied 2.1 Aims and contributions
random selection to select the sample. We collected 17
papers from each journal, as 17 is the number of papers Concerning the aims/contributions of the research (RQ1),
available in one of the journals under analysis (IJD) and we started from an empty list of research targets which
we chose to use the same number of papers per journal was enriched as the number of reviewed papers increased.
to avoid bias (i.e., giving more importance to one journal Finally, the following research goals were identified through
than another) in the study. For the journals with more than the coding process:
17 papers in the period of analysis, random selection was To study or propose a methodology, that focuses on
applied. We focused on papers published between Novem- papers whose main objective is to study an existing design
ber 2018 and November 2019, which was the most recent methodology by analysing its validity in works that propose
available time window when this work was started. a new design methodology or that develop a part of it more
This methodology led to a final total of 68 papers. We deeply.
followed a collaborative team-coding approach (Saldaña To delve into a given aspect of design, which includes
2021). Papers were selected and assigned randomly to a papers that focus on exploring an aspect of a design (team
pair of reviewers. Each reviewer coded two papers every communication, sketching, generation of ideas, materi-
two weeks. Disagreements and new code proposals were als...) or that explore one area of design that is recognised
resolved in periodic meetings involving the four research- as challenging (social design, inclusive design, ecological
ers/authors. The first author of this paper played the role of design...).
“codebook editor” (MacQueen and Guest 2008), updating To design, develop, or test a specific product, which
the code list after the meetings and he used the data from includes those papers that set out the process of creation or
the analysis to build the final tables and present the result- development of a specific product or a group of them. Some
ing themes derived from the study. of these works describe the overall process of creating a
With the aim of answering the general question of this product, and others focus on a specific phase of its develop-
review, RQ:, “What is the current landscape of research ment (research, ideation, testing, and validation).
methods in engineering design?”, we focused on the fol- To make recommendations or propose guidelines,
lowing more specific sub-questions: which include articles whose main aim is to systematize the
results of their research to provide advice, either at a meth-
RQ1: What are the research goals pursued by the analysed odological level or in the design of new products.
works? Proposing a theory includes those articles that use logi-
RQ2: What are the main experimental approaches found cal reasoning or mental operations, such as imagination,
in the reviewed papers? intuition, abstraction, and deduction, with the aim of enunci-
RQ3: What data collection methods are employed in the ating concepts or creating models, explanations, or theories
reviewed works? about the phenomena under study.
RQ4: Which instruments are normally used to collect Proposing a framework of analysis or a taxonomy that
these data? enables concepts or objects to be classified into categories.
More than one code could be assigned to each of the
To answer these questions, we followed an anticipated papers. This could be the case of a paper that aims to develop
data condensation approach (Miles et al. 2020). We defined a specific product and ends by proposing guidelines.
four overarching topics corresponding to the research sub-
questions: aims and contributions of the research; research
approach; data collection techniques; and instruments for 2.2 Research approach
the collection of data. For each topic, we defined a set of
categories, based on our revision of engineering design Concerning experimental approaches found in the reviewed
methods (see Sect. 2). During the iterative coding work, papers (RQ2), as explained in the introduction, we propose
emerging categories were included when required. The the use of the distinction between quantitative, qualitative,
new categories were used to re-codify all the works. This mixed, and analytical research methods, defined as:
combination of deductive and inductive coding enabled us Quantitative empirical studies are those that aim at test-
to derive new meanings from the data. ing theories by examining relationships between variables,

13

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

226 Research in Engineering Design (2023) 34:221–256

based on the collection of numerical data which is analysed they live in the world (Stigliano 1989). This technique is
using statistical procedures. popular in architecture (Pérez-Gómez 1999).
Qualitative empirical studies are those that aim at Case study research is an empirical strategy of inquiry
exploring and understanding in depth the meaning that indi- that investigates a contemporary phenomenon within its
viduals or groups give to a problem. They usually involve real-life context (Yin 2009). It uses descriptions of pro-
the collection of non-numerical data obtained in the partici- grams, events, or other phenomena to construct a complete
pants’ settings and follow inductive analysis approaches in portrayal of a case for interpretation and possible action
which the researchers interpret the meanings of the collected (Lapan et al. 2012).
data. Eikeland (2006) describes different approaches to action
Mixed-methods studies are those that combine both research that involve applied research, moving experimen-
quantitative and qualitative approaches at diverse levels tation from laboratories to field, inviting the subjects of
(data sources, analytical methods, etc.), so that the overall research to join the community of researchers and involv-
study is stronger than using each of the two approaches (i.e., ing practitioners in research with the insistence of think-
quantitative, or qualitative) separately. ing through personal practices. Action research is a very
Analytical studies are those that focus on the formali- popular approach in social sciences (Stringer 2008; Clark
zation of a model and its demonstration. They start out by et al. 2020) and it is also proposed for architecture (Herr
proposing a formal model with a mathematical formulation, 2015) and for the practice of product design (Swann 2002).
derive results using deductive approaches, and, if possible, This method is related to the terms research-through-design,
compare these results with empirical observations. practice-based-design research or research-by-design (Red-
With respect to quantitative empirical studies, we subcat- ström 2017; Vaughan 2017), that has been discussed to be a
egorize them into experiments, quasi-experiments and non- kind of action research in works like (Kennedy-Clark 2013;
experiments, depending on the way the subjects of interest Motta-Filho 2021).
are assigned to an experimental group or to a control group: Case study is generally used for exploratory research
Experiments: the assignment of subjects to the experi- or for pre-testing some research hypotheses (Blessing and
mental or to the control group is random. Chakrabarti 2009). Action research requires a high degree of
Quasi-experiments: there is not a random assignment of flexibility and is usually qualitative, data-driven, participa-
a subject to the groups. tory, and makes use of multiple data sources. Case study and
Non-experiments: there is not control on the grouping action research also appear in the following criteria of classi-
of subjects. fication, following the proposal of Blessing and Chakrabarti
When a known qualitative strategy of inquiry is used, (2009) referring to data-collection techniques.
it is also tagged. According to the definition proposed by
Creswell (2009), strategies of inquiry are types of methods, 2.3 Data‑collection techniques
designs or models that provide specific direction for proce-
dures in a research design. In this subsection, we present the list of data-collection tech-
Ethnographic research documents the beliefs and prac- niques we have tagged, to analyse what is proposed in RQ3.
tices of a particular cultural group or phenomenon in its Following the list of data-collection methods presented in
natural environment from the perspective of insiders (Lapan section A.4 of Blessing and Chakrabarti (2009), excluding
et al. 2012). The researcher stays on site for a considera- experiments, case studies and action research we prefer to
ble amount of time to analyse practices and behaviours of include in the list of inquiry research strategies presented in
groups, by observing, interviewing and (sometimes) partici- the previous subsection.
pating in the process under analysis. Very popular in social Observation is a technique in which the researcher
sciences, it is also used in architecture (Cranz 2016). records, in real time, what is happening, either by hand,
In phenomenological research, the researcher identifies recording it or using measuring equipment. As Blessing and
the essence of human experiences about a phenomenon as Chakrabarti (2009) explain: ‘The quality of observational
described by participants, while the researcher sets aside his data is highly dependent on the skill, training and compe-
or her own perspective (Wilson 2015). tency of the observer’ (Blessing and Chakrabarti 2009).
Grounded theory is a strategy of inquiry in which the Observations are the main source of data in ethnographic
researcher derives a general theory grounded in the views studies (see Sect. 2.1), but this strategy is also commonly
of participants, involving the use of multiple stages of data used in social sciences (Creswell 2009) and in visual design
collection (Jørgensen 2001). (Goodwin 2000), architecture (Cuff 1992) and product
Hermeneutics inquiry focuses on disclosing how partici- design practice (Wasson 2000).
pants’ interpretations of a phenomenon determine the way Simultaneous verbalization refers to the situation in
which the participants speak aloud while using a system,

13

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Research in Engineering Design (2023) 34:221–256 227

with the aim of providing information about the cognitive mixes aspects of interviews and observations, as it provides
behaviour of the participants, which may not be obtained information from the study of the interactions between par-
through normal observation (Ohnemus and Biers 1993). ticipants. Focus groups can provide richer information than
Often used to analyse problem-solving behaviour, its most interviews, but they can have a negative effect on the con-
important feature is the real-time aspect. Simultaneous ver- tribution of specific participants.
balization sessions usually last a few hours and never more
than a day, due to the effort required by both the partici- 2.4 Instruments for the collection of data
pants and the researchers in their corresponding analysis.
Although audio recordings are sometimes used to record Data collection methods are supported by instrumentation.
simultaneous verbalization, they are understood as inappro- This section describes the categories we found to respond
priate for a process such as design, which usually involves to RQ4, exposing the instruments that are normally used to
drawings and gestures, so video recordings are considered collect these data. Independently of the strategy of inquiry
more appropriate. applied, there are several instruments that are used to keep
Collecting technical documents consists of obtaining records of the observations. These recordings are impor-
technical documents related to a particular project, topic tant to keep evidence and to enable the reproducibility of
or product, from various sources (Rapley and Rees 2018). the analysis. We tagged the papers depending on the use of
Analysis of these documents is often used early in a research classical audio, video and image recordings and the more
project to understand the organisation, the background of the recent technique of eye tracking (Bergstrom and Schall
project and the experience of the designers. It is commonly 2014).
employed in most observational studies. However, if it is In experiments and case studies, we are also inter-
used as a single source of information, it can result in such ested in physical measurements that are used to objectify
limitations as the usual lack of data on the context in which observations.
the documents were created and the reason for their content. When questionnaires and/or interviews are the data-col-
It is, therefore, convenient to complement them with other lection techniques, we tagged who is the attendee, distin-
methods such as interviews. guishing between stakeholders, users of products or par-
Collecting physical objects involves mock-ups, proto- ticipants (observed people) in the research and experts or
types and other physical models that may be relevant for designers. We also found it relevant to tag when the study
developing a product or testing it. The model or prototype uses workshops as a means to obtain information.
could refer to a part of the product or the whole product. For The last topic of interest that has been tagged is the fact
traditional engineering research, which focuses, for example, that the research work uses simulation algorithms or tools
on the analysis of product behaviour, the products are the as a source of information. We use this tag when the simula-
main source of data (Blessing and Chakrabarti 2009). In our tion tools are a fundamental part of the research, as it pro-
review, we consider those works that start collecting differ- vides the information analysed in the paper (Behera et al.
ent objects to carry out a study on their usefulness, or on 2019), or because the tool or the algorithm itself is the main
the behaviour of users, for example. The object is a general contribution (Mathias et al. 2019).
term that can refer both to drawings and physical objects.
Among the former, we find all those sketches, drawings and
diagrams that have emerged throughout the conception of a 3 Results
product or its development, or throughout a research process,
which could yield important information to organise ideas 3.1 Aims and contributions of the reviewed papers
and draw conclusions.
Questionnaires are used to collect people´s thoughts or Table 2 shows the codes assigned to each of the papers ana-
opinions about a certain product, process or method (Rad- lysed. This section summarises the results related to RQ1
hakrishna 2007). A priori, they seem easier to use than real- (research goals). As shown in Table 2, most of the works
time methods, such as observation or simultaneous verbali- focus on methodologies or on the analysis of a specific
zation, and they are useful to obtain data from a greater aspect of the design processes. The presentation of a product
number of cases. However, some of its disadvantages, such and the building up of knowledge with taxonomies, guide-
as the time required by the participants and the potential bias lines, theories, or reviews, are exceptions.
of the results, must also be taken into account. Five papers propose a theory: (Comi et al. 2019) pre-
Interviews have the same purpose as the questionnaires sent the concept of shared professional vision; (Benavides
but are carried out face-to-face (King et al. 2019). Some- and Lara-Rapp 2019) present the principle of weaker
times they are not carried out individually but using a group dependencies in axiomatic design; (Martinec et al. 2019)
dynamic known as focus group: a group interview that introduce the state-transition model (synthesis, analysis,

13

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

 Table 2  Coding of the papers analysed, according to the categories identified under each topic of the study
228

Ref Aim/results Approach

13
To study or To study a To design, Recom- Proposing a Proposing a Analytical Quantitative Mixed Qualitative
propose a given aspect develop or mendations/ theory framework of
methodology of designing evaluate Proposing a analysis or a
a specific guideline taxonomy
product
Design stud- (Bresciani X X X X
ies 2019)
(Comi et al. X X X
2019)
(Cooper X X
2019)
(Goucher- X X
Lambert and
Cagan 2019)
(Hanrahan X X X
et al. 2019)
(Hyysalo X X X
et al. 2019b)
(Khalaj and X X
Pedgley
2019)
(Lloyd 2019) X X
(Luck 2019) X X
(Mathias et al. X X
2019)
(McDonald X X
and Michela
2019)
(McKinnon and X X X
Sade 2019)
(Reimlinger X X X
et al. 2019)
(Roy and War- X X X
ren 2019)

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

(Self 2019) X X
(Van der X X
Linden et al.
2019b)
(Van Kuijk X X X
et al. 2019)
Research in Engineering Design (2023) 34:221–256
 Table 2  (continued)
Ref Aim/results Approach
To study or To study a To design, Recom- Proposing a Proposing a Analytical Quantitative Mixed Qualitative
propose a given aspect develop or mendations/ theory framework of
methodology of designing evaluate Proposing a analysis or a
a specific guideline taxonomy
product

International (Aktas and X X X

Journal of Mäkelä
Design 2019)
(Barati et al. X X
2019)
(Daalhuizen X X
Research in Engineering Design (2023) 34:221–256

et al. 2019)
(Feijs and X X
Toeters
2018)
(Genç et al. X X X
2018)
(Hobye and X X X
Ranten
2019)
(Hyysalo X X
et al. 2019a)
(Li and Luxi- X X X
mon 2018)
(Park-Lee X X X
and Person
2018)
(Pedgley et al. X X X
2018)
(Petreca et al. X X X X
2019)
(Roesler et al. X X
2019)
(Selvefors X X
et al. 2018)

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

(Takahashi X X
et al. 2018)
(Tsai and Van X X
Den Hoven
2018)

13
229
 Table 2  (continued)
230

Ref Aim/results Approach

13
To study or To study a To design, Recom- Proposing a Proposing a Analytical Quantitative Mixed Qualitative
propose a given aspect develop or mendations/ theory framework of
methodology of designing evaluate Proposing a analysis or a
a specific guideline taxonomy
product

(Van der X X
Linden et al.
2019a)
(Vegt et al. X X X X
2019)
Journal of (Abi Akle X X
Engineering et al. 2019)
Design
(Chen et al. X X
2019a)
(Belkadi et al. X X
2019)
(Benavides X X X
and Lara-
Rapp 2019)
(Boussuge X X X
et al. 2019)
(Chen et al. X X
2019b)
(Cheong and X X X
Butscher
2019)
(Graeff et al. X X
2019)
(Hagedorn X X X
et al. 2019)
(Morkos et al. X X X
2019)
(Ozer and X X
Cebeci

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

2019)
(Pakkanen X X
et al. 2019)
(Saravanan X X
and Jerald
2019)
Research in Engineering Design (2023) 34:221–256
 Table 2  (continued)
Ref Aim/results Approach
To study or To study a To design, Recom- Proposing a Proposing a Analytical Quantitative Mixed Qualitative
propose a given aspect develop or mendations/ theory framework of
methodology of designing evaluate Proposing a analysis or a
a specific guideline taxonomy
product

(Sung et al. X X
2019)
(Valverde X X X
et al. 2019)
(Wand et al. X X
2019)
Research in Engineering Design (2023) 34:221–256

(Wlazlak X X
et al. 2019)
Research in (Behera et al. X X
Engineering 2019)
Design
(De Lessio X X X
et al. 2019)
(Franceschini X X X
and Mai-
sano 2019)
(Garcia et al. X X
2019)
(Gyory et al. X X
2019)
(Jagtap 2019) X X
(Martinec X X
et al. 2019)
(Menold et al. X X
2019)
(Piccolo et al. X X
2019)
(Saliminamin X X
et al. 2019)

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

(Santolaya X X
et al. 2019)
(Tahera et al. X X
2019)
(Wood and X X
Mattson
2019)

13
231
 Table 2  (continued)
232

Ref Aim/results Approach

13
To study or To study a To design, Recom- Proposing a Proposing a Analytical Quantitative Mixed Qualitative
propose a given aspect develop or mendations/ theory framework of
methodology of designing evaluate Proposing a analysis or a
a specific guideline taxonomy
product

(Yang et al. X X X
2019)
(Li et al. X X
2019a)
(Li et al. X X X
2019b)
(Zhang and X X X
Thomson
2019)
25 32 6 12 5 15 10 17 12 32
Ref Research method Data-collect method
Ethnogra- Phenom- Herme- Grounded Action Case study Experi- Observa- Simul- Collecting Collecting Question- Interview-
phy enological neutics theory research ment tions taneous technical objects naires ing
study verbalisa- documents
tion

Design (Bresciani X X X X X
studies 2019)
(Comi X X X X
et al.
2019)
(Cooper X
2019)
(Goucher- X X X
Lambert
and
Cagan
2019)
(Hanrahan X X X X
et al.

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

2019)
(Hyysalo X X X
et al.
2019b)
Research in Engineering Design (2023) 34:221–256
 Table 2  (continued)
Ref Research method Data-collect method
Ethnogra- Phenom- Herme- Grounded Action Case study Experi- Observa- Simul- Collecting Collecting Question- Interview-
phy enological neutics theory research ment tions taneous technical objects naires ing
study verbalisa- documents
tion

(Khalaj X X X X
and
Pedgley
2019)
(Lloyd X X
2019)
(Luck X X
Research in Engineering Design (2023) 34:221–256

2019)
(Mathias X X X
et al.
2019)
(McDon- X X X
ald and
Michela
2019)
(McKin- X X
non and
Sade
2019)
(Reim- X X
linger
et al.
2019)
(Roy and X X
Warren
2019)
(Self X X
2019)
(Van der X X X X
Linden
et al.

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

2019b)
(Van Kuijk X X X X
et al.
2019)

13
233
 Table 2  (continued)
234

Ref Research method Data-collect method

13
Ethnogra- Phenom- Herme- Grounded Action Case study Experi- Observa- Simul- Collecting Collecting Question- Interview-
phy enological neutics theory research ment tions taneous technical objects naires ing
study verbalisa- documents
tion

Interna- (Aktas and X X

tional Mäkelä
Jour- 2019)
nal of
Design
(Barati X X X
et al.
2019)
(Daalhui- X
zen et al.
2019)
(Feijs and X X
Toeters
2018)
(Genç X X X
et al.
2018)
(Hobye X X
and
Ranten
2019)
(Hyysalo X X X X X
et al.
2019a)
(Li and X X X
Luximon
2018)
(Park-Lee X X
and
Person
2018)

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

(Pedgley X X X
et al.
2018)
(Petreca X
et al.
2019)
Research in Engineering Design (2023) 34:221–256
 Table 2  (continued)
Ref Research method Data-collect method
Ethnogra- Phenom- Herme- Grounded Action Case study Experi- Observa- Simul- Collecting Collecting Question- Interview-
phy enological neutics theory research ment tions taneous technical objects naires ing
study verbalisa- documents
tion

(Roesler X X X X X
et al.
2019)
(Selvefors X X X X
et al.
2018)
(Takahashi X X X X
Research in Engineering Design (2023) 34:221–256

et al.
2018)
(Tsai and X X X X
Van Den
Hoven
2018)
(Van der X X
Linden
et al.
2019a)
(Vegt et al. X X X X
2019)
Journal of (Abi Akle X X
Engi- et al.
neering 2019)
Design
(Chen X
et al.
2019a)
(Belkadi X
et al.
2019)
(Benavides X
and

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Lara-
Rapp
2019)
(Boussuge X X
et al.
2019)

13
235
 Table 2  (continued)
236

Ref Research method Data-collect method

13
Ethnogra- Phenom- Herme- Grounded Action Case study Experi- Observa- Simul- Collecting Collecting Question- Interview-
phy enological neutics theory research ment tions taneous technical objects naires ing
study verbalisa- documents
tion

(Chen X X
et al.
2019b)
(Cheong X X
and
Butscher
2019)
(Graeff X X X X
et al.
2019)
(Hagedorn X
et al.
2019)
(Morkos X X
et al.
2019)
(Ozer and X X
Cebeci
2019)
(Pakkanen X X X X X X
et al.
2019)
(Saravanan X
and
Jerald
2019)
(Sung X X X
et al.
2019)
(Valverde X X X
et al.

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

2019)
(Wand X
et al.
2019)
(Wlazlak X X X X
et al.
2019)
Research in Engineering Design (2023) 34:221–256
 Table 2  (continued)
Ref Research method Data-collect method
Ethnogra- Phenom- Herme- Grounded Action Case study Experi- Observa- Simul- Collecting Collecting Question- Interview-
phy enological neutics theory research ment tions taneous technical objects naires ing
study verbalisa- documents
tion

Research (Behera X X
in Engi- et al.
neering 2019)
Design
(De Lessio X X X X X
et al.
2019)
Research in Engineering Design (2023) 34:221–256

(Frances- X X X
chini and
Maisano
2019)
(Garcia X X
et al.
2019)
(Gyory X X X
et al.
2019)
(Jagtap X
2019)
(Martinec X X X X
et al.
2019)
(Menold X X
et al.
2019)
(Piccolo X X
et al.
2019)
(Salimi- X X
namin
et al.

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

2019)
(Santolaya X X X
et al.
2019)
(Tahera X X X
et al.
2019)

13
237
 Table 2  (continued)
238

Ref Research method Data-collect method

13
Ethnogra- Phenom- Herme- Grounded Action Case study Experi- Observa- Simul- Collecting Collecting Question- Interview-
phy enological neutics theory research ment tions taneous technical objects naires ing
study verbalisa- documents
tion

(Wood and X X X X
Mattson
2019)
(Yang X X
et al.
2019)
(Li et al. X X
2019a)
(Li et al. X X
2019b)
(Zhang X X
and
Thomson
2019)
4 3 4 1 7 37 24 19 3 26 17 15 22
Ref Instrument Human input
Measure- Audio record- Video record- Photographs Eye tracking Simulation/ Stakeholder User/partici- Expert/ Workshops
ments ings ings software opinions pant opinions designer
opinions

Design stud- (Bresciani X

ies 2019)
(Comi et al. X X X X
2019)
(Cooper
2019)
(Goucher- X
Lambert
and Cagan
2019)

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

(Hanrahan X
et al. 2019)
(Hyysalo X X
et al. 2019b)
Research in Engineering Design (2023) 34:221–256
 Table 2  (continued)
Ref Instrument Human input
Measure- Audio record- Video record- Photographs Eye tracking Simulation/ Stakeholder User/partici- Expert/ Workshops
ments ings ings software opinions pant opinions designer
opinions

(Khalaj and X X X
Pedgley
2019)
(Lloyd 2019)
(Luck 2019)
(Mathias et al. X X
2019)
(McDonald X X X
Research in Engineering Design (2023) 34:221–256

and Michela
2019)
(McKinnon X X X
and Sade
2019)
(Reimlinger X X X X
et al. 2019)
(Roy and
Warren
2019)
(Self 2019) X X X X
(Van der X X X
Linden et al.
2019b)
(Van Kuijk X X
et al. 2019)
International (Aktas and X X X
Journal of Mäkelä
Design 2019)
(Barati et al. X X X
2019)
(Daalhuizen
et al. 2019)

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

(Feijs and X
Toeters
2018)
(Genç et al. X X X
2018)

13
239
 Table 2  (continued)
240

Ref Instrument Human input

13
Measure- Audio record- Video record- Photographs Eye tracking Simulation/ Stakeholder User/partici- Expert/ Workshops
ments ings ings software opinions pant opinions designer
opinions

(Hobye and X
Ranten
2019)
(Hyysalo X X X
et al. 2019a)
(Li and Luxi- X
mon 2018)
(Park-Lee X X
and Person
2018)
(Pedgley et al.
2018)
(Petreca et al.
2019)
(Roesler et al. X
2019)
(Selvefors X X
et al. 2018)
(Takahashi X X X X X
et al. 2018)
(Tsai and Van X X X
Den Hoven
2018)
(Van der X X X
Linden et al.
2019a)
(Vegt et al. X X
2019)
Journal of (Abi Akle X X
Engineering et al. 2019)
Design

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

(Chen et al. X
2019a)
(Belkadi et al. X
2019)
(Benavides
and Lara-
Rapp 2019)
Research in Engineering Design (2023) 34:221–256
 Table 2  (continued)
Ref Instrument Human input
Measure- Audio record- Video record- Photographs Eye tracking Simulation/ Stakeholder User/partici- Expert/ Workshops
ments ings ings software opinions pant opinions designer
opinions

(Boussuge X
et al. 2019)
(Chen et al.
2019b)
(Cheong and X
Butscher
2019)
(Graeff et al. X
Research in Engineering Design (2023) 34:221–256

2019)
(Hagedorn X X X
et al. 2019)
(Morkos et al. X
2019)
(Ozer and X X X
Cebeci
2019)
(Pakkanen X X
et al. 2019)
(Saravanan X
and Jerald
2019)
(Sung et al. X X X
2019)
(Valverde X
et al. 2019)
(Wand et al. X
2019)
(Wlazlak X X
et al. 2019)
Research in (Behera et al. X
Engineering 2019)

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Design
(De Lessio X X
et al. 2019)
(Franceschini X
and Mai-
sano 2019)

13
241
 Table 2  (continued)
242

Ref Instrument Human input

13
Measure- Audio record- Video record- Photographs Eye tracking Simulation/ Stakeholder User/partici- Expert/ Workshops
ments ings ings software opinions pant opinions designer
opinions

(Garcia et al. X X
2019)
(Gyory et al. X X
2019)
(Jagtap 2019)
(Martinec X X
et al. 2019)
(Menold et al. X X
2019)
(Piccolo et al. X
2019)
(Saliminamin X X X
et al. 2019)
(Santolaya X
et al. 2019)
(Tahera et al. X X
2019)
(Wood and X X X X
Mattson
2019)
(Yang et al. X X
2019)
(Li et al.
2019a)
(Li et al. X X
2019b)
(Zhang and X X
Thomson
2019)
10 14 11 6 3 20 2 22 20 11

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Research in Engineering Design (2023) 34:221–256
Research in Engineering Design (2023) 34:221–256 243

evaluation) in conceptual design and Lloyd (2019) defends The most frequent type of works delve into a particu-
the theory of the social turn in design, Aktas and Mäkelä lar aspect of product design such as sketching (Sung et al.
(2019) focus on the relation between craft, materials, 2019; Self 2019), prototyping (Menold et al. 2019; Mathias
makers. et al. 2019), material (Pedgley et al. 2018; Aktas and Mäkelä
Six works focus on the evaluation of a specific product: 2019; Barati et al. 2019; Petreca et al. 2019), interaction
a software product in Takahashi et al. (2018) and Belkadi (Hobye and Ranten 2019; Valverde et al. 2019), briefing
et al. (2019); or physical objects in the case of Roesler (Park-Lee and Person 2018), working in groups (Graeff et al.
et al. (2019), Hyysalo et al. (2019b) and McKinnon and 2019), iterations and testing (Tahera et al. 2019; Piccolo
Sade (2019). et al. 2019); behavioural complexity (Hobye and Ranten
Concerning the works related to methodologies, we find 2019), manufacturing (Yang et al. 2019), or usability (Van
papers that propose a method based on analytical methods Kuijk et al. 2019).
or algorithmic solutions such as those related to axiomatic
design (Chen et al. 2019a) and those related to such meth- 3.2 Strategies of inquiry and methodologies
ods as research-through-design, where the importance of
the method followed is prominent in the study (Tsai and Van This section summarises the results related to RQ2 (main
Den Hoven 2018; Hyysalo et al. 2019b; McKinnon and Sade experimental approaches founded): qualitative approaches
2019; Hanrahan et al. 2019); or methodologies for product are a majority, but the number of quantitative or mixed-
development such as Daalhuizen et al. (2019), with emphasis methods studies is also relevant. Other approaches, such
on different aspects such as work in groups (Gyory et al. as the use of analytical methods, are less frequent. Table 3
2019), sustainability (Santolaya et al. 2019) or democratised shows that, when the goal of the paper is related to propos-
design (Hyysalo et al. 2019a). ing or studying a methodology (first column in Table 3),
A good number of papers present frameworks of analysis the percentage of pure quantitative papers is lower than in
or classifications with different purposes. Bresciani (2019) the rest of the cases. Regarding whether there is a tendency
for classifying visualization dimensions, McDonald and towards any methodology depending on the journal; Table 2
Michela (2019) to classify moral goods, Roy and Warren shows that the Journal on Engineering Design seems to
(2019) for card sets, Park-Lee and Person (2018) identify focus more than the other journals on non-qualitative strat-
three practices on briefing, Vegt et al. (2019) deduce 3 types egies of inquiry.
of invasiveness evoked by the rules in gamified brainstorm- When quantitative methods are used, experiments are
ing, Valverde et al. (2019) classify the type of feedback in more frequent than quasi-experiments and non-experiments
automotive push buttons, Cooper (2019) presents the five (14 out of the 17 quantitative studies present an experiment).
waves in design research, Luck (2019) describes the frame- We found 26 experimental studies, with 5 quasi-experiments
work to distinguish between design, design research, archi- (Saliminamin et al. 2019; Vegt et al. 2019; Sung et al. 2019;
tectural design research and practice, Hobye and Ranten Self 2019; Santolaya et al. 2019) and 4 non-experiments
(2019) present five behavioural strategies for interactive (Selvefors et al. 2018; Morkos et al. 2019; Roesler et al.
products and Van Kuijk et al. (2019) presents a framework to 2019; Piccolo et al. 2019).
analyse usability concepts of electronic products and Petreca The use of case studies is pervasive in qualitative research
et al. (2019) for analysing the relation between sensors and (more than half the studies that classified as qualitative base
textile. We also include in this category the papers related to the research on a case study). Furthermore, many quantita-
ontologies, that are used to represent knowledge. tive studies support results from case studies; for example,
Proposing recommendations is a common result in the some analytical studies in which case studies are used as
analysed research papers, including a variety of themes such proof of concept of the proposed models (Chen et al. 2019b;
as recommendations on the use of guidelines by new design- Zhang and Thomson 2019; Li et al. 2019a).
ers (Reimlinger et al. 2019); the use of specific materials Nevertheless, other qualitative methods, such as ethnog-
(Genç et al. 2018; Pedgley et al. 2018; Aktas and Mäkelä raphy, hermeneutics, action research and phenomenological
2019; Petreca et al. 2019); how to orient future studies on the studies, are also used. The use of specific methods related to
use of mobile technology by elderly people (Li and Luximon design is scarce (the discussion about this concern is dealt
2018), or about design and poverty (Jagtap 2019) or ethno- with in detail below). Ethnography is used in three cases
graphic studies in developing countries (Wood and Matt- (Roesler et al. 2019; Van der Linden et al. 2019a; Comi et al.
son 2019); appliance design (Selvefors et al. 2018); use of 2019)—also the annotation as observation in the tables—and
games in brainstorming (Vegt et al. 2019); or specifying one more paper uses ethnography as the study focus (Wood
requirements (Morkos et al. 2019). Cooper (2019) proposes and Mattson 2019). Hermeneutics is used by (McDonald
interprets the history of design research through five waves. and Michela 2019; Cooper 2019; Lloyd 2019; Luck 2019).

13

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

 244

13
Table 3  Number of works in each (sub)category per research approach
Aim/results Research method Data-collect method

To study To study To Recom- Propos- Propos- Ethnog- Phe- Herme- Grounded Action Case Experi- Obser- Simul- Collect- Col- Ques- Inter-
or pro- a given design, menda- ing a ing a raphy nom- neutics theory research study ment vations taneous ing tech- lecting tion- viewing
pose a aspect of develop tions/ theory frame- eno- verbali- nical objects naires
method- design- or evalu- Proposing work of logical sation docu-
ology ing ate a a guide- analysis study ments
specific line or a tax-
product onomy

Analyti- 7 1 0 1 1 4 0 0 0 0 0 9 2 0 0 5 0 1 0
cal
Quanti- 6 10 0 2 1 2 0 0 0 0 0 7 14 2 2 9 7 5 1
tative
Mixed 4 5 3 4 0 0 1 2 0 0 1 5 10 2 1 1 1 6 5
Qualita- 11 16 3 5 3 9 3 1 4 1 6 18 0 15 0 13 9 4 16
tive
Instrument Human input

Measurements Audio record- Video record- Photographs Eye tracking Simulation/ Stakeholder User/participant Expert/designer Workshops
ings ings software opinions opinions opinions

Analytical 2 0 0 0 0 7 0 2 1 0
Quantitative 7 3 3 1 0 5 0 5 5 1
Mixed 3 1 3 0 3 4 2 7 4 3
Qualitative 0 10 5 5 0 6 0 9 10 7

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Research in Engineering Design (2023) 34:221–256
Research in Engineering Design (2023) 34:221–256 245

Action research is used by Pakkanen et al. (2019) to 2018; Sung et al. 2019) sketches are the main concern of
investigate, in combination with case studies, modular the research.
systems in industrial environments. The work of Bresciani Questionnaires are less frequently used, and when this
(2019) could be considered an action research study with happens, they are designed ad-hoc for each study. Given
the goal of building a grounded theory evaluation technique the wide variety of topics and aims of the reviewed works,
for visual thinking. McKinnon and Sade (2019) align their no standardised questionnaires have been found. Question-
work in the field of research through design using a set of naires, therefore, take different formats: Amazon Mechanical
gadgets to obtain information about environmental home Turk is used once (Goucher-Lambert and Cagan 2019); a
good practices. Research through design is also used by Likert scale tool evaluation (Graeff et al. 2019); binary and
Genç et al. (2018) to explore new materials and Tsai and open questions (Pakkanen et al. 2019); ranking of prefer-
Van Den Hoven (2018) to explore user experience. Hyysalo ences (Franceschini and Maisano 2019); or ad-hoc software
et al. (2019b) and present the evaluation of a panel follow- tools (Li et al. 2019a).
ing the principles of research through design. Close to this Interviews are frequently used as a source of information
method is that presented by Barati et al.(2019), who comple- in qualitative and mixed strategies of inquiry. Interviews are
ment their study with workshops where a group of students associated with phenomenological studies (Li and Luximon
explores their proposals. 2018; Park-Lee and Person 2018; Selvefors et al. 2018) and
also in ethnographic studies (Roesler et al. 2019; Van der
3.3 Data‑collection methods Linden et al. 2019a; Wood and Mattson 2019; Comi et al.
2019). The interviewed population can be a group of users
Results regarding RQ3 (data collection methods) are sum- of a given technology (Li and Luximon 2018) or a group of
marised in this section. Table 3 shows which main methods experts (Bresciani 2019).
and techniques for collecting data are used in the different Concerning the sample size used in the 24 papers whose
studies. The analysis of the sources of information is com- research method has been classified as experimental, and
pleted with a revision of the instruments used to collect data taking into account that the sample may refer to studied
and with a discussion about the role of human input pre- objects or to participants/users, which, in turn, may be indi-
sented in the following sections. None of the data-collec- viduals or teams, the number of participants/users varies
tion methods identified seem to be dominant in the papers between 4, in Martinec et al. (2019), and 169, in Ozer and
studied. Cebeci (2019). The number of studied objects also varies
Technical documents of diverse nature are the main from 6, in Mathias et al. (2019) to 256, in Li et al. (2019b).
source of information used (Table 2 reports 23 out of the In Santolaya et al. (2019) a methodology is experimentally
68 papers analysed using technical documents). Interview- tested in 2 case studies.
ing is also frequent (22 times reported in Table 2). Expert
and user opinions are both used as sources of information, 3.4 Instruments
but neither is a majority (22 and 20 papers, respectively,
reported in Table 2. Observation is mostly used in qualita- Results regarding RQ4 (instruments used to collect data) are
tive studies, where almost half use this technique. Concern- summarised in this section. Measurements refer both to met-
ing quantitative studies, apart from measurements, expert rics obtained with a physical device and to qualitative ratings
opinions appear as a frequent resource. This is because it obtained from human-based scores. In the first group, we
is common to collect the opinions of experts in question- can mention the metrics of energetic consumption (Selvefors
naires or in evaluation templates that convert opinions into et al. 2018; Santolaya et al. 2019), mass material (Santolaya
numeric values. et al. 2019), volumes of objects (Mathias et al. 2019), dis-
Verbalization is used in Martinec et al. (2019) and Gyory placement of buttons (Valverde et al. 2019), online shopping
et al. (2019) for team work analysis and in (Khalaj and user interaction data (Ozer and Cebeci 2019), or the timing
Pedgley 2019), where designers and users had to verbalize of tasks in (Mathias et al. 2019). In the second group, we
impressions. can cite (Saliminamin et al. 2019; Gyory et al. 2019), which
Objects are collected as a data source in a relevant num- score the quality of design proposals, and (Franceschini and
ber of studies. Some are the results of students’ work as in Maisano 2019), who use design preferences as the input for
Gralla et al. (2019); brainstorming outputs (Vegt et al. 2019); an analytical model.
prototypes (Feijs and Toeters 2018; Barati et al. 2019), or Simulations and/or software developments of algorithms
commercial products (Roy and Warren 2019). Sketches take on an important role in several papers. Belkadi et al.
are the type of object analysed in (Genç et al. 2018; Mar- (2019) present a software tool; Chen et al. (2019a), Feijs and
tinec et al. 2019; Gyory et al. 2019; Goucher-Lambert and Toeters (2018), Mathias et al. (2019) and Takahashi et al.
Cagan 2019; Comi et al. 2019); while for (Li and Luximon (2018) present or test software tools for different goals, such

13

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

246 Research in Engineering Design (2023) 34:221–256

as analysing Lego buildings, and generating fashion patterns four journals analysed with only minor differences among
for projecting requirements into design parameters. Li et al. them. Thus, DS (Design Studies) and RED (Research in
(2019a) focus on modelling knowledge; Piccolo et al. (2019) Engineering Design) seem to focus more on methodologi-
use analysis and visualization tools to present results; while cal aspects, while IJD (International Journal of Design) and
Ozer and Cebeci (2019) and Saravanan and Jerald (2019) use JED (Journal of Engineering Design) focus more on delving
machine learning techniques such as neural networks and into particular aspects of the design process or on products,
clustering. De Lessio et al. (2019) present a software tool to but at most 7 papers out of the 17 falls into one of the cat-
support planning and Yang et al. (2019) to support manu- egories. According to the results, DS and IJD journals attract
facturing. Boussuge et al. (2019) propose using ontologies more papers with a qualitative approach (only 2 papers in
to capture high-level modelling and idealisation decisions, each journal are purely quantitative), while most of the
characterising the simulations of CAE models from CAD papers from JED and RED follow a quantitative or analyti-
assemblies. Other papers related to ontologies use software cal approach (only 3 and 7 papers, respectively, are purely
to model them (Cheong and Butscher 2019; Hagedorn et al. qualitative). However, we have found papers with both
2019; Wang et al. 2019). approaches in all the journals. RED uses less self-reported
Workshops are frequently used for evaluating results and data (interviews, questionnaires or workshops), while DS
sharing experiences by a group of experts with discussions uses this source of data the most, but in both journals there
(Van der Linden et al. 2019a, b; McKinnon and Sade 2019; are exceptions, such as the works of Mathias et al. (2019) in
Self 2019; Wlazlak et al. 2019). In (Genç et al. 2018; Marti- DS or Garcia et al. (2019) in RED.
nec et al. 2019), the workshops become designing activities Despite this broad spectrum of papers, we found a clear
in the research-through-design methodology. In Takahashi interest in methodologies and the in-depth analysis of a
et al. (2018), workshops are used to observe users while given aspect of the whole process of designing generally
they interact with a system and, in Pakkanen et al. (2019), applied to a particular case study. The interest in both topics
to collect information from experts. In Garcia et al. (2019), is justified by the nature of the design and the youth of the
workshops are meetings with stakeholders. discipline. As a process of searching for optimum solutions,
The opinions of stakeholders can be the core of the design is clearly related to methodological concerns. As a
research study (Self 2019) or they can be used as part of young discipline, the space for contributing to the different
usability tests (Takahashi et al. 2018). Most often, question- tasks of the whole design process is huge. The analysis of
naires and interviews are performed with users of a product the process of engineering design has evolved from being
(Selvefors et al. 2018; Roesler et al. 2019; Hanrahan et al. considered from a purely technical perspective to being stud-
2019; Ozer and Cebeci 2019); by active participants of the ied as a socio-technical process. From a technical point of
process under analysis, such as professionals in companies view, (Beitz et al. 1996) distinguished between conceptual
(Reimlinger et al. 2019; Wlazlak et al. 2019); or by students design and embodied design for identifying a list of tasks
that are required to do a project (Vegt et al. 2019; Li et al. that contribute to facing problems of engineering design in
2019a; Abi Akle et al. 2019; Graeff et al. 2019). The experts an effective and systematic way. From a socio-technical per-
that participate in questionnaires or interviews are designers, spective, different authors have pointed out that the design
architects, engineers (Li and Luximon 2018; Park-Lee and process is influenced by aspects related to teamwork capa-
Person 2018; Pakkanen et al. 2019), or academic staff evalu- bilities (Dorst 2004), the inclusion of participants (Van der
ating results (Morkos et al. 2019; Sung et al. 2019; McKin- Bijl-Brouwer and Dorst 2017) or by the institutional com-
non and Sade 2019). In interviews occurring in ethnographic plexity (Reich and Subrahmanian 2020). Our study shows
studies, the subjects providing information could be con- that there is space for research works that focus on both
sidered the topic of analysis (Wood and Mattson 2019), but perspectives of analysis, being found works that are closely
at the same time, they could be experts (Comi et al. 2019). related to tasks that affect conceptual design (Martinec et al.
2019; Benavides and Lara-Rapp 2019; Self 2019), embodied
design (Petreca et al. 2019) and also to social aspects of the
4 Discussion design process (Piccolo et al. 2019).
It has been observed that there are a relatively low num-
4.1 Variety of aims and approaches ber of papers proposing recommendations, guidelines,
frameworks, and taxonomies. We understand how difficult
The principal finding of our research is that there is a very it is generalizing and classifying a discipline with multiple
high diversity in the works we have analysed in the journals tasks, agents, approaches and sub-domains. Nevertheless,
related to engineering design. This variety affects the aims generating these types of representations of knowledge could
and scopes of the research works, the methods, and the data be a substrate for the growth of the discipline. Design is a
sources. Table 4 shows that variety affects the papers in the context-specific endeavour, but trying to generalize results

13

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

 Table 4  Number of works in the different journals
Journal Aim/results Approach Research method
To study To study To Recom- Propos- Propos- Analyti- Quanti- Mixed Qualitative Eth- Phenom- Herme- Grounded Action Case Experi-
or pro- a given design, menda- ing a ing a cal tative nogra- enologi- neutics theory research study ment
pose a aspect of develop tions/ theory frame- phy cal study
method- design- or evalu- Propos- work of
ology ing ate a ing a analysis
specific guide- or a tax-
product line onomy

Design 7 8 2 1 2 6 0 2 4 11 2 0 4 1 3 5 5
Studies
Interna- 4 10 2 6 1 4 0 2 4 11 1 3 0 0 3 9 5
tional
Journal
Research in Engineering Design (2023) 34:221–256

of Design
Journal of 6 8 1 2 1 5 7 5 2 3 0 0 0 0 1 13 6
Engi-
neering
Design
Research 8 6 1 3 1 0 3 8 2 7 1 0 0 0 0 10 8
in Engi-
neering
Design
Total 25 32 6 12 5 15 10 17 12 32 4 3 4 1 7 37 24
Journal Data-collect method Instrument Human input
Observa- Simul- Collecting Col- Ques- Inter- Measure- Audio Video Photo- Eye Simula- Stakeholder User/ Expert/ Work-
tions taneous technical lecting tion- viewing ments record- record- graphs track- tion/ opinions par- designer shops
verbali- docu- objects naires ings ings ing software ticipant opinions
sation ments opinions

Design 4 1 5 7 5 8 1 5 4 2 2 1 1 3 10 4
Studies
Interna- 10 0 1 6 3 9 1 5 4 3 0 4 1 7 5 3
tional
Journal
of Design
Journal of 2 0 9 1 4 2 2 1 1 0 1 8 0 5 4 2

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Engi-
neering
Design
Research in 3 2 11 3 3 3 6 3 2 1 0 7 0 7 1 2
Engi-
neering
Design

13
247
 248 Research in Engineering Design (2023) 34:221–256

so that other authors could reuse the generated knowledge

Expert/ Work-
in other domains would be positive for the growth of the

designer shops

11
discipline. The selected papers include product development
and engineering design, which are two different areas, albeit
opinions
overlapping. Recommendations and guidelines are always

20
useful for the practice of engineering design, but more
importantly, classifying concepts and types of activities
opinions
ticipant
Stakeholder User/

with frameworks and taxonomies is an essential process in

par-

22

the building of knowledge in any research area. The variety

Human input

of aims and approaches is probably the reason for this defi-

opinions

cit, but research in engineering design would benefit from

works analysing the many methodologies proposed from a
2

meta level that permits obtaining general concepts that are

domain-independent and universally applicable.
software
Simula-

Results presented in Table 2 and summarised in Table 3

tion/

20

could be used to derive patterns or preferred styles in

research design. Papers using analytical approaches mainly
track-

use case studies to validate the proposed models and they

Eye

ing

use simulations to compare results with expectations. Here,

the case studies are used as proof of concept of the proposed
graphs
Photo-

models. They do not consider human input as a main feature

6

of analysis. The ones related to methodological concerns are

the papers focusing on axiomatic design and the ones relat-
record-
Video

ing to specific aspects or to frameworks are the ones related

ings

11

to ontologies. Most papers with quantitative approaches

use experimental setups in which they compare different
record-
Measure- Audio

configurations of a given problem. The means to collect

ings

14

numerical data highly depend on the type of work, with no

Instrument

outstanding method or instrument. This approach is mainly

used when the goal is to study a given aspect of design,
viewing ments

which is coherent with the fact that experiments are meant

10

to measure variables that can be isolated, and therefore these

studies need to focus on specific features of the design pro-
Inter-

22

cess. Like analytical papers, qualitative approaches are

mainly based on case studies. The main difference is related
Ques-

naires

to the nature of these case studies. In qualitative approaches,

tion-

15

the case studies aim at gaining insight into the complexity

of the studied design processes from the point of view of
objects
technical lecting

the participants. In consequence, the preferred data collec-

Collecting Col-

17

tion methods are observations and interviews and/or work-

shops, to collect data from users and experts. They use rich
data sources (audio, photography, video or software tools)
ments
docu-

to make observations rigorously. Qualitative approaches

26

are the most used methods, independently of the aim of the

Data-collect method

verbali-
taneous
Observa- Simul-

paper, but they are dominant for proposing frameworks of

sation

analysis or deriving guidelines and recommendations, prob-

3

ably because the active interpretation of experts is a must

for these concerns. Papers using mixed methods triangulate
Table 4  (continued)

tions

the information obtained in quantitative experiments with

19

information obtained with qualitative methods. Therefore,

their pattern is closer to one of the papers using quantitative
Journal

methods than to the ones using qualitative methods.

Total

13

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Research in Engineering Design (2023) 34:221–256 249

The application of one approach or another should on the journal in which it is published but on its contribu-
respond to what Subrahmanian et al. (2020) call the dif- tion to the growth of the discipline (Bladek 2014). However,
ferent models of designing. When the artifact or the pro- there is a universal tendency to identify research quality and
cess is clear, analytical, and quantitative methods, closer to impact with these publications, and students that pursue a
approaches followed in natural science can be applied. When research career usually need to accomplish certain goals
people, culture, society, and politics must be taken into con- related to publishing. For this reason, we think that doctoral
sideration, the use of analytical and quantitative methods is students in engineering design can find this work useful, as
not appropriate. When individual designers play a role, and, it provides an overview and pointers to different types of
especially, when social aspects and context must be taken research work published in four top-quality journals in the
into consideration, design processes become more complex field, and this may give them tips on the kind of knowledge
and dynamic, involving aspects that are better studied by they need to acquire to have their work published in these
qualitative approaches that are able to capture the complex- journals or similar ones.
ity of the object of study and the participants' perspectives.
4.3 Relation to other surveys
4.2 Implications for the research in the engineering
design community Probably due to the youth of engineering design as a research
discipline, the number of papers devoted to literature reviews
As mentioned in the introduction, one of the objectives in these fields is still sparse. From the few reviews found,
of this paper was to provide suggestions about the course most refer to particular aspects of engineering design: such
contents that doctoral studies in the domain of engineering as inspiration and fixation (Crilly 2019); sustainability
design must carry out. The first implication of our analysis (Coskun et al. 2015); user value (Boztepe 2007); Alzheimer
relates to the type of research methodologies that students and play experience (Anderiesen et al. 2015); performance
must be introduced to. According to the analysed papers, in industrial design(Candi and Gemser 2010); relation
it seems essential that future researchers receive training between creativity, functionality, and aesthetics (Han et al.
in both qualitative and quantitative methods. The analysis 2021); fuzzy front-ends for product development (Park et al.
shows that qualitative research is very common and that rich 2021); surrogate models and computational complexity (Ali-
sources of data, such as observations or users and experts zadeh et al. 2020); smart design (Pessôa and Becker 2020);
opinions collected through interviews are frequent. Further- design and poverty (Jagtap 2019); mass customization (Fer-
more, pure qualitative research approaches, like ethnogra- guson et al. 2014); product stigma (Schröppel et al. 2021);
phy and phenomenology are commonly found. Neverthe- uncertainty (Han et al. 2020); decision-making methods
less, experimental approaches should also have a relevant (Renzi et al. 2017); modular product design (Bonvoisin et al.
role in the student curricula because it is frequently used as 2016); or product-service systems (Vasantha et al. 2012).
well. We understand that this qualitative-quantitative duality More interesting, for their similarity with respect to the
responds to the nature of engineering design, a complex field present study, are the works presented by Tempczyk (1986)
that requires both technical background and the considera- and Cantamessa (2003), both presenting reviews or surveys
tion of behavioural and social aspects related to design. about research and studies on engineering design. These
A second implication has to do with the instruments two works and the one presented in this paper differ in their
and data collection methods that researchers on engineer- sources of information. Tempczyk (1986) made a survey
ing design must get familiar with. Research studies in this by sending questionnaires to academic staff concerning
domain could require accessing real design scenarios that research subjects and methods; Cantamessa (2003) made
are authentic field studies rather than controlled lab studies. a review of the proceedings of two editions of the confer-
This is a relevant divergence with respect to other research ence on engineering design. There is a temporal distance
domains that permit isolating variables and participants. of 17 years between the work of Tempczyk (1986) and the
There are implications for the instruments used for collect- one of Cantamessa (2003) and 18 years between the work
ing data, with the need of considering techniques that per- of Cantamessa (2003) and the present study, but we must
mit collecting information in real settings and during longer highlight the fact that the three studies report methodologies
periods of time. but also, that human fact is a relevant vari- as one of the main topics of research. Computer-aided prod-
able that affects both design teams managements, commu- ucts are reported by Tempczyk (1986) as a relevant topic,
nication with users and social aspects. This fact justifies the and Cantamessa (2003) also refers to software tools as a
use of technical reports, questionnaires, and observation as recurrent topic, while we also identified a category named
the main sources of information in these studies. simulation which included software tools and algorithms.
It must be noted that publishing in a journal should not be The three works also report a high variety of approaches and
an end in itself, and the real value of a paper does not rely themes. The main difference between these studies and the

13

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

250 Research in Engineering Design (2023) 34:221–256

present one is that Tempczyk (1986) reports on training as The analysis of the sample of journal papers selected has
an important concern for researchers and Cantamessa (2003) permitted us to build a consistent set of categories for clas-
observes different streams of research, loosely coupled with sifying research works in engineering design. We consider
an excess of referencing to previous works. As regards refer- this sample comprehensive, based on a saturation analysis
ences to training concerns, we did not find any paper related carried out on the sample, that showed that all the catego-
to training, probably because, nowadays, there are journals ries used in the analysis could be identified with 69% of the
specifically devoted to learning in the domain of engineering papers that were actually used in the analysis. Nevertheless,
and design. As regards the criticism of Cantamessa (2003) while selecting 68 papers from only four journals, we could
concerning the notable amount of self-references in the ana- have discarded other works that could include other alterna-
lysed papers, we did not observe such a circumstance in tive approaches also valid for research in engineering design.
the journal papers we have reviewed. On the contrary, our Moreover, the choice of a single year-window is another
review has found that the papers reviewed contain complete limitation of this study, as it does not enable us to provide
state-of-the-art sections in which other research groups are a full vision of the field and its evolution. Nevertheless, we
referenced and other studies are discussed. This finding think that the classification presented in this paper could be
partially contradicts what Cantamessa (2003) found in his the basis for subsequent studies, which should consider a
review. We think that the nature of the sources of data in his broader timeframe, and therefore, a larger selection of papers
review, based on proceedings which are shorter could have across several years. Other approaches for selecting the ana-
influenced these divergent results. Our study may point to a lysed papers like sampling at the same rate in all the journals
more mature stage of research that builds on the knowledge could also have led to representative results.
already offered in the community. This finding may be based
on the fact we are working on journal papers that offer more
mature results. 5 Conclusions
4.4 Limitations In this paper, we have presented a systematic review of
recent literature on research methods and instruments used
The systematic literature review presented in this paper cov- in a one-year period of research papers in the field of engi-
ers a recent period of time spanning one year of publications. neering design. By taking this approach, we offer a "fixed
The sample is representative of recent research in engineer- image" of recent research in the area and point to some gaps
ing design, but it does not provide information about tenden- and challenges in the field.
cies in the field. For example, we have observed a relevant The review shows that there is no single methodological
number of quantitative studies in comparison to qualitative approach accepted as the standard in the field; and that there
ones, but we cannot say if this is a tendency. Future work is a large variety of goals, approaches, data collection meth-
would be required to compare our results with those of a ods and instruments to collect them. In spite of this variety,
longitudinal study covering a larger period of years. We we have observed a certain preference towards qualitative
expect that our work can be considered as the first step in methods, which can be justified by the increasing considera-
this longer-term study that could provide useful information tion of engineering design as a complex process affecting
about the evolution of research into the young discipline of humans and their contexts.
engineering design. We think that this paper contributes to research in engi-
By selecting Blessing and Chakrabarti (2009) as a frame- neering design by providing initial evidence for researchers
work to categorize research papers, we did not pay attention about the kind of work that are expected by high-impact sci-
to the important concern of the success of the research which entific journals in this domain. Additionally, academics can
could be a critical point for connecting the study aim, with find in this paper a list of topics (methodologies, data-col-
the approach, research method, etc. Reich and Subrahmanian lection procedures, instruments, etc.…) that must be part of
(2021) show that it is possible to use the PSI framework the programme of courses on research in engineering design.
(Problem, Social and Institutional space) to describe what
researchers and designers did in case studies to analyse the
matching of methods, aims and approaches with the success
of the projects. In spite of our work being merely descriptive
6 Appendix: Coding scheme: categories
of the aims, methods and techniques used by authors, we
and examples
offer a corpus of categorised research papers for analysing
The tables included in this Appendix have aim to present the
in future works on whether the research design is appropri-
knowledge generated in this paper in the form of a coding
ate for its goals.
scheme, that can be used as an instrument to describe the

13

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Research in Engineering Design (2023) 34:221–256 251

Table 5  Categories and examples of works corresponding to the topic of aim of the research
Topic: Aim of the research
Category Example

Proposing a methodology and analysing its validity Khalaj and Pedgley (2019) propose a methodology for identifying dis-
continuities between intended and realized semantics when comparing
users’ product impressions vs. designers’ product expressions
To study a given aspect of designing or a challenging area of design Sung et al. (2019) study a given aspect such as the influence of sketch-
ing instruction on students’ design cognition within elementary sci-
ence classrooms
To design or develop a specific product by describing the process of Roesler et al. (2019) present the design process of the Anesthesia Medi-
creation or development of a specific product cation Template which aims at improving medication handling safety
Proposing a guideline by providing advice, either at a methodological Selvefors et al. (2018) formulate guidelines that can aid appliance
level or in the design of new products designers in designing for less energy-intensive use
Proposing a theory by using logical reasoning with the aim of enunci- Comi et al. (2019) propose a theory about how architects and engineers
ating concepts mobilize visual objects to coordinate their professional visions around
a design issue
Proposing a framework of analysis that enables concepts or objects to Bresciani et al. (2019) establish a framework aimed at helping designers
be classified into categories make more informed decisions regarding the visualizations they work
with

Table 6  Categories and examples of works corresponding to the topic of research approaches

Topic: Research approaches

Categories Example

Analytical formalization of a model and its demonstration Franceschini and Maisano (2019) formalize a model to support the deci-
sions of teams of designers in early design stages
Quantitative experiments by examining relationships between vari- Goucher-Lambert and Cagan (2019) use quantitative experiments to
ables with random selection of subjects to experimental and control explore the potential of using an untrained crowd workforce to gener-
groups ate stimuli for trained designers
Quantitative quasi-experiments where there is no random assignment Santolaya et al. (2019) evaluate a methodology to project the design of
of a subject to the experimental and control groups more sustainable products by comparing results before and after its
implementation
Quantitative non-experiments with no control on the grouping of Piccolo et al. (2019) study the role of iterations in design by developing
subjects in experimental and control groups a statistical model to test multiple hypotheses related to technical and
social factors
Qualitative ethnographic research to document the beliefs and prac- Van der Linden et al. (2019b) use a mix of ethnographic techniques to
tices of a particular group in its natural environment analyse the knowledge about the user experience the architects man-
age during their projects
Qualitative phenomenological studies to identify the essence of experi- Li and Luximon (2018) develop a phenomenological study about the
ences about a phenomenon as described by participants mobile technology usability by elder people for a designer to build
specialized interfaces
Qualitative hermeneutic studies to disclose how participants’ interpre- McDonald and Michela (2019) perform a hermeneutic study into the
tations determine the way they live in the world moral goods that are significant for design
studio instructors
Qualitative grounded theory to derive a general theory grounded in the Bresciani et al. (2019) establish a theoretically grounded framework
views of participants aimed at helping designers make more informed decisions regarding
visualizations they work with
Qualitative action research, that involve applied research, moving Tsai and Van Den Hoven (2018) perform an action research study to
experimentation from laboratories to field investigate how the accumulation of human traces on objects influ-
ences people’s remembering and usage
Qualitative case study methods, that are in-depth, detailed examination Tahera et al. (2019) analyze the relationship between testing and design
of particular cases within a real-world context process, by combining literature study with cases studies about design
and testing practice
Mixed to combine both quantitative and qualitative approaches at Ozer and Cebeci (2019) use both qualitative and quantitative criteria to
diverse levels analyse big data to offer customised and personalised online products
with appealing features

13

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

252 Research in Engineering Design (2023) 34:221–256

Table 7  Categories and examples of works corresponding to the topic of data collection techniques

Topic: Data collection techniques

Category Example

Observations where researchers records, what is happening, either by Hyysalo et al. (2019a) detail the process of democratized design of
hand, recording it or using measuring equipment spaces and services of a public library where authors take field notes
from the meetings and workshops
Simultaneous verbalisation where participants verbalize their thoughts Martinec et al. (2019) model design activities of ideation and concept
when performing tasks review by collecting verbalisations in teamworks
Collecting technical documents generated during the engineering Morkos et al. (2019) study the impact of requirement elicitation on
design processes the final project. The requirement documents are quantified and cor-
related with the final results
Collecting objects like mock-ups, prototypes and other physical models Roy and Warren (2019) study card-based design tools making a collec-
that may be relevant for the designing process tion, review and analysis of 155 card decks for designers
Questionnaires and surveys that permit collecting people's thoughts or Vegt et al. (2019) investigates the effects of adding game rules to
opinions about a certain product, process or method brainstorms. Participants filled in a questionnaire about their behav-
ior and engagement
Interviews that are carried out face to face with people who provide Genc et al. (2019) provides recommendations for incorporating tech-
relevant information for the research nological components in fashion designs collecting information from
interviews with experts

Table 8  Categories and examples of works corresponding to the topic of instruments

Topic: Instruments
Category Example

Measurements, referring both to metrics obtained with a physical device Valverde et al. (2019) explore the quality of push-buttons’ haptic
and to ratings obtained from human scores feedback with kinaesthetic parameters measured from force-dis-
placement curves
Audio, video and image recordings that permit saving user and or prod- Gyory et al. (2019) compare individual versus group problem-solving
uct interaction for offline analysis using audio recordings to measure the similarity of the teams’
discourses.
Eye tracking systems for measuring the point of gaze where an inform- Reimlinger et al. (2019) evaluate how engineers benefit from design
ant is looking guidelines by capturing gaze sequences with eye-tracking glasses
Simulation and software tools with performing analytical studies or Zhang and Thomson (2019) model the development of complex prod-
modelling ucts with an agent-based simulation model
Opinions of stakeholders or groups of people supporting or involved in Self (2019) studies communication through design sketches analysing
the research project stakeholders’ interpretations
Opinions of participants in the research projects or product users Menold et al. (2019) explore how prototyping affects user satisfaction
Opinions of expert/designer with a recognized knowledge of the domain Barati et al. (2019) study the understanding of smart materials collect-
ing expert opinions of designers and scientists
Workshops with participation of different agents that jointly discuss and Wlazlak et al. (2019) study visual representations for the communica-
cooperate tion of new products in joint analysis workshops with researchers
and the project managers

taxonomy of research aims (Table 5), approaches (Table 6), Open Access This article is licensed under a Creative Commons Attri-
data collection techniques (Table 7), and instruments bution 4.0 International License, which permits use, sharing, adapta-
tion, distribution and reproduction in any medium or format, as long
(Table 8) in engineering design. as you give appropriate credit to the original author(s) and the source,
provide a link to the Creative Commons licence, and indicate if changes
Funding Open Access funding provided thanks to the CRUE-CSIC were made. The images or other third party material in this article are
agreement with Springer Nature. The authors acknowledge financial included in the article's Creative Commons licence, unless indicated
support from Universidad de Valladolid (PID 2020 21 038), Agencia otherwise in a credit line to the material. If material is not included in
Estatal de Investigación (PID2020-118216RB-I00) and Agencia Estatal the article's Creative Commons licence and your intended use is not
de Investigación (PID2020-112584RB-C32). permitted by statutory regulation or exceeds the permitted use, you will
need to obtain permission directly from the copyright holder. To view a
Data Availability All data generated or analysed during this study are copy of this licence, visit http://​creat​iveco​mmons.​org/​licen​ses/​by/4.​0/.
included in this published article [and its supplementary information
files].

13

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Research in Engineering Design (2023) 34:221–256 253

References Chen B, Hu J, Chen W (2019a) DRE-based semi-automation of the axi-

omatic design transformation: from the functional requirement
to the design parameter. J Eng Des 30:255–287. https://​doi.​org/​
Abi Akle A, Yannou B, Minel S (2019) Information visualisation for
10.​1080/​09544​828.​2019.​16272​96
efficient knowledge discovery and informed decision in design by
Chen R, Liu Y, Fan H et al (2019b) An integrated approach for auto-
shopping. J Eng Des 30:227–253. https://​doi.​org/​10.​1080/​09544​
mated physical architecture generation and multi-criteria evalu-
828.​2019.​16233​83
ation for complex product design. J Eng Des 30:63–101. https://​
Adrion WR (1993) Research methodology in software engineering. In:
doi.​org/​10.​1080/​09544​828.​2018.​15632​87
Summary of the Dagstuhl workshop on future directions in soft-
Cheong H, Butscher A (2019) Physics-based simulation ontology: an
ware engineering” Ed. Tichy, Habermann, and Prechelt, ACM
ontology to support modelling and reuse of data for physics-
software engineering notes, SIGSoft, pp 36–37
based simulation. J Eng Des 30:655–687. https://​doi.​org/​10.​
Aktas BM, Mäkelä M (2019) Negotiation between the maker and mate-
1080/​09544​828.​2019.​16443​01
rial: observations on material interactions in felting studio. Int
Clark JS, Porath S, Thiele J, Jobe M (2020) Action research. New
J Des 13:55–67
Prairie Press, Paris
Alizadeh R, Allen JK, Mistree F (2020) Managing computational com-
Comi A, Jaradat S, Whyte J (2019) Constructing shared professional
plexity using surrogate models: a critical review. Res Eng Des
vision in design work: the role of visual objects and their mate-
31:275–298. https://​doi.​org/​10.​1007/​s00163-​020-​00336-7
rial mediation. Des Stud 64:90–123. https://​doi.​org/​10.​1016/j.​
Anderiesen H, Scherder E, Goossens R et al (2015) Play experiences
destud.​2019.​06.​003
for people with Alzheimer’s disease. Int J Des 9:155–165
Cooper R (2019) Design research—its 50-year transformation. Des
Atkinson P (1988) Ethnomethodology: A Critical Review. Annu Rev
Stud 65:6–17. https://​doi.​org/​10.​1016/j.​destud.​2019.​10.​002
Sociol 14:441–465. https://​d oi.​o rg/​1 0.​1 146/​a nnur​ev.​s o.​1 4.​
Coskun A, Zimmerman J, Erbug C (2015) Promoting sustainability
080188.​002301
through behavior change: a review. Des Stud 41:183–204. https://​
Barati B, Karana E, Hekkert P (2019) Prototyping materials experi-
doi.​org/​10.​1016/j.​destud.​2015.​08.​008
ence: towards a shared understanding of underdeveloped smart
Cranz G (2016) Ethnography for designers. Routledge, London
material composites. Int J Des 13:21–38
Creswell JW (2009) Research design: qualitative, quantitative and
Behera AK, McKay A, Earl CF et al (2019) Sharing design definitions
mixed approaches, 3rd edn
across product life cycles. Res Eng Des 30:339–361. https://​doi.​
Crilly N (2019) Creativity and fixation in the real world: a literature
org/​10.​1007/​s00163-​018-​00306-0
review of case study research. Des Stud 64:154–168. https://​doi.​
Beitz W, Pahl G, Grote K (1996) MRS Bull 21:71. https://​doi.​org/​10.​
org/​10.​1016/j.​destud.​2019.​07.​002
1557/​S0883​76940​00357​76
Cuff D (1992) Architecture: the story of practice. MIT Press, London
Belkadi F, Le DuigouDall’Olio JL et al (2019) Knowledge-based plat-
Daalhuizen J, Timmer R, Van Der Welie M, Gardien P (2019) An
form for traceability and simulation monitoring applied to design
architecture of design doing: a framework for capturing the ever-
of experiments process: an open source architecture. J Eng Des
evolving practice of design to drive organizational learning. Int
30:311–335. https://​doi.​org/​10.​1080/​09544​828.​2019.​16424​63
J Des 13:37–52
Benavides EM, Lara-Rapp O (2019) Ideal output for a robust concep-
De Leeuw ED (2008) Choosing the method of data collection
tual design process. J Eng Des 30:103–154. https://​doi.​org/​10.​
De Lessio MP, Wynn DC, Clarkson PJ (2019) Modelling the planning
1080/​09544​828.​2019.​15985​52
system in design and development. Res Eng Des 30:227–249.
Bergstrom JR, Schall A (2014) Eye tracking in user experience design.
https://​doi.​org/​10.​1007/​s00163-​017-​0272-5
Elsevier, Amsterdam
Diggle PJ, Chetwynd AG, Chetwynd A (2011) Statistics and scientific
Bladek M (2014) DORA: San Francisco declaration on research assess-
method: an introduction for students and researchers. Oxford
ment (May 2013). Coll Res Libr News 75:191–196
University Press, Oxford
Blessing LTM, Chakrabarti A (2009) DRM: a design reseach method-
Dorst K (2004) On the problem of design problems—problem solving
ology. Springer, Berlin
and design expertise. J Des Res 4:185–196. https://​doi.​org/​10.​
Bogle D (2018) 100 years of the PhD in the UK. In: Proceedings of
1504/​JDR.​2004.​009841
vitae researcher development international conference 2018, p 12
Eikeland O (2006) The validity of action research—validity in action
Bonvoisin J, Halstenberg F, Buchert T, Stark R (2016) A systematic
research. In: Aagaard Nielsen K, Svensson L (eds) Action
literature review on modular product design. J Eng Des 27:488–
research and interactive research. Shaker Publishing, Maastricht,
514. https://​doi.​org/​10.​1080/​09544​828.​2016.​11664​82
pp 193–240
Boussuge F, Tierney CM, Vilmart H et al (2019) Capturing simulation
Feijs L, Toeters M (2018) Cellular automata-based generative design
intent in an ontology: CAD and CAE integration application. J
of Pied-de-poule patterns using emergent behavior: case study of
Eng Des 30:688–725. https://​doi.​org/​10.​1080/​09544​828.​2019.​
how fashion pieces can help to understand modern complexity.
16308​06
Int J Des 12:127–144
Boztepe S (2007) User value: competing theories and models. Int J
Ferguson SM, Olewnik AT, Cormier P (2014) A review of mass
Des 1:55–63
customization across marketing, engineering and distribution
Bresciani S (2019) Visual design thinking: a collaborative dimensions
domains toward development of a process framework. Res Eng
framework to profile visualisations. Des Stud 63:92–124. https://​
Des 25:11–30. https://​doi.​org/​10.​1007/​s00163-​013-​0162-4
doi.​org/​10.​1016/j.​destud.​2019.​04.​001
Fink AS (2000) The role of the researcher in the qualitative research
Brewer MB, Crano WD (2014) Research design and issues of validity.
process. A potential barrier to archiving qualitative data. In:
In: Reis HT, Judd CM (eds) Handbook of research methods in
Forum Qualitative Sozialforschung/Forum: Qualitative Social
social and personality psychology, 2nd edn. Cambridge Univer-
Research
sity Press, New York, NY, USA, pp 11–26
Franceschini F, Maisano D (2019) Design decisions: concordance of
Candi M, Gemser G (2010) An agenda for research on the relationships
designers and effects of the Arrow’s theorem on the collective
between industrial design and performance. Int J Des 4:67–77
preference ranking. Res Eng Des 30:425–434. https://d​ oi.o​ rg/1​ 0.​
Cantamessa M (2003) An empirical perspective upon design research.
1007/​s00163-​019-​00313-9
J Eng Des 14:1–15. https://​doi.​org/​10.​1080/​09544​82031​00007​
8126

13

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

254 Research in Engineering Design (2023) 34:221–256

Garcia JJ, Pettersen SS, Rehn CF et al (2019) Overspecified vessel Kennedy-Clark S (2013) Research by design: design-based research
design solutions in multi-stakeholder design problems. Res Eng and the higher degree research student. J Learn Des 6:26–32
Des 30:473–487. https://​doi.​org/​10.​1007/​s00163-​019-​00319-3 Khalaj J, Pedgley O (2019) A semantic discontinuity detection (SDD)
Genç Ç, Buruk OT, Yılmaz Sİ et al (2018) Exploring computational method for comparing designers’ product expressions with users’
materials for fashion: recommendations for designing fashion- product impressions. Des Stud 62:36–67. https://​doi.​org/​10.​
able wearables. Int J Des 12:1–19 1016/j.​destud.​2019.​02.​002
Glass RL (1995) A structure-based critique of contemporary comput- King N, Horrocks C, Brooks J (2019) Interviews in qualitative research,
ing research. J Syst Softw 28:3–7. https://d​ oi.​org/​10.​1016/​0164-​ 2nd edn. Sage, London
1212(94)​00077-Z Kitchenham B, Pearl Brereton O, Budgen D et al (2009) Systematic
Goodwin C (2000) Practices of seeing visual analysis: an ethnometh- literature reviews in software engineering—a systematic litera-
odological approach. SAGE Publications Ltd, London ture review. Inf Softw Technol 51:7–15. https://d​ oi.o​ rg/1​ 0.1​ 016/j.​
Goucher-Lambert K, Cagan J (2019) Crowdsourcing inspiration: using infsof.​2008.​09.​009
crowd generated inspirational stimuli to support designer idea- Koskinen I, Zimmerman J, Binder T et al (2011) Design research
tion. Des Stud 61:1–29. https://​doi.​org/​10.​1016/j.​destud.​2019.​ through practice: from the lab, field, and showroom. Elsevier,
01.​001 Amsterdam
Graeff E, Maranzana N, Aoussat A (2019) Biomimetics, where are the Kothari CR (2004) Research methodology: methods and techniques.
biologists? J Eng Des 30:289–310. https://d​ oi.o​ rg/1​ 0.1​ 080/0​ 9544​ New Age International, New Delhi
828.​2019.​16424​62 Lapan SD, Quartaroli MT, Riemer FJ (eds) (2012) Qualitative research:
Gralla EL, Herrmann JW, Morency M (2019) Design problem an introduction to methods and designs. Jossey-Bass/Wiley, New
decomposition: an empirical study of small teams of facility York
designers. Res Eng Des 30:161–185. https://​doi.​org/​10.​1007/​ Li Q, Luximon Y (2018) Understanding older adults’ post-adoption
s00163-​018-​0300-0 usage behavior and perceptions of mobile technology. Int J Des
Gyory JT, Cagan J, Kotovsky K (2019) Are you better off alone? Miti- 12:93–110
gating the underperformance of engineering teams during con- Li Y, Roy U, Saltz JS (2019a) Towards an integrated process model
ceptual design through adaptive process management. Res Eng for new product development with data-driven features
Des 30:85–102. https://​doi.​org/​10.​1007/​s00163-​018-​00303-3 (NPD3). Res Eng Des 30:271–289. https://​doi.​org/​10.​1007/​
Hagedorn TJ, Smith B, Krishnamurty S, Grosse I (2019) Interoper- s00163-​019-​00308-6
ability of disparate engineering domain ontologies using basic Li Y, Shieh M-D, Yang C-C (2019b) A posterior preference articu-
formal ontology. J Eng Des 30:625–654. https://d​ oi.o​ rg/1​ 0.1​ 080/​ lation approach to Kansei engineering system for product
09544​828.​2019.​16308​05 form design. Res Eng Des 30:3–19. https://​doi.​org/​10.​1007/​
Han J, Forbes H, Schaefer D (2021) An exploration of how creativity, s00163-​018-​0297-4
functionality, and aesthetics are related in design. Res Eng Des Lloyd P (2019) You make it and you try it out: Seeds of design disci-
32:289–307. https://​doi.​org/​10.​1007/​s00163-​021-​00366-9 pline futures. Des Stud 65:167–181. https://​doi.​org/​10.​1016/j.​
Han X, Li R, Wang J et al (2020) A systematic literature review of destud.​2019.​10.​008
product platform design under uncertainty. J Eng Des 31:266– Luck R (2019) Design research, architectural research, architectural
296. https://​doi.​org/​10.​1080/​09544​828.​2019.​16990​36 design research: an argument on disciplinarity and identity. Des
Hanrahan BV, Yuan CW, Rosson MB et al (2019) Materializing inter- Stud 65:152–166. https://​doi.​org/​10.​1016/j.​destud.​2019.​11.​001
actions with paper prototyping: a case study of designing social, MacQueen KM, Guest G (2008) An introduction to team-based quali-
collaborative systems with older adults. Des Stud 64:1–26. tative research. In: Guest G, MacQueen KM (eds) Handbook
https://​doi.​org/​10.​1016/j.​destud.​2019.​06.​002 for team-based qualitative research. Altamira Press, Lanham, pp
Hatchuel A (2001) Towards design theory and expandable rationality: 3–19
the unfinished program of Herbert Simon. J Manag Gov 5:260– Martinec T, Škec S, Horvat N, Štorga M (2019) A state-transition
273. https://​doi.​org/​10.​1023/A:​10140​44305​704 model of team conceptual design activity. Res Eng Des 30:103–
Herr CM (2015) Action research as a research method in architecture 132. https://​doi.​org/​10.​1007/​s00163-​018-​00305-1
and design. In: Proceedings of the 59th annual meeting of the Mathias D, Snider C, Hicks B, Ranscombe C (2019) Accelerating prod-
ISSS-2015 Berlin, Germany uct prototyping through hybrid methods: coupling 3D printing
Hiebl MRW (2021) Sample selection in systematic literature reviews and LEGO. Des Stud 62:68–99. https://d​ oi.o​ rg/1​ 0.1​ 016/j.d​ estud.​
of management research. Organ Res Methods. https://d​ oi.o​ rg/1​ 0.​ 2019.​04.​003
1177/​10944​28120​986851 McDonald JK, Michela E (2019) The design critique and the moral
Hobye M, Ranten MF (2019) Behavioral complexity as a computa- goods of studio pedagogy. Des Stud 62:1–35. https://​doi.​org/​10.​
tional material strategy. Int J Des 13:39–53 1016/j.​destud.​2019.​02.​001
Hyysalo S, Hyysalo V, Hakkarainen L (2019a) The work of democra- McKinnon H, Sade G (2019) Exploring the home environment: fusing
tized design in setting-up a hosted citizen-designer community. rubbish and design to encourage participant agency and self-
Int J Des 13:69–82 reflection. Des Stud 63:155–180. https://​doi.​org/​10.​1016/j.​des-
Hyysalo S, Marttila T, Perikangas S, Auvinen K (2019b) Codesign tud.​2019.​05.​001
for transitions governance: a mid-range pathway creation toolset Menold J, Simpson TW, Jablokow K (2019) The prototype for X frame-
for accelerating sociotechnical change. Des Stud 63:181–203. work: exploring the effects of a structured prototyping framework
https://​doi.​org/​10.​1016/j.​destud.​2019.​05.​002 on functional prototypes. Res Eng Des 30:187–201. https://​doi.​
Jagtap S (2019) Design and poverty: a review of contexts, roles of poor org/​10.​1007/​s00163-​018-​0289-4
people, and methods. Res Eng Des 30:41–62. https://​doi.​org/​10.​ Miles MB, Huberman AM, Saldaña J (2020) Qualitative data analysis:
1007/​s00163-​018-​0294-7 a methods sourcebook, 4th edn. SAGE Publications Inc., London
Joost G, Bredies K, Christensen M et al (2016) Design as research: Morkos B, Joshi S, Summers JD (2019) Investigating the impact of
Positions, arguments, perspectives. Birkhäuser, Basel requirements elicitation and evolution on course performance in
Jørgensen U (2001) Grounded theory: methodology and theory con- a pre-capstone design course. J Eng Des 30:155–179
struction. Int Encycl Soc Behav Sci 1:6396–6399

13

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Research in Engineering Design (2023) 34:221–256 255

Motta-Filho MA (2021) Brand experience manual: bridging the gap Saldaña J (2021) The coding manual for qualitative researchers, 4th
between brand strategy and customer experience. Rev Manag edn. SAGE Publications Ltd, Thousand Oaks
Sci 15:1173–1204 Saliminamin S, Becattini N, Cascini G (2019) Sources of creativity
Ohnemus KR, Biers DW (1993) Retrospective versus concurrent think- stimulation for designing the next generation of technical sys-
ing-out-loud in usability testing. In: Proceedings of the human tems: correlations with R&D designers’ performance. Res Eng
factors and ergonomics society annual meeting. SAGE Publica- Des 30:133–153. https://​doi.​org/​10.​1007/​s00163-​018-​0299-2
tions Sage CA, Los Angeles, pp 1127–1131 Santolaya JL, Lacasa E, Biedermann A, Muñoz N (2019) A practical
Ozer M, Cebeci U (2019) Affective design using big data within the methodology to project the design of more sustainable prod-
context of online shopping. J Eng Des 30:368–384. https://​doi.​ ucts in the production stage. Res Eng Des 30:539–558. https://​
org/​10.​1080/​09544​828.​2019.​16568​03 doi.​org/​10.​1007/​s00163-​019-​00320-w
Pakkanen J, Juuti T, Lehtonen T (2019) Identifying and addressing Saravanan A, Jerald J (2019) Ontological model-based optimal
challenges in the engineering design of modular systems—case determination of geometric tolerances in an assembly using
studies in the manufacturing industry. J Eng Des 30:32–61. the hybridised neural network and genetic algorithm. J Eng Des
https://​doi.​org/​10.​1080/​09544​828.​2018.​15527​79 30:180–198. https://​doi.​org/​10.​1080/​09544​828.​2019.​16055​85
Paluck EL, Cialdini RB (2014) Field research methods. In: Reis HT, Schröppel T, Miehling J, Wartzack S (2021) The role of product
Judd CM (eds) Handbook of research methods in social and per- development in the battle against product-related stigma—a
sonality psychology, 2nd edn. Cambridge University Press, New literature review. J Eng Des 32:247–270. https://​doi.​org/​10.​
York, NY, USA, pp 81–97 1080/​09544​828.​2021.​18790​31
Park-Lee S, Person O (2018) Briefng beyond documentation: an inter- Self JA (2019) Communication through design sketches: implica-
view study on industrial design consulting practices in Finland. tions for stakeholder interpretation during concept design. Des
Int J Des 12:73–91 Stud 63:1–36. https://​doi.​org/​10.​1016/j.​destud.​2019.​02.​003
Park D, Han J, Childs PRN (2021) 266 Fuzzy front-end studies: current Selvefors A, Marx C, Karlsson MAIC, Rahe U (2018) (How) can
state and future directions for new product development. Res Eng appliances be designed to support less energy-intensive use?
Des 32:377–409. https://​doi.​org/​10.​1007/​s00163-​021-​00365-w Insights from a field study on kitchen appliances. Int J Des
Pedgley O, Şener B, Lilley D, Bridgens B (2018) Embracing material 12:35–55
surface imperfections in product design. Int J Des 12:21–33 Simon HA (1996) The science of design: creating the artificial
Pérez-Gómez A (1999) Hermeneutics as discourse in design. Des Solomon M (2007) Social empiricism. MIT Press, Cambridge
Issues 15:71–79 Stigliano A (1989) Hermeneutical practice. Saybrook Rev 7:47–67
Pessôa MVP, Becker JMJ (2020) Smart design engineering: a literature Stringer ET (2008) Action research in education. Pearson Prentice
review of the impact of the 4th industrial revolution on product Hall, Upper Saddle River
design and development. Res Eng Des 31:175–195 Subrahmanian E, Reich Y, Krishnan S (2020) We are not users: dia-
Petreca B, Saito C, Baurley S et al (2019) Radically relational tools: a logues, diversity, and design. MIT Press, Cambridge
design framework to explore materials through embodied pro- Sung E, Kelley TR, Han J (2019) Influence of sketching instruc-
cesses. Int J Des 13:7–20 tion on elementary students’ design cognition: a study of three
Piccolo SA, Maier AM, Lehmann S, McMahon CA (2019) Iterations as sketching approaches. J Eng Des 30:199–226. https://​doi.​org/​
the result of social and technical factors: empirical evidence from 10.​1080/​09544​828.​2019.​16174​13
a large-scale design project. Res Eng Des 30:251–270. https://​ Swann C (2002) Action research and the practice of design. Des
doi.​org/​10.​1007/​s00163-​018-​0301-z Issues 18:49–61
Radhakrishna RB (2007) Tips for developing and testing question- Tahera K, Wynn DC, Earl C, Eckert CM (2019) Testing in the incre-
naires/instruments. J Ext 45:1TOT2 mental design and development of complex products. Res Eng
Randolph MF (2003) Science and empiricism in pile foundation Des 30:291–316. https://​doi.​org/​10.​1007/​s00163-​018-​0295-6
design. Géotechnique 53:847–875 Takahashi I, Oki M, Bourreau B et al (2018) An empathic design
Rapley T, Rees G (2018) Collecting documents as data. In: Flick U approach to an augmented gymnasium in a special needs
(ed) The SAGE handbook of qualitative data collection. Sage, school setting. Int J Des 12:111–125
London, pp 378–391 Tempczyk H (1986) A survey of research and studies on design.
Redström J (2017) Making design theory. MIT Press, London Des Stud 7:199–215. https://​doi.​org/​10.​1016/​0142-​694X(86)​
Reich Y, Subrahmanian E (2021) Mapping and enhancing design 90037-2
studies with psI meta-theoretic design framework. Proc Des Tenopir C, King DW (2014) The growth of journals publishing. In:
Soc 1:2007–2016. https://​doi.​org/​10.​1017/​pds.​2021.​462 The future of the academic journal. Elsevier, Amsterdam, pp
Reich Y, Subrahmanian E (2020) The PSI framework and theory of 159–178
design. IEEE Trans Eng Manag. https://​doi.​org/​10.​1109/​TEM.​ Thiese MS (2014) Observational and interventional study design
2020.​29732​38 types; an overview. Biochem Medica 24:199–210
Reimlinger B, Lohmeyer Q, Moryson R, Meboldt M (2019) A com- Tsai WC, Van Den Hoven E (2018) Memory probes: exploring ret-
parison of how novice and experienced design engineers ben- rospective user experience through traces of use on cherished
efit from design guidelines. Des Stud 63:204–223. https://​doi.​ objects. Int J Des 12:57–72
org/​10.​1016/j.​destud.​2019.​04.​004 Valverde N, Ribeiro AMR, Henriques E, Fontul M (2019) An engi-
Renzi C, Leali F, Di Angelo L (2017) A review on decision-making neering perspective on the quality of the automotive push-but-
methods in engineering design for the automotive industry. J tons’ haptic feedback in optimal and suboptimal interactions. J
Eng Des 28:118–143. https://​doi.​org/​10.​1080/​09544​828.​2016.​ Eng Des 30:336–367. https://​doi.​org/​10.​1080/​09544​828.​2019.​
12747​20 16568​02
Roesler A, Grigg EB, Martin LD et al (2019) Practice-centered Van der Bijl-Brouwer M, Dorst K (2017) Advancing the strategic
design of an anesthesia medication template to reduce medica- impact of human-centred design. Des Stud 53:1–23. https://​
tion handling errors in the operating room. Int J Des 13:53–68 doi.​org/​10.​1016/j.​destud.​2017.​06.​003
Roy R, Warren JP (2019) Card-based design tools: a review and Van der Linden V, Dong H, Heylighen A (2019a) Populating archi-
analysis of 155 card decks for designers and designing. Des tectural design: introducing scenario-based design in residen-
Stud 63:125–154. https://​doi.​org/​10.​1016/j.​destud.​2019.​04.​002 tial care projects. Int J Des 13:21–36

13

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

256 Research in Engineering Design (2023) 34:221–256

Van der Linden V, Dong H, Heylighen A (2019b) Tracing architects’ Wilson A (2015) A guide to phenomenological research. Nurs Stand
fragile knowing about users in the socio-material environment 29:38
of design practice. Des Stud 63:65–91. https://​d oi.​o rg/​1 0.​ Wlazlak P, Eriksson Y, Johansson G, Ahlin P (2019) Visual rep-
1016/j.​destud.​2019.​02.​004 resentations for communication in geographically distributed
Van Kuijk J, Daalhuizen J, Christiaans H (2019) Drivers of usability new product development projects. J Eng Des 30:385–403.
in product design practice: induction of a framework through https://​doi.​org/​10.​1080/​09544​828.​2019.​16613​62
a case study of three product development projects. Des Stud Wood AE, Mattson CA (2019) Quantifying the effects of various
60:139–179. https://​doi.​org/​10.​1016/j.​destud.​2018.​06.​002 factors on the utility of design ethnography in the develop-
Vasantha GVA, Roy R, Lelah A, Brissaud D (2012) A review of ing world. Res Eng Des 30:317–338. https://​doi.​org/​10.​1007/​
product–service systems design methodologies. J Eng Des s00163-​018-​00304-2
23:635–659. https://​doi.​org/​10.​1080/​09544​828.​2011.​639712 Yang S, Santoro F, Sulthan MA, Zhao YF (2019) A numerical-based
Vaughan L (2017) Practice-based design research. Bloomsbury Pub- part consolidation candidate detection approach with modulari-
lishing, London zation considerations. Res Eng Des 30:63–83. https://​doi.​org/​
Vegt N, Visch V, Vermeeren A et al (2019) Balancing game rules 10.​1007/​s00163-​018-​0298-3
for improving creative output of group brainstorms. Int J Des Yin RK (2009) Case study research: design and methods, 4th edn.
13:1–19 Sage Publications, Thousand Oaks
Wang R, Nellippallil AB, Wang G et al (2019) Ontology-based Zhang X, Thomson V (2019) Modelling the development of com-
uncertainty management approach in designing of robust plex products using a knowledge perspective. Res Eng Des
decision workflows. J Eng Des 30:726–757. https://​doi.​org/​ 30:203–226. https://​doi.​org/​10.​1007/​s00163-​017-​0274-3
10.​1080/​09544​828.​2019.​16689​18
Wasson C (2000) Ethnography in the field of design. Hum Organ
377–388

13

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Terms and Conditions
Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center GmbH (“Springer Nature”).
Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers and authorised users (“Users”), for small-
scale personal, non-commercial use provided that all copyright, trade and service marks and other proprietary notices are maintained. By
accessing, sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of use (“Terms”). For these
purposes, Springer Nature considers academic use (by researchers and students) to be non-commercial.
These Terms are supplementary and will apply in addition to any applicable website terms and conditions, a relevant site licence or a personal
subscription. These Terms will prevail over any conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription
(to the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of the Creative Commons license used will
apply.
We collect and use personal data to provide access to the Springer Nature journal content. We may also use these personal data internally within
ResearchGate and Springer Nature and as agreed share it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not
otherwise disclose your personal data outside the ResearchGate or the Springer Nature group of companies unless we have your permission as
detailed in the Privacy Policy.
While Users may use the Springer Nature journal content for small scale, personal non-commercial use, it is important to note that Users may
not:

1. use such content for the purpose of providing other users with access on a regular or large scale basis or as a means to circumvent access
control;
2. use such content where to do so would be considered a criminal or statutory offence in any jurisdiction, or gives rise to civil liability, or is
otherwise unlawful;
3. falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association unless explicitly agreed to by Springer Nature in
writing;
4. use bots or other automated methods to access the content or redirect messages
5. override any security feature or exclusionary protocol; or
6. share the content in order to create substitute for Springer Nature products or services or a systematic database of Springer Nature journal
content.
In line with the restriction against commercial use, Springer Nature does not permit the creation of a product or service that creates revenue,
royalties, rent or income from our content or its inclusion as part of a paid for service or for other commercial gain. Springer Nature journal
content cannot be used for inter-library loans and librarians may not upload Springer Nature journal content on a large scale into their, or any
other, institutional repository.
These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not obligated to publish any information or
content on this website and may remove it or features or functionality at our sole discretion, at any time with or without notice. Springer Nature
may revoke this licence to you at any time and remove access to any copies of the Springer Nature journal content which have been saved.
To the fullest extent permitted by law, Springer Nature makes no warranties, representations or guarantees to Users, either express or implied
with respect to the Springer nature journal content and all parties disclaim and waive any implied warranties or warranties imposed by law,
including merchantability or fitness for any particular purpose.
Please note that these rights do not automatically extend to content, data or other material published by Springer Nature that may be licensed
from third parties.
If you would like to use or distribute our Springer Nature journal content to a wider audience or on a regular basis or in any other manner not
expressly permitted by these Terms, please contact Springer Nature at

onlineservice@springernature.com