Public Relations Journal

Vol. 11 Issue 3 (February 2018)

© 2018 Institute for Public Relations

A Delphi Study to Identify Standards for Internal Communication

Julie O’Neil, Ph.D.

Texas Christian University, Professor
[Link]@[Link] |

Michele Ewing, M.A. & APR

Kent State University, Associate Professor

Stacey Smith, APR, Fellow PRSA

Jackson, Jackson & Wagner, Senior Counsel and Partner

Sean Williams, M.A.

True Digital Communications, Vice President

Abstract

In recent years, significant advancement has been made in establishing standards for measuring
external public relations efforts. Little has been done, however, to standardize measurement for
those responsible for internal communication. Employees are critical to the success of any
organization, and the effectiveness of communication with them is of paramount importance.
Seeking to rectify this gap, the Institute for Public Relations and the Commission on Research,
Measurement, and Evaluation created an international 11-member task force comprising
academics and practitioners in 2015 to develop internal communication measurement standards.
The task force identified and tentatively defined possible standards for internal communication.

The researchers of this study extended the global task force’s initial work by conducting a Delphi
study in 2016 with a purposive sample of 22 internal communication thought leaders to
determine if a wider audience of internal communication practitioners agreed with the task
force’s recommended standards and definitions. This research describes the findings of the
Delphi study and introduces and defines measurement standards that internal communication
practitioners can use to create more effective communication plans and measure the value
communication bring to their organizations. The proposed standards also supersede the current
"engagement" catchall goal that tends to be amorphous and difficult to address. In the future, the
researchers plan to test the reliability and validity of the internal communication standards
developed in this study, so that practitioners can measure in a consistent and comparable
manner—the ultimate goal of standardization.

1
Public Relations Journal
Vol. 11 Issue 3 (February 2018)
© 2018 Institute for Public Relations

Introduction
Effective internal communication has been repeatedly linked to financial outcomes
(Dortok, 2006; Ehling, White, & Grunig, 1992; Grossman, 2013; Harter, Schmidt, & Hayes,
2002; Meng & Berger, 2012; Towers Watson, 2013). A 2007-2008 study by Watson Wyatt
Worldwide (now Towers Watson), an international business research firm, indicated that high-
effective organizations are much more likely to measure and evaluate internal communication
compared to low-effective organizations (as cited in Meng & Berger, 2012). Ironically, only
about 50% of internal communication practitioners have a formal approach to measuring and
evaluating their communication initiatives (Meng & Berger). In Meng and Berger’s survey and
interview research with international public relations practitioners, participants cited barriers to
internal communication measurement, including a lack of money and staff, difficulty
determining a direct link between communication initiatives and business results, and time
constraints.
Not only do some public relations practitioners have difficulty knowing how to measure
and evaluate internal communication, some also appear to do so in different ways (Mendez,
Casadesus, & Gimenez, 2013; Meng & Pan, 2012; Ruck, 2015; Ruck & Welch, 2012). The lack
of a standardized approach to measuring internal communication ultimately adds to potential
inefficiency, because practitioners and their organizational leaders do not have a shared
vocabulary to compare and contrast results.
To remedy these problems, the Institute for Public Relations and the Commission on
Research, Measurement, and Evaluation created an international 11-member task force
comprising academics and practitioners to promulgate the development of internal
communication measurement standards. A standard is defined as a published specification in the
public domain that provides a common language for comparison purposes (Institute for PR,
2013). The task force identified and tentatively defined possible standards for internal
communication measurement in 2015.
The researchers of this study extended the international task force’s initial work by
conducting a Delphi study in 2016 with a purposive sample of internal communication thought
leaders to determine if a wider audience of internal communication practitioners agreed with the
task force’s recommended standards and definitions. This paper describes the findings of the
Delphi study and introduces and defines measurement standards that internal communication
practitioners can use to create more effective communication plans and measure the value
communication bring to their organizations. Moving forward, the researchers plan to test the
validity and reliability of the proposed internal communication standards, so that practitioners
can measure internal communication in a consistent and comparable manner.

Literature Review
Challenges of Measuring Internal Communication
Scholars have emphasized the importance of measurement and evaluation of internal
communication since the 1980s, but have experienced difficulties in choosing appropriate
metrics to evaluate all aspects of an organization’s internal communication (Meng & Berger,
2012). In Ruck and Welch’s (2012) review of 12 different internal communication assessment
studies, they found an overreliance on management-centric rather than employee-centric
assessments. Mendez, Casadesus, and Gimenez (2013) suggested problems in developing a
single measurement model that could account for all factors affecting any type of organization.

2
Public Relations Journal
Vol. 11 Issue 3 (February 2018)
© 2018 Institute for Public Relations

They also expressed concern about the number of items and excessive time requirements
necessary to complete such a model. Many public relations practitioners also have expressed
frustration in knowing how to measure internal communication efforts (Davis, 2015; Meng &
Berger, 2012; Meng & Pan, 2012; Richardson, 2015). Limitations at the individual and
organizational level create a range of challenges for internal communication measurement. For
example, an employee is often overloaded with information from a wide array of channels
throughout the day, making it difficult to ensure critical information is received, processed,
understood, and retained in a way that is useful (Davis, 2015; Zwijze-Koning, 2016).
Internal communication measurement and evaluation approaches can include both
financial indicators, such as ROI, and non-financial indicators, such as trust, satisfaction, and
advocacy (Meng & Pan, 2012). Financial indicators tend to be short-term, whereas non-financial
indicators are typically long-term (Meng & Pan). Research conducted with senior corporate
communication executives indicated that measurement approaches typically focus on increased
awareness and understanding, employee engagement, and increased productivity and job
performance (Meng & Berger, 2012). “However, respondents admitted that there is no single
route that can be applied to all situations and clients” (Meng & Berger, p. 349).
In a 2011 survey, 42% of public relations practitioners agreed that common terms and
definitions for the measurement of public relations in general “do not exist at all” (Michaelson &
Stacks, p. 2). Not only does confusion exist regarding how to measure basic public relations
activity in general, there is variability in proposed measurement and evaluation frameworks. For
example, Lindenmann (2003) purported an output, outtakes, and outcomes orientation.
Michaelson and Stacks (2011) recommended a communication lifecycle framework consisting of
awareness, interest, desire, and action indicators. The International Association for the
Measurement and Evaluation of Communication (AMEC, 2014) suggested an outputs (public
relations activity), outcomes (intermediary effect), and target audience effect approach. Laskin
(2016) advocated an output, outreach, outcome, outgrowth, and outperform orientation.

Value of Measurement Standards

In light of the variability of measurement terminology, terms, and models, the Institute
for Public Relations has prioritized the development and testing of standards in recent years.
Standards are defined as an idea or thing used as a measure, norm, or model in comparative
evaluations (Michaelson & Stacks, 2011, p. 4). Using a common language for measurement,
standards are advantageous for management, public relations practitioners, clients, agencies, and
researchers. This common language enables comparisons in pre- and post- campaigns; among
campaigns spanning brands, business units, and geographies; among communication efforts used
both internally and externally to an organization; among approaches used by multiple parties and
providers; and among industries. Most important, standards enable “comparison, increase
reliability, promote efficiency and bring more credibility to public relations” (“About Public
Relations,” 2016).
Standards are not synonymous with best practices. Standards define what needs to be
measured whereas a best practice indicates how to best meet the objective of the standard
(Michaelson & Stacks, 2011). A best practice is a “technique, method or practice that is more
effective than others in reaching an established goal” (Michaelson & Macleod, 2007, p. 3).

3
Public Relations Journal
Vol. 11 Issue 3 (February 2018)
© 2018 Institute for Public Relations

A Vision for Public Relations Standards

In 2012, the Coalition for Research Standards was created to develop a broad platform of
standards within the public relations industry. The five charter member organizations were the
Council of Public Relations Firms, the Global Alliance for Public Relations and Communication
Management, the Institute for Public Relations, the International Association for Measurement
and Evaluation of Communication, and the Public Relations Society of America. The Coalition
promoted its vision as “Excellence in public relations enabled by excellence in research,
measurement, and evaluation” (“About Public Relations,” 2016, para. 2). Standards represent a
critical piece of that envisioned excellence.
The Coalition for Research Standards supports the standardization process recommended
by the International Organization for Standardization (ISO). Specifically, the Coalition supports
the ISO principles that (a) standards are voluntary and based upon a market need; (b) standards
are created by global experts in the marketplace; and (c) standards are developed via a consensus
among a wide range of audiences, including organizations, agencies, research consultants and
measurement firms, and academics (“About Public Relations,” 2016).
To date, 85 organizations have pledged their support for public relations standards in
general (“About Public Relations,” 2016). Different types of organizations have pledged support
for the standards, including companies, universities, public relations agencies, research firms,
consultancies, trade organizations, among others (see “Pledge Support to Public Relations
Standards” for complete list). Standards have been developed for traditional media measurement
(Eisenmann, O’Neil, & Geddes, 2016), ethics (Bowen & Stacks, 2013), and the communication
lifecycle (Michaelson & Stacks, 2011) and have been proposed for social media measurement
and return on investment (“About Public Relations,” 2016). This paper seeks to expand these
efforts by developing measurement standards for internal communication. Specifically, this
research project seeks to answer the following research questions:

RQ1: Which of the proposed internal communication standards are currently measured and
valued by practitioners?
RQ2: Do internal communication practitioners agree with the proposed list of standards?
RQ3: How should the internal communication standards be defined?

Method
The first step of the international Internal Communication Standards task force was to
review professional and academic literature on internal communication to identify possible
standards and current measurement methods. Following that review, the task force met multiple
times over the course of a year to discuss the findings of the literature review and to decide
which concepts to include as possible standards. In 2015 and 2016, leaders of the task force (the
researchers of this study) presented the potential standards to more than 150 communication
practitioners and academics at three international public relations conferences.
Researchers then refined the proposed standards and definitions and conducted Round 1
and Round 2 of a Delphi study with a purposive sample of internal communication thought
leaders. A Delphi study consists of structured questions answered by a panel of experts in at least
two rounds, working toward the goal of obtaining a group consensus (Wakefield & Watson,
2014). The method is particularly appropriate to use regarding an issue where there is a “lack of
agreement or incomplete state of knowledge” (Delbecq, Van de Ven, & Gustafson, 1975, p. 5).

4
Public Relations Journal
Vol. 11 Issue 3 (February 2018)
© 2018 Institute for Public Relations

The goal was to determine the level of agreement with the task force’s recommended standards
and definitions. Round 1 of a Delphi study elicits feedback from participants though loosely
structured and open-ended questions followed by a summary of findings and structured
discussion among experts in subsequent rounds (Watson, 2008). The Delphi method is also
advantageous in its ability to obtain feedback from myriad diverse participants without the
inherent bias and groupthink that is often present in group interviews and focus groups (Boyton,
2006; Wakefield & Watson). Finally, unlike a survey that may elicit quick and gut responses
from respondents, a Delphi study affords participants the opportunity to reflect and spend time
thinking, in this case about internal communication evaluative practices.
Participants for this Delphi study were identified based upon the researchers’ professional
contacts and snowball sampling. Criterion for participation was a minimum of ten years of
experience in the communication industry, including responsibility for internal communication
for at least five years. Recruitment emails were sent to 84 participants in both Rounds 1 and 2,
although two people declined because they claimed they were not qualified to participate.
Recruitment emails were sent to all 84 participants in both rounds in order to secure as robust a
sample as possible. A total of 22 out of 82 people completed the survey in both rounds, for a
response rate of 27%. While the sample might be considered small compared to other studies,
given the goal of reaching specifically thought leaders in internal communication, the responses
reflect a comprehensive perspective of experts in this specific area of internal communication.
All respondents who agreed to participate were from North America. Respondents have
many years of professional communication experience; nearly half of respondents have between
20 and 30 years of experience and roughly one quarter of participants have more than that.
Roughly three quarters of respondents work for an organization with 5,000 or more employees.
Industries represented include finance/insurance/real estate, healthcare, manufacturing,
services/consulting, mining, and transport, among others.
Round 1 asked participants to indicate their level of agreement with the recommended
academic and practitioner definitions of 21 proposed standards (see Table 1) and to provide edits
and feedback. The standards were grouped as (a) outtakes, whether employees received, paid
attention to, comprehended, or retained particular messages; (b) outcomes, evidence of changes
to or reinforcement of opinions, attitudes, or behaviors; and (c) organizational impact, if and how
internal communication has influenced organizational performance. The study did not focus on
outputs, like some measurement models, such as Lindenmann’s (2003) or AMEC’s (2014),
because the researchers presumed that there would be very little disagreement on basic public
relations activity, such as number of stories read, number of click-throughs, number of
employees in attendance, and so forth. Participants were next asked a series of questions about
their use of the standards, whether they recommended additional standards, and which standards
they viewed as most important and why.
Following Round 1, the researchers also obtained feedback on the proposed standards list
and definitions of the standards from internal communication managers attending a highly
regarded public relations conference. Based upon feedback obtained from Round 1 and
conference participants, the researchers amended the standard list and definitions for Round 2.
Researchers sent a summary of results and the amended version to all 82 participants in
November 2016. For Round 2, participants were instructed to indicate whether they agreed or
disagreed with the amended list of 22 standards and definitions and to offer additional edits on
the proposed definitions.

5
Public Relations Journal
Vol. 11 Issue 3 (February 2018)
© 2018 Institute for Public Relations

Round One Results

Outtakes
As indicated by Table 1, participants currently measure the majority of the proposed
outtake standards of awareness, understanding, knowledge, and retention of information.
Respondents averred that although awareness is the easiest and most commonly measured
standard, it is the least useful. One respondent described awareness as a superficial measure and
“that it doesn't equate to an action - behavior, discussion, inquiry.” Respondents further
explained the difficulty in measuring the more meaningful standards of understanding,
knowledge, and retention of information, consistently citing both a lack of time and resources. In
particular, respondents talked about the difficulty in measuring retention of information. One
participant explained a lack of knowledge in how to do so. Another said, “There is not support at
a leadership level to continue measurement that far into a campaign.”
Respondents were then asked if they recommended that other outtake standards be added
to the list. One respondent mentioned the importance of measuring “relevance,” explaining that
if the employee does not understand why the message is relevant to her in her job (or believes it
is not relevant), the message dies there.
When asked which outttakes standards are the most important and why, respondents
provided many responses. Awareness was mentioned by only two of the 22 participants.
Respondents explained that awareness merely represented the “first step” in measurement and
that if employees have sufficient understanding and retention, it can be assumed that awareness
was present. Nearly half of participants (n=9) said that understanding is the most important
outtake standard. A few participants explained that understanding is a precursor to other more
important standards, such as support, action, and engagement. A total of 27% (n=6) of
participants cited retention and 23% of participants (n=5) listed knowledge as most important.
Some participants explained how the standards are inextricably linked together and difficult to
untangle. One respondent said:

I believe to truly make an impact in employee beliefs and attitudes and drive action, the
program should be understood at a minimum. Ideally employees would have a deeper
level of knowledge to help connect the program and concepts to their team and their own
performance and ultimately be able to retain and recall the information long past the
initial campaign.

Outcomes
More than 50% of participants reported measuring five of the proposed outcomes
standards: satisfaction, attitude, empowerment, collaboration, and trust (see Table 1).
Respondents said that they do not measure some of the outcome standards due to a lack of time
and resources. One participant explained that some of these standards simply aren’t a priority,
and another said, “some of these are more intangible and thus much harder to adequately
quantify.” A handful of participants offered that many of these standards are captured on
employee engagement surveys conducted by external third parties. One respondent said that
these standards are outside the purview of internal communication.
A few respondents recommended adding employee engagement and continuous
improvement to the outcome standards. Two participants also recommended that a standard be

6
Public Relations Journal
Vol. 11 Issue 3 (February 2018)
© 2018 Institute for Public Relations

included that captures the ability of the employees to connect the dots between their performance
and the achievement of strategic business goals.
When asked which standards they believed are the most important in measuring outtakes
and why, roughly one-third of respondents (n = 7) mentioned trust. One respondent explained,
“Most individuals need to experience an inherent sense of trust in an organization, its leaders and
fellow workers in order to feel confident enough to work well collaboratively and in team
settings.” After trust, the standards mentioned as most important by 20% of participants were
satisfaction and empowerment. Procedural justice, collaboration, and teamwork were all
mentioned as most important by only one respondent each. Two people mentioned engagement
(which was not included on the list) as the most important outcome standard.

Organizational Impact
Participants reported measuring many of the organizational impact standards; the two
measured most frequently were retention of employees and safety (see Table 1). Respondents
said they do not measure all of the organizational impact standards due to lack of time and
resources and a lack of knowledge about how to do so. One participant said these standards are
“outside the scope of the assignment.” A few respondents alluded to a lack of support from
leadership in taking the time and resources to measure some of the standards. A few participants
mentioned the difficulty in specifically measuring productivity. A participant explained,
“Productivity is difficult to define in a consistent manner because the group is so diverse in its
roles.” When asked which concepts do you believe are the most important in measuring
outcomes and why, roughly one-third (n=7) of participants mentioned productivity, due its
connection to business results. The other standards most frequently cited as being important
included reputation and retention. No one mentioned authenticity as the most important standard
to measure.

Incorporating feedback
After taking into consideration participants’ feedback and sytematically analyzing the
suggested edits and feedback to the proposed definitions of the 21 standards, the researchers
made the following adjustments to the working list:

1. Added relevance as an outtakes standard, bringing the total number of standards to 22;
2. more clearly defined the standards of understanding versus knowledge, and collaboration
versus teamwork for improved differentiation;
3. moved the authenticity standard from organizational impact to the outcomes standards
list; and
4. changed the name of the standard procedural justice to fairness;
5. amended the standard definitions for clarity (see Table 2).

Round Two Results

In Round 2 of the Delphi study, participants indicated whether they agreed or disagreed
with the modified standards and definitions and provided additional feedback for improvement.
Participants indicated strong agreement with the modified standards list, with most standards
securing more than 90% approval (see Table 2). Participants had the least level of agreement
with the discretionary effort standard—only 77% agreement. Researchers had originally

7
Public Relations Journal
Vol. 11 Issue 3 (February 2018)
© 2018 Institute for Public Relations

proposed the definition of discretionary effort as follows: “The amount of effort employees are
able to give to an organization, a team, or a project, above and beyond what is required.” Many
respondents recommended replacing “are able to” with “give” in the definition.
Even though participants indicated strong agreement with the proposed standards and
their definitions, researchers carefully analyzed each suggestion and comment offered by
respondents (see Table 2 for representative feedback). In light of these comments, researchers
made further edits to the standard definitions. Respondents offered particularly insightful
feedback on two standards: relevance and engagement. In Round 2, respondents were asked
whether they believed relevance, defined as "the degree to which employees find the issue or
topic in the communication from the organization meaningful and useful," should be added as a
standard. Approximately 82% of respondents said yes. One person explained, “I think this is
something that many of us work towards and it's important to measure.”
Respondents were also asked whether engagement should be added to the internal
communication standards, as suggested by a few people in Round 1. A total of 68% of
participants said that engagement is embeded in other concepts and should not be added as a
stand-alone standard. A total of 32% of respondents said engagement should be added as a
standard. One of these participants explained, “Given that CEOs monitor engagement and invest
in engagement surveys, it's important to define it and for communicators to incorporate it into
their efforts.” However, after careful consideration and conversation, the researchers decided not
to include engagement as a standard, because it is a function of several other standards, including
knowledge, understanding, discretionary effort, trust, and satisfaction. This decision will enable
organizations to pinpoint issues related to engagement; it is more actionable to measure the
antecedents to engagement.
The final proposed internal communication standards and their conceptual definitions are
provided in Table 3. Figure 1 provides an operational model of the proposed standards,
suggesting how outtakes lead to outcomes, which in turn, lead to organizational impact. The
researchers of this study believe that a successful internal communication program must fully
fulfill standards associated with outtakes and outcomes in order to achieve standards of
organizational impact. Future research can test this proposed path as well as the relationships
between the various internal communication standards.

Figure 1. Final Proposed Standards

Outtcomes
Attitude
Outtakes Advocacy Organization
Awareness Authenticy Impact
Knowledge Empowerment Productivity
Understanding Collaboration Innovation
Relevance Teamwork Continuous
Retention of Discretionary Effort Improvement
Information Trust Reputation
Satisfaction Employee Retention
Transparency Safety
Fairness

8
Public Relations Journal
Vol. 11 Issue 3 (February 2018)
© 2018 Institute for Public Relations

Conclusions
Based upon this two-year project that included conversations with hundreds of
communication practitioners and academics and two rounds of the Delphi study, the researchers
have created a list of proposed internal communication standards and definitions (see Table 3).
The research process followed the guidelines of the ISO process that stipulates that global
experts from a variety of organizations, agencies, research consultants and measurement firms,
and academics create standards through a collaborative process (“About Public Relations,”
2016). The researchers hope that these standards will be adopted and used by internal
communication professionals to effectively and efficiently guide measurement and evaluation of
communication initiatives.
Next steps include providing suggestions for measuring the standards and testing them
with select organizations for validity and reliability. The researchers ultimately plan to provide a
tested and effective internal communication guidebook and measurement instructions to enable
internal communication practitioners the ability to measure and evaluate and to demonstrate a
process for testing reliability.
Despite the study’s progress in creating internal communication standards, the research
project has limitations. First, even though the Internal Communication Standards task force
included communication practitioners from around the world and the fact that the proposed
standards were presented to three international conferences where researchers listened to the
feedback of participants, the Delphi study was conducted with participants from only North
America. Therefore, testing the standards with organizations from outside North America is
crucial. Second, this research assumes that the standards are discrete measurements. In reality,
many of these standards—such as awareness and knowledge, collaboration and teamwork, and
trust and satisfaction—are dependent upon or correlated with one another, or in certain instances,
perhaps even causal. Additional testing of the standards will map out the relationships among the
standards and perhaps even uncover more complex standards than first imagined. Third, the
standards presented in this study are those recommended by IC thought leaders, all representative
of senior management. Future research might interview employees to learn what standards they
believe are most important for understanding their attitudes, beliefs, and behaviors.

9
Public Relations Journal
Vol. 11 Issue 3 (February 2018)
© 2018 Institute for Public Relations

References
About Public Relations Research Standards Center (PRRSC). (2016). Retrieved from
[Link]
AMEC. (2014, December). How to use the AMEC valid metrics. Retrieved from:
[Link]
[Link]
Boynton, L. A. (2006). What we value: A Delphi study to identify key values that guide ethical
decision-making in public relations. Public Relations Review, 32(4), 325-330.
doi:10.1016/[Link].2006.09.001
Bowen, S., & Stacks, D. (2013). Toward the establishment of ethical standardization in public
relations research, measurement and evaluation. Public Relations Journal, 7(3), 1-28.
Retrieved from
[Link]
Delbecq, A. L., Van de Ven, A. H., & Gustafson, D. H. (1975). Group techniques for program
planning: A guide to nominal group and Delphi processes. Glenview, IL: Scott Foresman.
Dortok, A. (2006). A managerial look at the interaction between internal communication and
corporate reputation. Corporate Reputation Review, 8(4), 322-338.
doi:10.1057/[Link].1540258
Ehling, W. P., White, J., & Grunig, J. E. (1992). Public relations and marketing practices. In J. E.
Grunig (Ed.), Excellence in public relations and communication management (pp. 357-
394). Hillsdale, NJ: Lawrence Erlbaum.
Eisenmann, M., O’Neil, J., & Geddes, D. (2015). An examination of the validity, reliability, and
best practices related to the standards for traditional media. Research Journal of the
Institute for Public Relations, 2(1), 1-28. Retrieved from
[Link]
standards-traditional-media/
Davis, A. (2015, December 17). Revitalizing internal communication: How to use strategic
thinking to engage employees. Retrieved from
[Link]
nal_communication_how_to_use_str#.WEG3NPkrLIV
Grossman, D. (2013). How internal CEO communications shapes financial performance.
Retrieved from [Link]
pdf/CEO_Research_V.[Link]?t=1484002301231
Harter, J. K., Schmidt, F. L., & Hayes, T. L. (2002). Business-unit-level relationship between
employee satisfaction, employee engagement, and business outcomes: A meta-analysis.
Journal of Applied Psychology, 87(2), 268-279. doi:10.1037/0021-9010.87.2.268
Institute for PR. (2013, July 23). What are standards? [Video File]. Retrieved from
[Link]

10
Public Relations Journal
Vol. 11 Issue 3 (February 2018)
© 2018 Institute for Public Relations

Laskin, A.V. (2016). Levels of evaluation: An agency’s perspective on measurement and

evaluation. Public Relations Journal, 10(2), 1-31. Retrieved from
[Link]
f_Evaluation_An_Agency's_Perspective_on_Measurement_and_Evaluation/links/57c47e
[Link]
Lindenmann, W. K. (2003). Guidelines for measuring the effectiveness of PR programs and
activities. Gainesville, FL: Institute for Public Relations.
Mendez, E., Casadesus, M., & Gimenez, G. (2013). Model for evaluating and improving internal
communication in an organization according to the principles of TQM. In I. Heras-
Saizarbitoria (Ed.), Shedding light on TQM: Some research findings (pp. 145-165).
Bilbao: University of the Basque Country UPV/EHU
Meng, J., & Berger, B. K. (2012). Measuring return on investment (ROI) of organizations'
internal communication efforts. Journal of Communication Management, 16(4), 332-354.
doi:10.1108/13632541211278987
Meng, J., & Pan, P-L. (2012). Using a balanced set of measures to focus on long-term
competency in internal communication. Public Relations Review 38(3), 484-490.
doi:10.1016/[Link].2012.03.005
Michaelson, D., & Macleod, S. (2007). The application of best practices in public relations
measurement and evaluation systems. Public Relations Journal, 1(1), 1-14. Retrieved
from
[Link]
s_In_Public_Relations_Measurement_and_Evaluation_Systems
Michaelson, D., & Stacks, D.W. (2011). Standardization in public relations measurement and
evaluation. Public Relations Journal, 5(2), 1-22. Retrieved from
[Link]
esearch/StandardsCommLifecycle/Standardizaton%20in%20Public%20Relations%20Me
asurement%20and%20Evaluation%[Link]
Pledge Support to Public Relations Standards. Retrieved January 3, 2018, from
[Link]
Richardson. M. (2015, September 17). Measuring the effectiveness of internal communication
Retrieved from [Link]
communication
Ruck, K. (2015, October). Top tips: internal communication measurement. Retrieved from
[Link]
Ruck, K., & Welch, M. (2012). Valuing internal communication: Management and employee
perspectives. Public Relations Review, 38(2), 294-302. doi:10.1016/[Link].2011.12.016

11
Public Relations Journal
Vol. 11 Issue 3 (February 2018)
© 2018 Institute for Public Relations

Towers Watson. (2013, December). 2013-2014 change and communication ROI study. Retrieved
from [Link]
Results/2013/12/2013-2014-change-and-communication-roi-study
Wakefield, R.., & Watson, T. (2014). A reappraisal of Delphi 2.0 for public relations research.
Public Relations Review, 40(3), 577-584. doi:10.1016/[Link].2013.12.004
Watson, T. (2008). Public relations research priorities: A Delphi study. Journal of
Communication Management, 12(2), 104-123. doi:10.1108/13632540810881938
Zwijze-Koning, K. H. (2016). Auditing organizational communication: Evaluating the
methodological strengths and weaknesses of the critical incident technique, network
analysis, and the communication satisfaction questionnaire (Doctoral dissertation,
University of Twente). Retrieved from [Link]
[Link]

12
Public Relations Journal
Vol. 11 Issue 3 (February 2018)
© 2018 Institute for Public Relations

APPENDIX

Table 1
Percentage of Respondents Measuring Proposed Standards in Round 1

Standard n %*
Outtakes
Awareness 18 90
Understanding 13 65
Knowledge 10 50
Retention 9 45
Outcomes
Satisfaction 14 82.35
Attitude 11 64.71
Empowerment 11 64.71
Collaboration 10 58.82
Trust 10 58.82
Advocacy 8 47.06
Teamwork 8 47.06
Discretionary 6 35.29
Effort
Transparency 6 35.29
Procedural 4 23.53
Justice
Organizational
Impact
Retention of 15 71.43
Employees
Safety 15 71.43
Reputation 12 57.14
Productivity 10 47.62
Innovation 9 42.86
Authenticity 7 33.33
Continuous 7 33.33
Improvement
* Percentages vary depending on the number of respondents per section.

13
Public Relations Journal
Vol. 11 Issue 3 (February 2018)
© 2018 Institute for Public Relations

Table 2
Feedback on Proposed Definitions from Participants in Round 2

Proposed Proposed Definition % % Representative Feedback

Standard Agree Disagree
Outtakes
Awareness Whether employees have heard of an 91 9 None
organizational message, issue, or
topic
Knowledge Employees’ level of comprehension 91 9 Include other parts of the business,
about organizational messages, competitors, the marketplace, and
issues, or topics customers
Understanding Employees' ability to think about, 95 5 Some employees may be able to think
relate to and use their knowledge in a about, but not relate to or use their
way that helps the organization knowledge to achieve goals. I like the idea
achieve its goals of a shorter, crisper statement … employees'
ability to use this knowledge in a way that
helps the organization achieve its goals
Relevance Degree to which employees find the 82 18 I don't want to evaluate my communications
issue or topic in the communication broadly based on an assumption that it was
from the organization meaningful relevant to all; don't see it as a standard.
and useful
Retention of Degree to which employees can 82 18 I just don't see this as a standard for
Information recall key messages or topics when communications on a regular basis. This
asked really is the same as knowledge since
neither definition involves a time element
Outcomes
Attitude A way of thinking or feeling about an 86 14 Attitude can also refer to how someone
organization, ranging from very thinks about a program, initiative, issue, etc.
positive to very negative
Advocacy Employees’ discretionary effort and 95 5 I don't see this as a standard for all
time to promote or defend an employees. How would results be
organization and its products and evaluated?
services
Authenticity Perception that an organization is 91 9 I think people can be authentic, not
transparent, honest, and fair, organizations
especially regarding the pursuit of its
organizational objectives
Empowerment Employees have the information, 86 14 Suggest "incentives" instead of "rewards;"
rewards, and power to take initiative I'd look to streamline this. I would simply
and make decisions to solve problems say that employees have the power (or
and improve performance maybe ability) to take initiative and make
decisions to solve problems and address
performance

Collaboration The process of employees across 82 16 Does this have to be a process? Believe
different divisions and or units "different divisions and/or units" is too
coming together to solve a problem limiting; employees coming together,
and/or create something successfully typically from different functions, divisions
or units.; These days, collaboration happens
both within and outside the company

14
Public Relations Journal
Vol. 11 Issue 3 (February 2018)
© 2018 Institute for Public Relations

Teamwork The process of employees within the 95 5 I'm a little worried about process; does it
same unit coming together to have to be under the leadership of an
successfully achieve a common goal appointed manager? It seems teamwork
or objective under the leadership of could also be from different divisions - not
an appointed manager just the same unit
Discretionary The amount of effort employees are 77 23 Able to give doesn't seem descretionary to
Effort able to give to an organization, a me. I view it as a choice, perhaps willing or
team, or a project, above and beyond demonstrated? Suggest adding "are willing
what is required and able to give ... "

Trust A belief in the reliability, truth, and 91 9 Could you just say "I believe in the
integrity of the organization's reliability truth and integrity of the
leadership, decision-making, and organization's leadership." It’s incomplete
communication and missing the core element of trust, which
is the belief that leadership will do the right
thing
Satisfaction Extent to which employees are happy 91 9 For the definition to work, the term needs to
or content with their job or work be "job satisfaction."

Transparency The willingness of the organization 91 9 Not just timely fashion but clear and
to share complete positive and forthright; Transparency is about sharing
negative information with employees information that employees need in order to
in a timely fashion make informed decisions; What about
comprehensive vs. complete?
Fairness Employee perception that 91 9 None
organizational processes that allocate
resources and resolve disputes are
impartial and just
Organizational
Impact
Productivity The quality and quantity of work 91 9 Every time we combine elements into a
output based on resources definition, we're asking for agreement (or
disagreement) on both; Not sure if I
understand 'based on resources.' It’s about
the work output to benefit/support the
organization's goals, etc.
Innovation Thinking differently and 100 0 Or ... to benefit the organization (vs. related
experimenting with new approaches, to the organization)
ideas, or behaviors related to the
organization
Continuous The process by which employees 100 0 Three choices again. Can we just say
Improvement offer small or large improvements to "Employees are able to offer ways to
improve efficiency, productivity, and improve their work."
quality of a product or process in the
work environment
Reputation Stakeholders’—both internal and 86 14 I think it should be about the company, its
external—evaluation of an practices and its communication; Needs a
organization based upon personal and bit more related to the actions and behaviors
observed experiences with the of the organization and the likelihood of its
company and its communication future behavior.

15
Public Relations Journal
Vol. 11 Issue 3 (February 2018)
© 2018 Institute for Public Relations

Employee The number or percentage of 100 0 None

Retention employees who remain employed
after X period of time
Safety Employees’ freedom from physical 91 9 Employees' expectation of and commitment
and emotional harm, injury or loss to a workplace that is free from

Table 3
Final Proposed Standards and Definitions
Standard Definition
Outtakes
Awareness Whether employees have heard of an organizational message, issue, or topic
Knowledge Employees’ level of comprehension about organizational messages, issues, or topics
Understanding Employees' ability to relate their knowledge to their work in a way that helps the organization
achieve its goals
Relevance Degree to which employees communication from the organization meaningful and useful
Retention of Degree to which employees can recall key messages or topics when asked after an x timeframe
Information
Outcomes
Attitude A way of thinking or feeling about a subject (about an organization, topic, or issue) ranging from
very positive to very negative
Advocacy Employees’ discretionary effort and time to promote or defend an organization and its products
and services
Authenticity Perception that an organization is transparent, honest, and fair, especially regarding the pursuit of
its organizational objectives
Empowerment Employees have the information, rewards, and power to take initiative and make decisions to solve
problems and improve performance
Collaboration The process of employees across different divisions and or units coming together to solve a
problem and/or create something successfully
Teamwork The process of employees within the same unit coming together to successfully achieve a common
goal or objective under the leadership of an appointed manager
Discretionary Effort The amount of effort employees give to an organization, a team, or a project, above and beyond
what is required
Trust A belief in the reliability, truth, and integrity of the organization's leadership, decision-making,
and communication
Satisfaction Extent to which employees are happy or content with their job or work
Transparency The willingness of the organization to share positive and negative information with employees in
a timely fashion
Fairness Employee perception that organizational processes that allocate resources and resolve disputes are
impartial and just
Organizational
Impact
Productivity The quality and quantity of work output based on resources
Innovation Thinking differently and experimenting with new approaches, ideas, or behaviors related to the
organization
Continuous The process by which employees offer small or large improvements to improve efficiency,
Improvement productivity, and quality of a product or process in the work environment
Reputation Stakeholders’—both internal and external—evaluation of an organization based upon personal and
observed experiences with the company and its communication
Employee Retention The number or percentage of employees who remain employed after X period of time
Safety Employees’ freedom from physical and emotional harm, injury or loss

16