How Users Evaluate Website Credibility

How Do People Evaluate a Web Site’s Credibility?
Results from a Large Study
A report of research by
B.J. Fogg, Ph.D., Cathy Soohoo, David Danielson

Persuasive Technology Lab
Stanford University
Leslie Marable
Consumer WebWatch
Julianne Stanford and Ellen R. Tauber

Sliced Bread Design, LLC
Report Released: October 29, 2002

Report Updated on November 11, 2002
Study term: June 15, 2002, to August 15, 2002
For more information about this report, contact bjfogg@stanford.edu or brenbe@consumer.org

Report Contents
Abstract ....................................................................................................... 4
Executive Summary...................................................................................... 5
About these joint studies .............................................................................. 5
Key findings ............................................................................................... 6
About Consumer WebWatch ....................................................................... 7

About the Stanford Persuasive Technology Lab ............................................. 8
About Sliced Bread Design, LLC .................................................................. 8
Introduction ................................................................................................. 9
How to read this report ...............................................................................10
Background & Methods .............................................................................. 12

Choosing 10 content categories ....................................................................12
Choosing 10 Web sites in each category ........................................................13
How we recruited study participants..............................................................13
Who participated in this study ......................................................................14
What people did in the study........................................................................14
Cleaning the data .......................................................................................19
How we coded and analyzed the comments....................................................20
How we calculated the rankings....................................................................21
Results & Discussion .................................................................................. 22
Part 1: Results & Discussion – Overall Analysis of Credibility Comments ... 23

1. Design Look – 46.1% Overall................................................................24
2. Information Design/Structure – 28.5% Overall.........................................26
3. Information Focus – 25.1% Overall ........................................................27
4. Company Motive – 15.5% Overall .........................................................31
5. Information Usefulness – 14.8% Overall .................................................32
6. Information Accuracy – 14.3% Overall ....................................................34
7. Name Recognition and Reputation – 14.1% Overall ..................................35
8. Advertising – 13.8% Overall ..................................................................39
9. Information Bias – 11.6% .....................................................................41
10. Writing Tone – 9.0% Overall................................................................43
11. Identity of Site Operator – 8.8% Overall................................................45
12. Site Functionality – 8.6% Overall.........................................................46
13. Customer Service – 6.4% Overall .........................................................48
14. Past Experience with Site – 4.6% Overall...............................................50
15. Information Clarity – 3.7% Overall .......................................................51
16. Performance on Test by User – 3.6% Overall .........................................54
17. Readability – 3.6% Overall ..................................................................55
18. Affiliations – 3.4% Overall ...................................................................56
Overall discussion on comments ...................................................................58

Why is Design Look so prominent? .............................................................58
Areas that did not emerge in this research ..................................................59
bjfogg@stanford.edu page 2
Part 2: Results & Discussion – A Focus on Individual Web Site Categories 61

A word about the credibility-ranking results.................................................61
E-Commerce Web Sites: Credibility Results ....................................................63

Entertainment Web Sites: Credibility Results ..................................................65
Finance Web Sites: Credibility Results ...........................................................67
Health Web Sites: Credibility Results.............................................................69
News Web Sites: Credibility Results ..............................................................71
Nonprofit Web Sites: Credibility Results .........................................................73
Opinion or Review Web Sites: Credibility Results.............................................75
Search Engine Web Sites: Credibility Results ..................................................77
Sports Web Sites: Credibility Results.............................................................79
Travel Web Sites: Credibility Results .............................................................81
Concluding Discussion Focusing on Web Site Categories...................................83

Concluding discussion about Web site rankings ............................................83
How Web site topic influences credibility assessments...................................84
How to View This Study & Other Web Credibility Research ........................ 85
Implications for Consumer WebWatch ....................................................... 88

The Need for Education ...............................................................................88
The Need for Evaluation Assistance ...............................................................89
References ................................................................................................. 90
Appendix A: Web Sites Selected for this Study ........................................... 92
Appendix B: Coding Categories .................................................................. 96
Appendix C: Consumer WebWatch Guidelines .......................................... 102
Endnotes .................................................................................................. 104
Abstract
In this study, 2,684 people evaluated the credibility of two live Web sites on a similar topic (such
as health or news). We gathered the comments people wrote about each site’s credibility and
analyzed the comments to find out what features of a Web site get noticed when people evaluate
credibility. We found that the “design look” of the site was mentioned most frequently, being
present in 46.1 percent of the comments. Next most common were comments about information
structure and information focus. In this paper we share sample participant comments in the top 18
areas that people noticed when evaluating Web site credibility. We discuss reasons for the
prominence of design look, interpret the findings in light of Prominence-Interpretation Theory,
and outline the implications of this research for Consumer WebWatch.
Executive Summary
With more than 50 percent of the U.S. population having Internet access, the World Wide Web
has become an important channel for providing information and services. As the Web becomes a
part of people’s everyday lives—booking travel, finding health information, buying products—
there is a growing need to help people figure out whether a Web site is credible or not: Can I trust
the information on this site? Can I trust in the services this site describes?
As part of the Stanford University Persuasive Technology Lab’s mission since 1998, the team has
investigated what causes people to believe − or not believe − what they find online. Similarly,
Consumer WebWatch, which commissioned this study, has the goal to investigate, inform, and
improve the credibility of information published on the World Wide Web. Consumer WebWatch
wanted to investigate whether consumers actually perform the necessary credibility checks-and-
balances while online that they said they did in an earlier national poll (e.g., read privacy policy
pages with at least some frequency). These shared missions created a nexus between the two
organizations, which led to collaboration on what we believe to be the largest Web credibility
project to date.
The resulting consumer-driven study titled, How Do People Evaluate a Web Site’s Credibility?
Results from a Large Study invited more than 2,600 people to rate the credibility of Web sites in
10 content areas. This study was launched jointly with a parallel, expert-focused project
conducted by Sliced Bread Design, LLC. In the expert study titled, Experts vs. Online
Consumers: A Comparative Credibility Study of Health and Finance Web Sites, 15 health and
finance experts were asked to assess the credibility of the same industry-specific sites as those
reviewed by the Stanford PTL consumers.
About these joint studies

Stanford Persuasive Technology Lab: In this study, 2,684 people evaluated the credibility of
two live Web sites randomly assigned from one of 10 content categories (e-commerce,
entertainment, finance, health, news, nonprofit, opinion or review, search engines, sports, and
travel.) A total of 100 sites were assessed. We gathered the comments people wrote (2,440 in all)
about each site’s credibility and analyzed these comments to track which features of a Web site
were noticed (or went unnoticed) when consumers evaluated credibility online. (See Appendix A
for a complete list of Web sites selected for this study.)
Sliced Bread Design: In this study, 15 experts from the health and financial fields were asked to
assess the credibility of sites in their respective areas. A total of 8 health and 7 finance experts
visited the same sites (10 health sites or 10 finance sites) as the consumers in the Stanford PTL
study. They were asked to rank from 1-to-10 the credibility of the sites specific to their area of
expertise, as well as provide detailed written assessments of each site under review. (See
Appendix B in the Sliced Bread Design study for a list of the expert participants and brief bios.)
Key findings
We found that when people assessed a real Web site’s credibility they did not use rigorous
criteria, a contrast to the findings of Consumer WebWatch’s earlier national survey, A Matter of
Trust: What Users Want From Web Sites, released April 16, 2002. In this poll of 1,500 U.S. adult
Internet users, people claimed that certain elements were vital to a Web site’s credibility (e.g.,
having a privacy policy). But this most recent Web-based credibility study showed that people
rarely used these rigorous criteria when evaluating credibility (e.g., they almost never referred to
a site’s privacy policy.) We found a mismatch, as in other areas of life, between what people say
is important and what they actually do.
The data showed that the average consumer paid far more attention to the superficial aspects of a
site, such as visual cues, than to its content. For example, nearly half of all consumers (or 46.1%)
in the study assessed the credibility of sites based in part on the appeal of the overall visual design
of a site, including layout, typography, font size and color schemes.
This reliance on a site’s overall visual appeal to gauge its credibility occurred more often with
some categories of sites than others. Consumer credibility-related comments about visual design
issues occurred with more frequency with finance (54.6%), search engines (52.6%), travel
(50.5%), and e-commerce sites (46.2%), and with less frequency when assessing health (41.8%),
news (39.6%), and nonprofit (39.4%) sites. In comparison, the parallel Sliced Bread Design study
revealed that health and finance experts were far less concerned about the surface aspects of these
industry-specific types of sites and more concerned about the breadth, depth, and quality of a
site’s information.
Topics our analysis did not find

Although our analysis probably did not reveal all issues related to the credibility of the sites in
this study, there were topics we looked for in our analysis but did not find. A relatively small
number of consumers registered credibility assessment comments that related to Consumer
WebWatch’s five general guidelines for improving credibility on the Web: Identity, Advertising
and Sponsorships, Customer Service, Corrections, and Privacy. (See
http://www.consumerwebwatch.org/bestpractices/index.html)
As we examined the 2,440 comments about credibility, we found that less than 10 percent of the
participants’ comments (or 8.8%) referred to the identity of the site or its operator. Nearly 7
percent (or 6.4%) of consumers in our study made comments about a site’s customer service or
related policies when assessing credibility. Nearly 3 percent (or 2.3%) of consumer comments
referred to a site’s sponsorships when assessing credibility – whether perceived as positive or
negative in nature. We found that people mentioned privacy policies in less than 1 percent of
their comments. We also looked for comments about correcting false or misleading
information and found no comments along these lines. These last two issues apparently had little
effect on how our participants evaluated the credibility of Web sites in this study.
Our result among consumers about the prominence of site design and overall look was not what
we had hoped to find. Participants seemed to make their credibility-based decisions about the
people or organization behind the site based upon the site’s overall visual appeal. We had hoped
to see people use more rigorous evaluation strategies while assessing sites. This result indicates
that Consumer WebWatch, along with librarians and information professionals, must increase
efforts to educate online consumers so they evaluate the Web sites they visit more carefully and
make better educated decisions, particularly when it could adversely affect their pocketbooks or
health situations.
Insights from Prominence-Interpretation Theory

In addition, this paper draws on Prominence-Interpretation Theory to explain what happens when
consumers assess credibility – and why a mismatch seems to occur between what people say they
do and what they actually do online.
There seem to be two pieces to the Web credibility evaluation puzzle. Previous research focused
on just one piece: the judgments people make about Web site features (e.g., who sponsors the site,
the presence of a privacy policy, broken links). The other piece of the puzzle deals with what
people notice when they evaluate a site for credibility. Until this study, there was no data about
this second piece. For this reason, the current study is special because it is the first to generate
findings about what people notice when they evaluate a Web site for credibility.
Putting these two pieces together − what people notice about a site and the judgments they make
as a result − gives a fuller picture of what occurs during an online credibility assessment. As a
result of this study, we finally have data about both elements in Prominence-Interpretation Theory
– what gets noticed and the judgments people make. Bringing the various studies together creates
a richer understanding of how people evaluate the credibility of Web sites. (See the “How to
View This Study & Other Web Credibility Research” section for a more detailed explanation
of Prominence-Interpretation Theory.)
---------------------
About Consumer WebWatch

Consumer WebWatch is a project of Consumers Union, the non-profit publisher of Consumer
Reports magazine and ConsumerReports.org. The project is supported by The Pew Charitable
Trusts, which invests in ideas that fuel timely action and results; the John S. and James L. Knight
Foundation, which promotes excellence in journalism worldwide and invests in the vitality of 26
U.S. communities; and the Open Society Institute, which encourages debate in areas in which one
view of an issue dominates all others. Consumer WebWatch’s Web site launched April 16, 2002.
http://www.consumerwebwatch.org
About the Stanford Persuasive Technology Lab

The Stanford Persuasive Technology Lab (captology.stanford.edu) creates insight into how
computers can be designed to change what people think and do, an area called “captology.”
Directed by experimental psychologist B.J. Fogg, the Stanford University team includes social
scientists, designers, and technologists who research and design interactive products (from Web
sites to mobile phones) that motivate and influence their users. Although this area can be
controversial, the lab’s focus is on using the power of persuasive technology to promote positive
changes in domains such as health, education, business, and safety. As part of the lab’s mission,
since 1998 the team has investigated what causes people to believe − or not believe − what they
find online.
http://credibility.stanford.edu
About Sliced Bread Design, LLC

Sliced Bread Design is a usability and interaction design agency established by Julie Stanford and
Ellen Tauber to help people effectively use and enjoy interactive products. Sliced Bread Design
provides user study and interface design services to help organizations create compelling online,
desktop, voice, and wireless software. Its work appears in a variety of products, ranging from
online enterprise resource planning software for Fortune 500 companies to consumer software for
mobile phones. The company is located in California’s Silicon Valley.
http://www.slicedbreaddesign.com

Results from a Large Study
Introduction
Can you trust what you find on the Web today? There’s no simple answer to this question.1
With more than 50% of the U.S. population having Internet access,2 the World Wide Web has
become an important channel for providing information and services. As the Web becomes a part
of people’s everyday lives—booking travel, finding health information, buying products—there is
a growing need to help people figure out whether a Web site is credible or noti: Can I trust the
information on this site? Can I trust in the services this site describes?
If people are unable to assess the credibility of the sites they visit, people will end up embracing
bad information and unreliable services. This could have devastating effects. For example, people
could damage their health or lose their retirement savings if they believe the shoddy information
they have found online. With enough bad experiences like these, large or small (or enough press
coverage of bad experiences), people could stop viewing the World Wide Web as a reliable
channel for information and services.
To take this line of thinking to an extreme, imagine a world in which people could not reliably
assess the credibility of what they find online. What would be the ultimate outcome? In our view,
people would eventually stop using the Web for anything that really matters. In an extreme
situation, the Web would become a channel for trivialities—for content and services that have
little impact on people’s lives. This would be a significant loss for institutions that benefit from
being online. But we believe the loss would prove to be even greater for individuals. So far in its
short lifetime, the Web has provided people with increased options for living rewarding and
productive lives.
One of our goals is to help see that the viability of the Web continues. An even more ambitious
goal—in fact, the essential mission of Consumer WebWatch—is to help make the Web a safe and
reliable channel for people who seek information and services. The study reported in this paper is
i
In this paper we adhere to the definition of credibility outlined by Fogg and Tseng (1999), with the following
discussion drawing largely from this work. In their view, credibility can be defined as believability. Credible
information is believable information. It’s important to note that credibility is a perceived quality. It is not a property of
a Web site, such as how many words the site contains or how many links are on the page. Instead, when one discusses
credibility, it is always from the perspective of the observer’s perception. It’s also important to understand that people
perceive credibility by evaluating multiple dimensions simultaneously. In general, these dimensions can be categorized
into two key components: trustworthiness and expertise. The trustworthiness component refers to the goodness or
morality of the source and can be described with terms such as well intentioned, truthful, or unbiased. The expertise
component refers to perceived knowledge of the source and can be described with terms such as knowledgeable,
reputable, or competent. People combine assessments of both trustworthiness and expertise to arrive at a final
credibility perception.
a step toward achieving this larger goal. A collaboration between Consumer WebWatch and
Stanford University’s Persuasive Technology Lab (with key contributions from Sliced Bread
Design, LLC), our research investigates how people evaluate the credibility of Web sites today.
Our work at this point is descriptive in nature (focusing on what people do) rather than
prescriptive (what people should do). With a basic understanding of how people tend to assess the
credibility of Web sites, Consumer WebWatch is now in a better position to create solutions that
help people evaluate the credibility of online information and services.
We believe that the future health of the Web hinges on issues relating to credibility. For this
reason, we have chosen to invest time and money to understand this domain, with the realization
that our reach is likely to exceed our grasp. To our knowledge this is the largest study to date on
the credibility of Web sites. While our study confirms some earlier assumptions and research
findings,3 the data from this project take our understanding of Web site credibility to a new level,
offering richer insight into credibility assessments. The study also suggests new areas for Web
credibility guidelines, the need for consumer education, and potential areas for future
investigations.
How to read this report

We’ve written the report to be a hybrid between an industry research report and an academic
paper (but with a friendlier style—we hope). To be clear, this report is not a news story—you
won’t find the most important elements in the first few paragraphs.
The first section of our report contains an extended description of the study background,
rationale, and research method (Background & Methods). For some readers this section will be
important and interesting. For other people, getting through the methods section will be a tedious
chore. Although some people may choose to skip or skim the methods section, understanding the
strengths and weaknesses of our research method will help readers make more sense—and
perhaps better personal interpretations—of our data.
Following the Background & Methods section, we present our study results and briefly discuss
those results as we present them. The Results & Discussion section is one that many readers will
care about most. To keep this report to a reasonable length, we do not discuss or interpret all the
data presented in this report. We hope that readers will examine the data and reach some of their
own conclusions, ones that we have not specifically outlined in this report.
After the Results & Discussion section, this paper presents a theory that explains how our study
findings fit with previous research on Web credibility. This theory— called “Prominence-
Interpretation Theory”—is not difficult to understand. For some readers these three pages about
theory may be the most enlightening part of our report, particularly for those who seek to
understand how people assess credibility of Web sites. Some readers may want to jump ahead to
the theory part of our report first (“How to View This Study & Other Web Credibility Research”)
and then return back to our Background & Methods section and continue reading. This nonlinear
approach will give readers more insight into our research throughout the report. (After some
debate, we decided to put the theory section toward the end of this document because we didn’t
want to burden all readers with theory in the first few pages of our report. We want you to keep
reading!)
The final section of the paper interprets the study results in light of the current Consumer
WebWatch Web credibility guidelines. This section will be most important for those interested in
the Consumer WebWatch mission: to make the Web a safe and reliable channel for people who
seek information and services.
The final pages of this document contain references, a set of appendices, and a collection of
endnotes.
Background & Methods

After three years of researching Web credibility, the Stanford Persuasive Technology Lab began
developing a new online method for comparing the credibility of live Web sites. To do this we
performed iterative design and pilot testing, including more than 200 people over the course of
six months. The results were encouraging. Not only did the data produce a relative ranking of the
Web sites that seemed to have high external validity, but also the participants contributed brief
evaluations about the Web sites that struck us as frank and insightful. We concluded from these
two pilot studies that (1) our online research method of paired comparison would work on a larger
scale, for a wider audience; (2) the relative rankings of Web sites that resulted from the research
was interesting and at times provocative, but these rankings had little practical or theoretical
value; and (3) the most valuable information from the pilot studies were the comments people
made about the sites they evaluated.
After refining the research method, the Stanford team began talking to people at Consumer
WebWatch, a nonprofit project of Consumers Union, publisher of Consumer Reports. Consumer
WebWatch commissioned the study covered in this report. This collaboration made sense, since
the goal of Consumer WebWatch is to investigate, inform, and improve the credibility of
information published on the World Wide Web.ii (Note: Consumer WebWatch is supported by
grants from The Pew Charitable Trusts, the John S. and James L. Knight Foundation, and the
Open Society Institute.)
Choosing 10 content categories

Choosing the categories to study was a collaborative task, including researchers from the Stanford
Persuasive Technology Lab, Consumer WebWatch, and Sliced Bread Design, LLC. We selected
categories that (1) are prominent on the Web today, (2) had relevance to Web credibility, and (3)
had a consumer focus (created for ordinary Web users, as opposed to specialists).
The categories we selected were:
• E-Commerce
• Entertainment
• Finance
• Health
• News
• Nonprofit
ii
B.J. Fogg is an adviser to the Consumer WebWatch project. He receives no compensation for advising Consumer
WebWatch, and he received no compensation for being involved with this study. This project was part of his academic
work at Stanford University.
• Opinion or Review
• Search Engines
• Sports
• Travel
Choosing 10 Web sites in each category

Choosing 10 Web sites to test in each category was not an easy task. To explore a range of sites
and, later, to narrow the field, we sought input from the Consumer WebWatch advisers and
researchers at Stanford, among others. We focused on sites that were created for ordinary Web
users, not experts (for example, health sites for consumers rather than for medical professionals).
We sought to include a wide range of Web sites in each category, and in making our selections
we paid attention to issues such as name recognition, popularity, visual design, best practices, and
so on. In the selection process, we tried to avoid selecting two sites that were quite similar (such
as two online bookstores) unless there was a compelling difference between the two, such as
Amazon (purely an online brand) and Barnes and Noble (a brick-and-mortar brand in addition to
an online one).
We knew the choice of Web sites would be important. The final rankings of the sites in each
category would be a direct result of the sites we chose. For example, if we chose only top-quality
sites within one category, then even a very good site could end up on the bottom end of the final
rankings. Even more important, the types of comments we would collect from participants during
the study would hinge on the sites we chose. If sites didn’t offer enough variety, the comments
from participants would also lack variety.
After almost two months of deliberations, we finally arrived at our final 100 Web sites for this
study. A complete list of all the sites within the 10 categories is included in Appendix A.
With the 100 sites selected and programmed into the Web-based research system, we were ready
to begin recruiting participants.
How we recruited study participants

We began recruiting participants in May 2002 by contacting nonprofit groups, such as the
Children’s Brain Tumor Foundation, and offering a $5 donation for each supporter who
completed the study. The nonprofit groups then let their supporters know through e-mail or a link
on their Web pages. We drew on diverse charity groups to get a broad range of participants. Our
goal was to work with charities that had supporters from all over the U.S. and whose supporters
would represent a diversity of ages, income levels, political leanings, and more (the screen shot in
Figure 5 shows the nonprofit groups that participated). From June 15th to August 15th, we
collaborated with 10 nonprofit groups, leading to over 2,600 people participating in the study.
Although the charity recruiting method does not provide a representative sample of Web users,iii
this recruitment method is entirely adequate for the purposes of this study, which are
fundamentally exploratory. In our view, this method is superior to other tractable alternatives
(e.g., offering money directly to people, setting up a contest, spamming.). We also believe that
people who participate to help a charity group will do a better job than people who are doing the
study for other motives, such as personal interest in winning a contest.
Who participated in this study

A total of 2,684 people completed the study. We did not require people to leave demographic
information, yet 60.1 percent of the participants did so. From the information gathered, we
constructed a demographic picture of the people who participated:
• Female: 58.1%; Male: 41.9%

• Average age: 39.9
• Average use of Web: 19.6 hours/week
The respondents who contributed demographic information came from 47 states (no one claimed
to be from Idaho, North Dakota, or Wyoming) and the District of Columbia. The vast majority of
participants live in the U.S., although people from 29 other countries also participated.
What people did in the study

We provide a more detailed description of what people did in this study below. The following
seven steps summarize the participants’ experience:
After being contacted by a nonprofit group or a friend, people interested in helping with
our study:
1. Logged on to www.mostcredible.org
2. Were welcomed and introduced to the study
3. Were randomly assigned to one of 10 Web site content categories (such as
health or news)
4. Were given two live Web sites to evaluate for credibility
iii
In an ideal world, this type of large-scale research would draw on a representative sample of Web users. However, in
the past we’ve found that obtaining a truly representative sample is not possible—or at least not possible without an
enormous budget. As an alternative, in this study we used a recruiting process that has worked well for us before:
charity collaborations. In our view, this recruitment method is better than other methods often used, such as spamming,
offering discounts at online retailers, or entering people in sweepstakes.
5. Reviewed the two sites assigned to them

6. Ranked the two sites according to credibility
7. Left comments about each site’s credibility
A more detailed description of the participants’ experience follows:
After being contacted by a nonprofit group or a friend, people interested in helping with the study
would log on to www.mostcredible.org and begin. The first page, shown in Figure 1, welcomed
people to the study, outlined the three steps, and reminded them that they could later select a
nonprofit group to receive a donation.iv
Figure 1: The first page participants saw in the study.
After participants read about the study, they would select “Click here to begin” to go to the next
page, shown in Figure 2.
iv
Nowhere in the study did we mention the participation of Consumer WebWatch or its affiliates. We suspected that the
influential reputation of Consumers Union would increase the likelihood that companies or people would log on to the
study many times and skew the results.
Figure 2: Participants were randomly assigned to view two Web sites from one of 10 content categories.
At this point, the Web-based research engine would randomly assign the participant to one of 10
content categories (health, travel, finance, etc.) and randomly select two sites for evaluation from
that category.
The Web page listed the category (such as “Finance Web Sites”) and listed two Web sites by
name and URL. The text on the page asked participants to visit the two sites, return and rank
which site was the more credible of the two, and share their comments. Participants could click
on the site name or URL to have a new browser window open containing a live version of that
Web site, as diagrammed in Figure 3.
Figure 3: Participants viewed two Web sites in the same category.
After participants examined the two Web sites, they returned to the main page to rank which of
the two sites they found more credible. Next, they shared comments about their decision, as
shown in Figure 4. The system required people to put in a ranking but did not require them to
leave comments; however, most people left comments.
Figure 4: Participants ranked the Web sites and left comments about each one.
After entering comments about the sites, participants submitted this information. They were then
taken to a page that asked for demographic information (again, not required, but most people
cooperated) and asked which nonprofit should receive a $5 donation, as shown in Figure 5.
Figure 5: Screen for demographics and for nonprofit donations.
Participants concluded their role in the study by submitting the page containing demographics
and their nonprofit selection. They then saw a “thank you” screen (not shown here) that provided
contact information about the study.
Cleaning the data

We asked participants to complete the study only once. We believe that the vast majority of
people honored this request. However, there was no foolproof way to assure that someone would
not repeat the study in order to earn more money for a nonprofit group or, for whatever motive, to
manipulate the study findings. To take steps toward ensuring the integrity of the data, we
designed our Web-based system to record the IP addresses of each participant. Finding duplicate
IP addresses in the data file would be a reasonable—though not certain—indicator that one
person had participated in the study multiple times. There were a number of cases where one IP
address is the source for multiple evaluations (in one case, as many as 20). We examined each of
these cases, realizing that two people in one household could have completed the study using the
same IP address. We also looked for unusual patterns in the data to find evidence of tampering: a
Web site that had lots of evaluations in a short period of time, a single category that was
evaluated significantly more than others, and so on. After inspecting these cases, we removed
data that showed compelling evidence of someone completing multiple sessions or tampering
with our randomization scheme.
How we coded and analyzed the comments

The most valuable data from this research are the comments people made about the Web sites
they evaluated. During the course of the study, the database stored these comments, along with
the ID number for the site under evaluation, the participants’ demographics, and other particulars.
These comments and associated information became the focus of analysis once data collection
was complete.
The first step in analyzing the data was to code the comments according to content. The study
generated 2,440 comments about Web credibility. Some of these comments were brief and others
were lengthy. Some were trivial and others were insightful. We believe that this collection of
comments about Web credibility is the largest to date and offers many opportunities for analysis.
We present one type of analysis here; other analyses and interpretations of the data are possible.
Two independent coders went through the participant comments and assigned codes to mark what
was said in the comment. A third coder then went through the data to resolve discrepancies. Each
comment could receive more than one code. For example, the comment below would be coded in
two categories: design look and information bias.
• “This Web site looks more professional than the other, but I believe it is also more
biased.”
Described more in the Appendix B, the categories for coding came from two sources—the current
version of the Consumer WebWatch guidelines (retrieved from
http://www.consumerwebwatch.org/bestpractices/index.html on August 15, 2002), and from the
emerging themes in the consumer comments themselves (visual design, previous experience with
the site, etc.).
After coding each comment, we tallied the frequency for each code category. In other words, we
calculated how often a specific issue was mentioned. For example, we found that information bias
was mentioned in 283 of the 2,440 comments—11.6 percent of the time. This frequency score
gave an indication of what criteria people used—or said they used—to make their credibility
evaluations of the sites they saw.v The Results & Discussion section of this report has more
information about how we analyzed the comments and what we found.
v
We did not require people to make comments in this study, and, of course, not everyone did.
How we calculated the rankings

The ranking system is straightforward. When a user evaluated a site as more credible than the
other site listed on the page, the more credible site won a point (+1 went into the database for that
site) and the less credible site lost a point (-1 went into the database). Over the course of the
study, each site was evaluated many times, gaining and losing points along the way. At the end of
the study, each site received a final score, which was the average (mean) of all the scores it had
received (the total number of points divided by the total number of times the site was ranked). So
if a site were to win all of its credibility pairings, it would have a score of +1.0. If it won half of
the time, the score would be 0.0. And if it won 75 percent of the time, the score would be +0.50.
The 10 sites in each category were then ranked according to their mean scores, highest to lowest.
This ranking gives a general idea about which sites people in this study found most and least
credible. Small differences in means between two sites in the same category are not practically
significant.
We should note here that study participants did not know our plans for this research. They did not
know we were studying 10 categories of Web sites and that we were ranking sites within each
category. Participants simply evaluated two Web sites. Had participants known we were
compiling data to create a credibility ranking of Web sites, we suspect that some people may have
tried to manipulate our results.
Results & Discussion

This section presents the results of our study and discusses the findings we judge as the most
notable. For the sake of clarity, we present the Results & Discussion section in two parts, as
shown below:
Part 1: Results & Discussion – An Overall Analysis of Credibility Comments

Part 2: Results & Discussion – A Focus on Individual Web Site Categories
In Part 1 we share our analysis of the 2,440 comments that participants made about the credibility
of the Web sites. Our analysis found 18 types of comments relating to credibility with incidence
over 3 percent. We present the data and discuss each type of comment, from most to least
common.
In Part 2 we shift our focus from the types of comments people made to the types of Web sites in
the study, presenting and discussing the results for each Web site category one by one in the
following order:
• E-Commerce
• Entertainment
• Finance
• Health
• News
• Nonprofit
• Search Engines
• Sports
• Travel
Part 1: Results & Discussion – Overall Analysis of Credibility Comments

As explained in the Methods section, participants were given the opportunity to leave brief
comments about the credibility of the two sites assigned to them. Discussed in more detail earlier,
we coded 2,440 comments according to categories described in the Appendix B. The following
pages share the results of our content analysis, starting with the overview in Table 1.
Table 1 presents an overall picture of our content analysis for comments about all 100 sites in this
study. This table shows 18 types of comments, from “design look” to “affiliations.” The
percentages in the table represent how often a comment on that topic appeared in the entire set of
comments. For example, participants in our study mentioned something about the “design look”
of the site in 46.1 percent of the 2,440 comments.
Percent Comment Topics

(of 2,440 comments) (addressing specific credibility issue)
1. 46.1% Design Look

2. 28.5% Information Design/Structure
3. 25.1% Information Focus
4. 15.5% Company Motive
5. 14.8% Information Usefulness
6. 14.3% Information Accuracy
7. 14.1% Name Recognition and Reputation
8. 13.8% Advertising
9. 11.6% Information Bias
10. 9.0% Writing Tone
11. 8.8% Identity of Site Operator
12. 8.6% Site Functionality
13. 6.4% Customer Service
14. 4.6% Past Experience with Site
15. 3.7% Information Clarity
16. 3.6% Performance on Test by User
17. 3.6% Readability
18. 3.4% Affiliations
(Categories with less than 3% incidence are not in this table.)
Table 1: How often participants commented on various issues when evaluating the credibility of Web sites.
While the percentages in Table 1 show the types and frequency of comments about Web
credibility, the table alone does not give rich information. Below we provide explanations and
examples for each type of comment, along with the incidence for each category of Web site.
1. Design Look – 46.1% Overall

When evaluating the credibility of a Web site, participants commented on the design look of the
site more often than any other Web site feature, with 46.1 percent of the comments addressing the
design look in some way. When coding for comments on design look, researchers included
comments on many elements of the visual design, including layout, typography, white space,
images, color schemes, and so on. The comments could be either positive or negative. Some of
the comments coded in this category are as follows:
• This site is more credible. I find it to be much more professional looking. —M, 38,
Washington
• More pleasing graphics, higher-quality look and feel — F, 52, Tennessee
• Just looks more credible. —M, 24, New Jersey
• Actually, despite the subject of the Web site, it looks very credible. This may be due to
the subdued color scheme and the font used on the left-hand side of the page. —F, 29,
California
• I know this is superficial, but the first thing that struck me is the color difference. The …
site is a soothing green (sort of like money) while the [other] site is a jarring purple. —
M, 56, Virginia
• The design is sloppy and looks like some adolescent boys in a garage threw this
together. —F, 48, California
• Not very professional looking. Don’t like the cheesy graphics. —F, 33, Washington.
• Looks childish and like it was put together in 5 minutes. —F, 25, Maryland
Design Look comments, by site category

As shown in Figure 6, nearly half of all site evaluators (46.1%) used visual cues, such as the site’s
overall design or look, to assess a site’s credibility. This happened slightly more frequently with
the finance (54.6%), search engines (52.6%) and travel (50.5%) categories, and less frequently
with the health (41.8%) and news (39.6%) categories.
Percentage of Comments
Related to Design Look
Finance 54.6%
Search Engines 52.6%
Travel 50.5%
Sports 48.8%
Category
Entertainment 46.8%
E-Commerce 46.2%
All Sites 46.1%
Health 41.8%
News 39.6%
Nonprofit 39.4%
Opinion or Review 38.1%
0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0%
Percentage of Site Evaluations
Figure 6: Percentage of comments related to design look, by category
What we find notable about Design Look

One of the overall findings from this study is that our participants relied heavily on the surface
qualities of a Web site to make credibility judgments. Our result about the prominence of design
look was not what we had hoped to find; we had hoped to see that people used more rigorous
evaluation strategies. However, our result is consonant with findings of other research (Cockburn
and McKenzie, 2001) that describes typical Web-navigation behavior as “rapidly interactive,”
meaning that Web users typically spend small amounts of time at any given page, moving from
page to page quickly. If such rapid navigation is indeed the norm for most types of Web use, then
it makes sense that Web users have developed efficient strategies, such as focusing on the design
look, for evaluating whether a Web site is worthwhile.
Our results about the connection between design look and perceived credibility suggests that
creating Web sites with quality information alone is not enough to win credibility in users’ minds.
In most cases Web site designers need also to focus on the impression that the visual design will
make, creating a site that achieves what many of our participants described as “a polished,
professional look.” But the connection between visual design and credibility may not be so
simple. Slick-looking Web sites frequently received negative comments. Participants seemed to
make judgments about the people behind the site on the basis of the design look. Many comments
were indicative of this attitude: “It looks like it’s designed by a marketing team, and not by people
who want to get you the information that you need.”
Based on the comments we’ve read from this study, we speculate that once a site is above a user’s
personal threshold to qualifying as having a “professional look,” then other aspects of the Web
site come into the credibility equation. In other words, the visual design may be the first test of a
site’s credibility. If it fails on this criterion, Web users are likely to abandon the site and seek
other sources of information and services.
We discuss the topic of design look in more depth at the end of this section.
2. Information Design/Structure – 28.5% Overall

After design look, the next category that people commented on in assessing credibility was the
structure of the site’s information, being mentioned in 28.5 percent of the total comments. The
participant comments discussed how well or poorly the information fit together, as well as how
hard it was to navigate the site to find things of interest. While information structure is often
associated with usability, the comments here show how information structure has implications for
credibility. Sites that were easy to navigate were seen as being more credible. Some sample
comments are below:
• This site is very well organized, which lends to more credibility. —M, 33, Illinois
• This one is more credible because it is more organized. —F, 57, Maryland
• Horrible site, information badly presented. They try to put everything on the front page,
instead of having multiple layers of navigation. This to me suggests that they
developed this thing on a whim. —M, 42, Canada
Information Design/Structure comments, by site category

As shown in Figure 7, more than a quarter (28.5%) of all site evaluations made reference to issues
of site information design—that is, the site’s overall information structure or organization. This
occurred much more frequently in the search engines (42.6%), finance (33.0%), and travel
(31.8%) categories, and less frequently in the opinion or review (24.3%), sports (22.3%), and
nonprofit (18.2%) categories.
Percentage of Comments Related

to Information Design/Structure
Finance 33.0%
Travel 31.8%
News 30.2%
All Sites 28.5%

Category
Health 28.3%
E-Commerce 26.5%
Entertainment 25.8%
Sports 22.3%
Nonprofit 18.2%
0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% 45.0%
Figure 7: Percentage of comments related to information design, by category.
What we find notable about Information Design/Structure

That information design affects credibility should come as no surprise. A well-organized site is
the antithesis to a site that purposely confuses and misleads a user toward advertisements and
other promotions.
Online usability research has made it clear that information structure is critical for task success on
the Web, and ease of use has been shown to contribute to credibility perceptions in previous
research (Fogg et al., 2000; Fogg et al., 2001; Fogg et al., 2002). The reason behind this
consistent finding isn’t completely clear. One might speculate that by providing a clear
information structure, a Web design team demonstrates expertise to the users. Users may then
assume this expertise extends to the quality of information on the site.
3. Information Focus – 25.1% Overall

In 25.1 percent of the comments about credibility, people in this study talked about the focus of
information on the site. The comments varied in content. At times a focused site was seen as
more credible, other times a narrow focus hurt credibility. What’s clear is that many people in this
study relied on information focus to determine whether a site was credible or not. Sample
comments are below:
• Credible because of the breadth of information available. —M, 35, California
• I find this site trustworthy because it offers a simple message to a very targeted
community. —F, 34, Massachusetts
• This Web site is filled with too much crap. I feel as though part of the reason it seems
less credible is the fact that the crap they fill it with is taking attention away from their
own Web site. — F, 23, Illinois
• Broad categories, but shallow reviews and comparisons. —M, 35, California
• This site seems focused on body image. They have articles about feeling good naked,
the perfect swimsuit for every body type, and toning exercises. Not a lot of solid health
information. —F, 22 Minnesota
Information Focus comments, by site category

As Figure 8 shows, about one-in-four site evaluations (25.1%) made comments about issues of
information focus, that is, the perceived scope or focus of the site. This generally occurred far
more frequently in the information-heavy categories (health: 33.0%; news: 31.9%; sports:
30.8%), and occurred less frequently in the more transactional-heavy categories (e-commerce:
24.7%; search engines: 24.5%; finance: 18.9%).

to Information Focus
Health 33.0%
News 31.9%
Sports 30.8%
Travel 28.5%
Entertainment 25.8%
Category
All Sites 25.1%
E-C ommerce 24.7%
Finance 18.9%
Nonprofit 17.8%
0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0%
Figure 8: Percentage of comments related to information focus, by category.
What we find notable about Information Focus

The site content’s focus (whether broad or narrow), clarity, potential bias, usefulness, and
organization were all examined in this study, with focus or depth appearing most important to our
participants when evaluating credibility. We suspect that in many cases in Web surfing, users
may not necessarily read the in-depth information, but simply having it available seemed to
produce a sense of credibility, suggesting the site is authoritative.
The other notable finding about information focus is how much this issue varied depending on the
type of site, with information focus being most prominent when evaluating health and news sites
and least prominent when evaluating nonprofit sites. The data suggest that people have clearer
expectations about the focus of certain types of Web sites. We speculate that the expectations
about site focus are higher for the types of information-rich sites people know best (e.g., health,
news, sports).
4. Company Motive – 15.5% Overall

We found that 15.5 percent of the comments in this study addressed the perceived underlying
motive of the site or the institution sponsoring the site. These comments often referred to how
Web sites lost credibility when the only purpose of a site seemed to be selling things or getting
money from users. In other cases, Web sites won credibility by conveying motives that people
found to be admirable. Sample comments are below:
• The fact that this site has a global conscience impressed me and made me feel it was
more credible. —F, 40, New Jersey
• This site looks like its goal is to help you find what you are looking for. —F, 55,
California
• I would trust this site because it’s run by a religious denomination whose aim is socially
responsible investing. —F, 54, New York
• Seems too “commercial” and therefore less objective. —M, 52, Texas
• This site says to me “Give us your money and get out.” —F, 29, British Columbia
• Doesn’t seem credible when they give a product a good review and give you a link to
order it too. —F, 38, Texas
Company Motive comments, by site category

As shown in Figure 9, a sizable percentage (15.5%) of all site evaluations commented on the
perceived motive of a company or organization behind the site. This occurred with far greater
frequency in the opinion or review (22.1%), finance (21.0%), and nonprofit (20.2%) categories,
and with less frequency in the search engines (14.2%), travel (12.8%), and news (5.9%)
categories.

to Company Motive
Finance 21.0%
Nonprofit 20.2%
E-C ommerce 19.0%
Health
Category
17.8%
All Sites 15.5%
Travel 12.8%
Sports 11.3%
Entertainment 9.4%
News 5.9%
0.0% 5.0% 10.0% 15.0% 20.0% 25.0%
Figure 9: Percentage of comments related to underlying company motive, by category
What we find notable about Company Motive

Generally, people commented positively on company motive if they felt the site provided useful
information or services without a commercial purpose. Ads or endorsements made the picture
more complicated. A good number of comments made it clear that at least some people
recognized a link between endorsements and company profit. Positive reviews of a product or
firm, for example, were met with skepticism if the organization behind the site might have
something to gain from such good reviews.
5. Information Usefulness – 14.8% Overall

When evaluating Web site credibility, people in this study commented on the usefulness of the
site’s information 14.8 percent of the time. As one might expect, useful information led people to
see the Web site as more credible. Below are sample comments we found in this category:
• This Web site provided useful and interesting knowledge about events in sports. —F,
30, New Jersey
• Liked it because it is something that would be useful to me and other family members.
—F, 18, Illinois
• I searched for a particular scientific term, and this search engine came up with more
useful Web sites than the other one. —F, 40, Washington
Information Usefulness comments, by site category

As Figure 10 illustrates, less than one-in-six site evaluations (14.8%) commented on information
usefulness, that is, how useful the site’s information was to the user. This happened with far
greater frequency in the niche-news categories of health (20.5%), entertainment (19.5%), and
opinion or review (17.1%), and with slightly less frequency in the travel (11.5%), sports (11.4%),
and nonprofit (11.1%) categories.

to Information Usefulness
Health 20.5%
Entertainment 19.5%
E-C ommerce 16.3%
Category
All Sites 14.8%

News 14.5%
Finance 14.1%
Travel 11.5%
Sports 11.4%
Nonprofit 11.1%
0.0% 5.0% 10.0% 15.0% 20.0% 25.0%
Figure 10: Percentage of comments related to information usefulness, by category.
What we find notable about Information Usefulness

As one would expect, Web sites that offer information people find useful win credibility points.
The data suggest that people in our study had the highest expectations about information
usefulness for sites relating to health and entertainment. They had the lowest expectations about
information usefulness for sports and nonprofit Web sites.
6. Information Accuracy – 14.3% Overall

In 14.3 percent of the comments about Web credibility, people talked about the accuracy of the
site’s information. This category includes comments in which people expressed doubt about the
information on the site. But this category also includes comments where people confirmed the
accuracy of what they found on the site. In assessing accuracy, people often drew on their own
knowledge. Samples of comments relating to information accuracy are below:
• Most of the articles on this Web site seem to be headline news that I have already
heard, so they are believable.—F, 50, Ohio
• I work at AOL Time Warner and read the article regarding accounting problems. It
accurately quoted an internal memo from Dick Parsons and the general tone was
positive, especially given the current business environment. —M, 45, New York
• This site is totally based upon personal opinion and admittedly old data and unscientific
methods. —F, 35, Colorado
Information Accuracy comments, by site category

As shown in Figure 11, less than one-in-six (14.3%) of all site evaluations commented on the
accuracy of site information. This occurred far more frequently in the opinion or review (25.4%),
news (21.7%), and health (18.7%) categories, and less frequently in the travel (11.1%), finance
(8.0%), and search engines (7.1%) categories.
Related to Information Accuracy
News 21.7%
Health 18.7%
Entertainment 16.3%
Sports 16.1%
Category
E-C ommerce 15.5%
All Sites 14.3%
Nonprofit 13.5%
Travel 11.1%
Finance 8.0%
Search Engines 7.1%
0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0%
Figure 11: Percentage of comments related to information accuracy, by category.
What we find notable about Information Accuracy

Users come to a site armed with a background of previous knowledge and expertise, and it was
clear from this study’s comments that this background is a critical tool for making credibility
assessments. In particular, comments tended to focus on matching what was found on a site to
what had previously been read or heard on television—that is, information confirmed from other
sources. With respect to credibility assessment, the importance of a site’s matching existing user
beliefs about the content domain, as opposed to challenging those beliefs, is not clear from the
comments and would make for interesting further research.
7. Name Recognition and Reputation – 14.1% Overall

One strategy for evaluating credibility seemed to be relying on the name recognition or reputation
of the site operator (such as the American Red Cross). In 14.1 percent of the comments, people
talked about issues of reputation and name recognition. One frequent comment had to do with
one’s never having heard about an organization before. This hurt the credibility of the site. In
other cases, people saw a familiar company name and inferred the site was credible because of
that. Below are the sample comments coded in this category:
• This site is less credible because the name is unfamiliar. —F, 22, Maryland
• It seems to me that credibility is all about the name and having heard about it. —M, 25,
Michigan
• CNN is well recognized in the US as a provider of news. Their reputation is not

something they would put at risk with unfounded claims or under-researched articles.
—M, 24, Illinois
• The Mayo Clinic has a great reputation. I would trust the info I found at this Web site.
—M, 34, Connecticut
Name Recognition and Reputation comments, by site category

As shown in Figure 12, brand name seems to play some role when consumers assess the
credibility of sites. One-in-seven (14.1%) site evaluations included comments related to the name
recognition or the reputation of a company or the site operator. This occurred more frequently in
the e-commerce (25.9%), finance (21.8%), and news (19.1%) categories, and less frequently in
the health (10.9%), travel (8.8%), and search engine (5.1%) categories.
Percentage of Comments Related to

Name Recognition and Reputation
E-Commerce 25.9%
Finance 21.8%
News 19.1%
Sports 18.6%
All Sites 14.1%

Category
Nonprofit 12.7%
Entertainment 11.1%
Health 10.9%
Travel 8.8%
Search Engines 5.1%
0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0%

Figure 12: Percentage of comments related to name recognition and reputation, by category.
What we find notable about Name Recognition and Reputation

The comments made clear that a site may be perceived as trustworthy if a user has dealt
successfully with the site previously, knows someone who has, knows of the site’s large
following, has heard good things about the site, or at least has heard of the brand name
(sometimes via the site’s real-world counterpart).The reputation of brick-and-mortar
organizations appeared generally to carry over to those organizations’ Web sites, although the
relationship between this tendency and a site’s identity information (see section 11) would make
for interesting further investigation.
8. Advertising – 13.8% Overall

People in this study used advertising on a site as a criterion for judging the site’s credibility. In
13.8 percent of the comments, people talked about advertising, usually negatively. But at times,
study participants talked about the judicious use of advertising in a positive way. Pop-up ads were
widely disliked and seemed always to reduce perceptions of site credibility. Sample comments
relating to advertising are below:
• The advertisements were distracting and reduced the credibility to me. Any site which
gives so much real estate to advertisers probably doesn’t have my best interests in
mind. —M, 25, Washington
• Every link brought pop-under ads as well as traditional ads. I feel their view is colored
by their desire to boost their advertising revenue: they perceive their primary clients to
be their advertising base, rather than the people who use their site. —F, 43, Illinois
• This [site] didn’t have any advertising, which makes it more credible in my opinion. —F,
34, Iowa
Advertising comments, by site category

As shown in Figure 13, nearly 14 percent (13.8%) of all site evaluations included comments
about a site’s advertising, whether positive or negative. This occurred far more frequently in the
search engines (24.6%), health (21.3%), and opinion or review (16.6%) categories, and less
frequently in the travel (10.5%), e-commerce (10.5%), news (9.3%), and finance (8.3%)
categories.
Related to Advertising
Sports 22.7%
Health 21.3%
Nonprofit 13.9%
Category
All Sites 13.8%

Travel 10.5%
E-Commerce 10.5%
News 9.3%
Entertainment 8.9%
Finance 8.3%
0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0%
Figure 13: Percentage of comments related to advertising, by category.
What we find notable about Advertising

Not surprisingly, advertisements are especially harmful to a site’s credibility if they lead a user to
believe that the site’s content is swayed or controlled by the advertisement or that the site is
connected to the advertisement, and is itself trying to sell the user something. The latter is a case
of not providing a clear line between advertising and the site itself, which has been shown to
harm credibility in previous studies (Fogg et al., 2001; Fogg et al., 2002; Princeton, 2002), and is
intimately related to a site’s connection with its sponsors. The comments make clear that some
users are fully aware of potential sponsor influence. They expect a clear line between the content
and advertisements so that sponsors do not compromise the site’s information.
9. Information Bias – 11.6% Overall

In 11.6 percent of the comments, people in this study talked about information bias when
evaluating the credibility of the Web sites they were reviewing. Sample comments related to
information bias are below:
• This site is more commentary, and thus more opinionated. Accordingly, I liked it more,
but the arguments are more disputable, and thus less “credible.” —M, 39, District of
Columbia
• The headlines and editorial copy didn’t even make the pretense of being unbiased,
something I think is critical for an organization or media outlet to call itself “news.” —F,
30, New York
• It is credible because the opinions contained therein are based on unbiased research.
—F, 32, Pennsylvania
Information Bias comments, by site category

As shown in Figure 14, more than one-in-ten site evaluations (11.6%) commented on the
perceived bias—positive or negative—of information for the site evaluated. This occurred with
far greater frequency in the news (30.2%) and opinion or review (23.8%) categories, and with
much less frequency in the finance (8.5%), search engines (3.8%), e-commerce (2.6%), and travel
(1.9%) categories.
Related to Information Bias
News 30.2%
Nonprofit 15.4%
Health 14.8%
Entertainment 14.2%
Category
All Sites 11.6%
Sports 9.9%
Finance 8.5%
Search Engines 3.8%
E-Commerce 2.6%
Travel 1.9%
0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0%

Figure 14: Percentage of comments related to information bias, by category.
What we find notable about Information Bias

The comments on this topic were not surprising: Web sites that are perceived to be biased are also
perceived to lack credibility. As the data show, bias was highly prominent to users when
evaluating sites dealing with news or sites providing opinions or reviews. In contrast, other site
categories seemed not to trigger thoughts about information bias: search engines, e-commerce,
and travel sites.
The participants’ attention to bias in news sites should be encouraging to those who see critical
thinking as essential for a healthy participatory democracy.
10. Writing Tone – 9.0% Overall

The tone of the writing on a Web site was something people noticed when assessing credibility.
The participant comments include writing tone as a criterion 9.0 percent of the time, usually in a
negative way. People generally said that sensationalism or slang hurt a site’s credibility, while a
straightforward, friendly writing style boosted credibility. Some sample comments relating to the
tone of writing are below:
• “Holy Crap” and other slang or poor language harms credibility. Credible people tend to
understate. —F, 53, California
• “Cops” to search lake again vs. “Police”, “8 hurt” vs. “8 injured”, and so on. This site
uses lower English and lowers its credibility. —M, 44, Texas
• Seemed less sensationalistic, more dry, and therefore more credible. —M, 38,
Washington
Writing Tone comments, by site category

As shown in Figure 15, nine percent of all site evaluations mentioned elements related to the tone
or attitude conveyed by a site’s content. Not surprisingly, this occurred in content categories in
which a distinct writer’s voice is more noticeable. For example, comments about the tone of
writing occurred with greater frequency in the news (14.8%), nonprofit (12.9%), and sports
(10.9%) categories, and with less frequency in the search engines (5.6%), and travel (5.2%)
categories.
Related to Writing Tone
New s 14.8%
Nonprofit 12.9%
Sports 10.9%
Entertainment 10.5%
Finance 9.0%
Category
All Sites 9.0%

Health 8.3%
E-Commerce 8.0%
Search Engines 5.6%
Travel 5.2%
0.0% 2.0% 4.0% 6.0% 8.0% 10.0% 12.0% 14.0% 16.0%
Figure 15: Percentage of comments related to writing tone, by category.
What we find notable about Writing Tone

The comments about tone were usually aimed at the particular kind of language or tone the site
used in its writing, which in some cases resulted in a perceived bias in the site’s content. That
people noted the tone of the writing suggests a deeper level of involvement with a Web site, since
it’s hard to judge the tone without engaging with the content. In some sense these issues represent
the second tier of credibility assessment: Once a user is past the look and feel of a site, she is
likely to next assess the specific content. Not surprisingly, use of slang and profanity were clearly
harmful to a site’s credibility. Moreover, participants tended to have clear stances on the type of
tone a credible site’s content would take: “Not impressed by the spinning. I like plain talk.”
Participants claimed to be able to detect a “sales pitch” or “marketing” language, and were
generally skeptical of sites with an abundance of either. Many participants explicitly
distinguished between content that seemed (or was proclaimed to be) factual, opinionated,
“gossipy,” religious, or overzealous.
11. Identity of Site Operator – 8.8% Overall

In 8.8 percent of the comments we collected, study participants talked about how site operators
disclosed information about themselves. Comments coded in this category indicate that a Web
site wins credibility points by giving information about the organization behind the Web site: who
they are, what they do, and how to contact them. Sample comments are below:
• This site contains a clear description of the goals and activities of this charity. There
are contact names and e-mail/snail-mail addresses. There is even a phone number. —
F, 44, Washington
• This site might be a good place to start, but I don’t really know what its mission is—
especially for a for-profit venture. —M, 34, Connecticut
Identity of Site Operator comments, by site category

Figure 16 shows that in general, site evaluators did not seem to focus much on the company or
organization identity when assessing a site’s credibility. Less than 10 percent of site evaluations
(or 8.8%) included comments about site identity. This occurred far more frequently in the
nonprofit (28.9%) category, and far less frequently in the news (4.7%), travel (4.6%), and search
engines (4.3%) categories.

to Identity of Site Operator
Nonprofit 28.9%
E-C ommerce 10.5%
Finance 10.3%
Category
Health 9.1%
All Sites 8.8%
Sports 7.1%
News 4.7%
Travel 4.6%
Search Engines 4.3%
Entertainment 2.6%
0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0%

Figure 16: Percentage of comments related to identity, by category.
What we find notable about Identity of Site Operator

The comments from this study suggest that Web sites win credibility by clearly disclosing the
people and the organization behind the site. This finding is consistent with previous studies on
Web credibility (Fogg et al., 2000; Fogg et al., 2001; Fogg et al., 2002; Princeton, 2002).
What’s most interesting in this part of the data is how nonprofit Web sites scored. The credibility
of nonprofit-organization Web sites depends more directly on demonstrating that there are real
people and a genuine organization behind the site than for any of the other site categories
investigated in this study. Many comments about the nonprofit sites questioned the use of donated
money. It seems clear that nonprofit Web sites are held to higher standards regarding being up-
front about site operator identity.
12. Site Functionality – 8.6% Overall

While study participants experienced the functionality (or lack of it) whenever they visited a Web
site, the comments talking about credibility included issues of site functionality 8.6 percent of the
time, usually in a negative way: The site was down, links were broken, search features were not
helpful. The functionality of a site, whether or not under the direct control of the site operator,
affected the perceived credibility of the site. Sample comments are below:
• The command lines which appear at the top—a bug—make it feel like no one is
watching, taking care of the site. —F, 35, California
• Biggest complaint is the poor search facility. A search produces only three items. —M,
50, California
Site Functionality comments, by site category

As shown in Figure 17, less than 10 percent (or 8.6%) of all site evaluations made comments
about a site’s functionality, which includes technical performance and the services a site offers.
Perhaps not surprisingly, this occurred with far greater frequency in the search engines (20.5%)
category, and with less frequency in the e-commerce (6.6%) and news (5.1%) categories.

to Site Functionality
Entertainment 12.6%
Travel 12.1%
Sports 10.4%
All Sites
Category
8.6%
Health 8.3%
Nonprofit 7.7%
Finance 7.6%
E-Commerce 6.6%
News 5.1%
0.0% 5.0% 10.0% 15.0% 20.0% 25.0%

Figure 17: Percentage of comments related to functionality, by category.
What we find notable about Site Functionality

The most utilitarian of categories in this study—search engines—received the most comments
about functionality. People seemed to see these sites as tools, and people brought expectations of
functionality into their credibility evaluations. In other words, for search engines Web sites (and
to a smaller extent Web sites about entertainment and travel) people seem to ask the question,
“What can you do for me?” If people were impressed with what the site offered in terms of
functionality, they also assessed it to be credible.
13. Customer Service – 6.4% Overall

We created a category called “Customer Service” so we could account for comments people
made about the perceived relationship between the company and the end user. In other words, the
comments in this category are about how the sponsoring organization operates and, especially,
how the organization treats customers along the way. People in our study commented on these
customer-service issues 6.4 percent of the time. Sample comments are below:
• This site seemed to have less accountability to its customers on the items that can be
purchased. —F, 46, Mississippi
• They spell out very clearly what one would get for becoming a member. —F, 34,
Massachusetts
Customer Service comments, by site category

As shown in Figure 18, less than 10 percent (or 6.4%) of all site evaluations commented on
customer service issues or related business policies. Not surprisingly, these comments occurred
with much greater frequency in the transactional-heavy categories (travel: 18.1%; e-commerce:
16.7%), and with far less frequency in the content-heavy categories (search engines: 1.0%; news:
0.8%; health: 0.4%).
Related to Customer Service
Travel 18.1%
E-Commerce 16.7%
Nonprofit 8.2%
All Sites 6.4%
Category
Finance 6.3%
Search Engines 1.0%
News 0.8%
Health 0.4%
Sports 0.0%
Entertainment 0.0%
0.0% 2.0% 4.0% 6.0% 8.0% 10.0% 12.0% 14.0% 16.0% 18.0% 20.0%
Figure 18: Percentage of comments related to customer service, by category.
What we find notable about Customer Service

Overall, the comments said that clear and straightforward explanations of a company’s business
practices and fees enhanced site credibility. But when the operations of a company were not clear,
then the site lost credibility.
The data show that people noticed issues of customer service most often when examining sites
dealing with travel and e-commerce. This makes sense. These sites are highly transactional, and
provide services for users, not just information. In contrast, site categories that are more
exclusively about information—such as sports, entertainment, news, or health—received few
comments about customer service. The marked difference between information and service sites
leads us to speculate that two main subclasses of credibility elements exist: one that applies to
information sites and another that applies to service sites. Making this distinction could be helpful
in future Web credibility studies.
14. Past Experience with Site – 4.6% Overall

In assessing the credibility of the sites in this study, people sometimes drew on their past
experiences with a site to reach a conclusion. This occurred in 4.6 percent of the comments we
collected. In most comments of this nature, past experience with a site boosted its credibility—but
not always. A negative previous experience with a site led to a lower credibility evaluation.
Sample comments are below:
• I’ve used this site before and it did not meet my expectations. —F, 50, Washington
• I have used it frequently and find it very useful. —F, 50, Missouri
Past Experience with Site comments, by site category

As shown in Figure 19, less than 5 percent (4.6%) of all site evaluations commented on the user’s
previous experiences with the site under evaluation, whether good or bad. This occurred with
greater frequency in the search engine (12.8%), news (9.0%), e-commerce (7.9%) and travel
(6.2%) categories, and with less frequency in the health (2.1%) and finance (1.5%) categories.
Percentage of Comments Related to

Past Experience with Site
News 9.0%
E-C ommerce 7.9%
Travel 6.2%
All Sites 4.6%

Category
Sports 3.3%
Health 2.1%
Entertainment 2.1%
Nonprofit 1.9%
Finance 1.5%
0.0% 2.0% 4.0% 6.0% 8.0% 10.0% 12.0% 14.0%

Figure 19: Percentage of comments related to past experience, by category.
What we find notable about Past Experience with Site

For this topic more than the others, the sites we chose to include in the research influenced the
comments people made. Had we chosen obscure sites, we would have found few comments about
past experience—people saying they had never used the site before (this seemed to be the case
with the sites we chose for opinion or review). The participants drew on their past experience in
making their credibility assessments, especially when it came to search engine sites like Google
and Yahoo! that people had used previously. We caution readers about interpreting the scores for
this topic too strictly, since more than any other comment category, when it comes to past
experience, the results would have been different had we selected other sites for this study.
15. Information Clarity – 3.7% Overall

In 3.7 percent of the comments, people in this study addressed the clarity of information (or the
extent to which the information was easily understood) when evaluating a site’s credibility.
Sample comments are below:
• Clear, concise information on home page—tells you what you need to know right away
in an up-front manner. —F, 51, Australia
• Easy to understand. . . . I felt comfortable reading and understanding the information

presented. —F, 33, New Jersey
• Very wordy and vague information. —F, 50, California
Information Clarity comments, by site category

Figure 20 shows that a mere 3.7 percent of all site evaluations included comments relating to
information clarity—that is, whether or not a site’s content is clear and understandable. This
occurred slightly more frequently in the finance (6.6%), health (6.0%), and search engines (4.2%)
categories, and slightly less frequently in the nonprofit and sports categories (1.9%, each), as well
as in the opinion or review (1.7%) and entertainment (1.6%) categories.

to Information Clarity
Finance 6.6%
Health 6.0%
Search Engines 4.2%
All Sites 3.7%
3.6%
Category
Travel
News 3.5%
E-C ommerce 2.2%
Sports 1.9%
Nonprofit 1.9%
Entertainment 1.6%
0.0% 1.0% 2.0% 3.0% 4.0% 5.0% 6.0% 7.0%
Figure 20: Percentage of comments related to information clarity, by category.
What we find notable about Information Clarity

We suspect that issues of information clarity figure more prominently into credibility evaluations
than the data show here. In our view, presenting information clearly is as important to a positive
user experience as having a good information structure, both of which contribute positively to
credibility evaluations. The relatively small percentage of comments about information clarity
could be due to a participant’s not wanting to admit he or she didn’t understand the information
on the site (people don’t like to admit they can’t understand something). If these numbers are any
indication, people were most confused by information on finance and health Web sites and were
least confused by information on entertainment and opinion or review sites. Although the
differences between the extremes is small (6.6% for finance sites vs. 1.6% for entertainment
sites), this result makes sense, since one might expect entertainment sites to have the simplest
information and finance sites to have the most complicated.
16. Performance on Test by User – 3.6% Overall

In relatively few cases, people devised their own tests in order to evaluate the credibility of a Web
site in this study (e.g., performing a search on the site). The results of the test helped them form
an assessment of the site credibility. Tests of this nature showed up in 3.6 percent of the
comments, some samples of which are below:
• Had more credible hits when searching for biogeochemical data. —M, 55, Tennessee
• I did not find hypothyroidism or thyroiditis on the Web site despite the commonality of
the disease. —F, 41, New York
Performance on a Test by User comments, by site category

As Figure 21 shows, less than four percent (or 3.6%) of all site evaluations referred to issues of
performance testing—that is, short tests performed by consumers that helped them to evaluate a
site’s credibility. Not surprisingly, this occurred with greater frequency in areas in which speedy
access to accurate information is perhaps more important (search engines: 13.8%; travel: 8.6%),
and occurred with less frequency in areas in which timely delivery of information is of less
concern (health: 3.4%; news: 2.2%; opinion or review: 1.7%). Consumers did not have access to
active brokerage accounts for each finance site for the purposes of this study, which may account
for the very low percentage of consumers who performed such tests in this category.

to Performance on Test by User
Travel 8.6%
Sports 4.8%
Entertainment 4.7%
All Sites 3.6%

Category
Health 3.4%
E-C ommerce 2.6%
News 2.2%
Nonprofit 1.0%
Finance 0.5%
0.0% 2.0% 4.0% 6.0% 8.0% 10.0% 12.0% 14.0% 16.0%
Figure 21: Percentage of comments related to performance on a test, by category.
What we find notable about Performance on Test by User

Although surface features tended to dominate many participants’ responses, search engines and
travel sites (where a user could go through the steps to set up a mock flight) allow the user to
make assessments based on a simple test. It is not clear from the study how far users will
typically go to test the credibility of a site in order to make an important decision (as opposed to
making a decision to satisfy the requirements of a study). The content of the comments suggested
that at least some people were quite familiar with testing a site in order to assess the site’s
credibility.
17. Readability – 3.6% Overall

A small number of people’s comments, 3.6 percent of the total, mentioned the readability of the
Web site as a credibility issue. Sites that were unreadable—for whatever reason—lost credibility
points. Sample comments are below:
• The format was easier for me to read. —M, 52, Pennsylvania
• The page is not easily readable. The font “Courier” contributed to this.—M, 40, Austria
Readability comments, by site category

As illustrated in Figure 22, less than four percent (3.6%) of all site evaluations referred to
comments about site readability—that is, how easy or hard it was for a user to read a site’s Web
pages. This occurred more frequently in the sports (5.7%), search engines and entertainment
categories (3.8%, each), and less frequently in the news (3.0%), opinion or review (2.8%), and e-
commerce (0.4%) categories. Nonprofit sites did not provoke a single comment related to
readability issues.
Related to Readability
Sports 5.7%
Search Engines 3.8%
Entertainment 3.8%
All Sites 3.6%
Health 3.5%
Category
Travel 3.2%
Finance 3.2%
News 3.0%
E-Commerce 0.4%
Nonprofit 0.0%
0.0% 1.0% 2.0% 3.0% 4.0% 5.0% 6.0%
Figure 22: Percentage of comments related to readability, by category.
What we find notable about Readability

Not all Web sites have excellent visual design or information structure. That’s understandable,
since these qualities require talent and hard work. However, one would expect virtually all Web
sites to have readable text. The data show that this was not always the case. In some situations
people had trouble simply reading the site’s content, frustrating users and damaging the site’s
credibility.
18. Affiliations – 3.4% Overall

In 3.4 percent of the comments, people claimed a site won credibility by showing an affiliation
with an organization they knew and trusted. Below are sample comments:
• Affiliation with a prestigious university adds to a sense of objectivity. —F, 27, California
• Credibility increased by seals of approval from known companies. —F, 21, Virginia
Affiliations comments, by site category

As shown in Figure 23, financial affiliations, support, and partnerships did not affect site
evaluators very much as they assessed the credibility of the sites in this study. A mere 3.4 percent
of all evaluations included comments about this issue. This occurred slightly more frequently in
the nonprofit (7.2%), health (5.6%), and search engines (4.8%) categories, and slightly less
frequently in the travel (3.3%), finance (2.7%), e-commerce (2.6%), and news (1.7%) categories.
Related to Affiliations
Nonprofit 7.2%
Health 5.6%
Search Engines 4.8%
All Sites 3.4%
Travel 3.3%
Category
Sports 2.8%
Finance 2.7%
E-Commerce 2.6%
News 1.7%
Entertainment 1.6%
0.0% 1.0% 2.0% 3.0% 4.0% 5.0% 6.0% 7.0% 8.0%
Figure 23: Percentage of comments related to affiliations, by category.
What we find notable about Affiliations

Although few comments mentioned issues of affiliation, it’s interesting to see that nonprofit Web
sites led the pack in these types of comments, followed by health sites. In these and other
categories the comments showed that at least some people used a site’s affiliation—good or
bad—as a cue about whether or not the site was credible. An odd result here is that news sites
generated so few comments about affiliation, suggesting that people in our study didn’t view
affiliation of news sites as a credibility issue
Overall discussion on comments

While this report could discuss the findings about the overall comments at length, we will focus
on two areas in this concluding discussion: (1) why design look is so prominent, and (2) what
topics may have been overlooked in this analysis.
Why is Design Look so prominent?

The results of this research show that the design look of Web sites was clearly the most
prominent issue when people evaluated Web site credibility in this study. Almost 50 percent of
comments about Web credibility contained something about the look of the site, either in general
(e.g., “looks professional”) or in specifics (the layout, the colors, etc.). The dominance of design
look may be surprising at first. One might ask, ‘Are people really so influenced by design look
and not by more substantial issues?’ The answer appears to be yes—at least in this setting.
It’s important to note that looking good is often interpreted as being good—and being credible.
Since at least the 1940s, social psychology research has shown that physically attractive sources
(usually people) have been perceived to be credible sources (Benoy, 1982; Berscheid, 1981;
Berscheid & Walster, 1974; Dion, Berscheid, & Walster, 1972; Eagly, Ashmore, Makhijani, &
Longo, 1991). This basic human processing bias— “looking good is being good” —also seems to
hold true for evaluating the credibility of Web sites, especially since design look is highly
noticeable.
The research context is another factor that likely contributed to the overwhelming prominence of
design look as a rationale for determining site credibility. Because people participated in this
study to earn a donation for a nonprofit organization—not because of a deep personal interest or
need—they did not likely have the motivation to process the Web sites deeply. According to the
Elaboration Likelihood Model (ELM) (Petty & Cacioppo, 1986), without deep motivation, people
will rely on peripheral cues, such as appearance, for making assessments. The ELM would
predict that if the participants had both the ability and the motivation to scrutinize these sites
carefully, the percentages in this study would change, with peripheral cues playing a less
significant role.
Although people in this study were probably not deeply involved in the evaluation task, this is not
a fatal flaw in the research. Our results are still valid. One could argue that people typically
process Web information in superficial ways, that using peripheral cues is the rule of Web use,
not the exception (for empirical research supporting this point, see Cockburn & McKenzie, 2001).
From a user perspective there are many sites available, with the next piece of information just one
click away. Even the words people use to describe Web use—”visiting sites” and “surfing the
Web”—suggest lightweight engagement, not deep content processing. Research has yet to
examine the relationship between engagement level and credibility assessments of Web sites.
An important follow-up study would be to manipulate the engagement level of the participants
(e.g., finding health information for a loved one in dire need) and see how the comments about
credibility change. Studies along these lines could show how involvement level affects what
people notice when evaluating a site’s credibility. Our hypothesis is this: Even for highly
involved Web surfers, design look would still play a role in credibility, though it would be less
dominant in overall evaluations.
The high value for design look is also due to the coding categories themselves. Design look may
be the broadest category, causing many Web site elements to be coded as design look. In a future
analysis of the comments, dividing the design look category into more focused categories could
be illuminating. We suspect that some interesting findings are still concealed in the data because
of the breadth of this category.
Areas that did not emerge in this research

We would be misleading readers if we implied that our analysis of the comments has brought to
light all the interesting issues in our data. In all likelihood, our analysis has missed some key
findings. For example, as the previous paragraph notes, our analysis categorized many issues into
the design look category. A subsequent analysis could further refine this category to give
additional insight into different aspects of design look.
Although we suspect other categories are also hiding key insights, it’s difficult to pinpoint where
the hidden insights are. We certainly have some suspicions about where to reanalyze the
comments. Shown to be important in other research (Fogg et al., 2001; Fogg et al., 2002), the
timeliness of a site’s information did not surface as an important issue in this study. It’s possible
that comments about timeliness were missed because we initially had no category for coding this
issue, or that these comments were coded as information accuracy (current information = accurate
information). Another area that likely exists in the data but that did not surface in our analysis is
that of information source—providing citations and references to show that the site content came
from an expert source. The issue of information source proved to be important in the expert study
performed in tandem with our study;vi the analysis for the tandem research showed that experts
are much more tuned in to the source of information than are consumers.
Although our analysis probably did not reveal all issues related to the credibility of the sites in
this study, there were topics we looked for in our analysis but did not find. For example, we
vi
Stanford, J., Tauber, E., Fogg, B., & Marable, L., (2002). Experts vs. Online Consumers: A Comparative Credibility
Study of Health and Finance Web Sites. Available online at www.consumerwebwatch.org.
coded the data for comments about privacy policy, and we found that people mentioned privacy
policy in less than 1 percent of the comments. We also looked for comments about correcting
false or misleading information and found no comments along these lines. These two issues
apparently did not affect how our participants evaluated the credibility of Web sites in this study.
Part 2: Results & Discussion – A Focus on Individual Web Site Categories
In this part of our Results & Discussion, we change our focus away from the overall comments in
order to focus specifically on the site categories (health, news, etc.). Some of the information for
this analysis may be gleaned by examining various charts in Part 1, but to do this readers would
need to flip back through the report and compare charts on various pages. We hope to make this
task easier by presenting the results one category at a time.
In the paragraphs that follow, we present data about how comments in one particular site category
differed significantly from the comments from all the categories. For example, we point out how
people’s comments on the e-commerce sites differed from people’s comments on all 100 sites.
After we present what we see as notable differences in the comments (those that are more than 5
percent), we then present a new type of data: the credibility rankings of the sites.
A word about the credibility-ranking results

This study was designed to generate a credibility ranking for Web sites within each category. As
discussed in the Methods section, these rankings may be interesting and provocative, but they are
not definitive credibility assessments for an individual Web site. A credible Web site could end
up with a low ranking if other sites in the same category were extremely credible sites. In other
words, the ranking for a site depended entirely on the competition; if the competition were
different, then the ranking would be different. Also, because some content categories were
undoubtedly more competitive than others, the ranking score for Web sites cannot be compared
across categories. And of course, these rankings reflect the views of those who happened to
participate in this study.
As the tables below show, each site was evaluated at least 35 times and some sites as many as 89
times (total number of rankings: 5,242, from 2,684 completions of the study). Sites in the health
and finance categories were ranked more often, since a study parallel to this one4 focused on
these two categories and we wanted to provide sufficient data for that project.
For each of the 10 tables, the first column represents the credibility ranking for the sites, with the
site listed #1 as being the most credible. The value in the “average score” column is the mean
score that the site received over the course of the study, as described in the Methods section. It
was this score that determined a site’s ranking.
The discussion below does not use the credibility comments to explain the ranking results (or vice
versa), because we do not want people to read more into these rankings than is warranted. Our
intent in the coming pages is to focus on a single Web site category and share two types of results
that relate to that category. At times, links between the comments and rankings seem obvious; at
other times the connection is not immediately apparent.
That said, we’ll begin presenting and discussing the results for each Web site category one by
one, in the following order:
• E-Commerce
• Entertainment
• Finance
• Health
• News
• Nonprofit
• Search Engines
• Sports
• Travel
E-Commerce Web Sites: Credibility Results

This section presents what we learned about the credibility of e-commerce Web sites, starting
with presenting our results about how the credibility comments for e-commerce sites differed
from the overall averages, and then presenting the final credibility rankings of the 10 e-commerce
sites.
Credibility comments about e-commerce Web sites: What stands out?

The data in our study show that e-commerce sites stand out from other types of sites in four ways.
First, issues of name recognition and reputation were mentioned more often by people evaluating
e-commerce sites, compared to the overall mean for this type of comment (25.9% compared to
14.1% overall). Next, comments about customer service were also relatively more frequent
(16.7% compared to 6.4%). On the other side of the scale, compared to the overall mean, e-
commerce sites received far fewer comments about information bias (2.6% vs. 11.6%). One final
difference, not discussed earlier in this report, has to do with comments that our coders classified
as “general suspicion.” These comments would include things like “It just makes me suspicious,”
and “I’m always wary of these type of sites.” These types of comments appeared in 9.4 percent
of all the comments in this study. The “general suspicion” comments appeared more often when
people evaluated e-commerce sites, in 17.2 percent of the comments people made.
Taken together, the data show that when people evaluated the credibility of e-commerce sites,
they more frequently mentioned issues of customer service and name recognition and reputation
as indicators of whether or not a site was credible. In addition, people approached e-commerce
sites with more suspicion than other sites in this study.
Credibility rankings of e-commerce Web sites

Table 2 shows the final ranking of e-commerce sites in this study.
E-Commerce Web Sites

# of
Final Average
Web Site Name URL Times
Ranking Score
Ranked
1 Amazon http://www.amazon.com +0.42 69
2 Barnes & Noble http://www.bn.com +0.42 55
3 McMaster-Carr http://www.mcmaster.com +0.23 44
4 Cars.com http://www.cars.com +0.18 44
5 eBay http://www.ebay.com +0.11 56
6 Best Buy http://www.bestbuy.com +0.07 73
7 Buy.com http://www.buy.com -0.04 71
8 Dogwise http://www.dogwise.com -0.11 54
9 MTE Nutrition http://www.mtenutrition.com/index.html -0.52 54
10 ThymuSkin http://www.thymuskin.com -0.61 72
+0.50 = judged as more credible in 75% of pairings
What average scores mean 0.0 = judged as more credible in 50% of pairings
-0.50 = judged as less credible in 75% of pairings
Table 2: Ranking and scores for e-commerce Web sites.
What’s notable about e-commerce site rankings?

The scores and rankings for the e-commerce sites led to three observations from our team. First,
big-name players Amazon and Barnes & Noble led the credibility rankings, in nearly a dead heat
for the top spot (the difference in average scores is less than .01). The top placement of these two
sites is not surprising, given the previous experience many people have had in dealing
successfully with these companies (what has been called “earned credibility”). The surprise
comes in seeing who landed in third place: McMaster-Carr—a relative unknown entity compared
to others who ranked high in this category. The McMaster-Carr site has relied on its own
merits—not on a public relations campaign or a big reputation—in conveying high levels of
credibility. This should be good news for people who want to set up an e-commerce site, or any
other kind of site, but are faced with competitors who own big brand names and wield significant
marketing budgets. At least when it comes to evaluations of Web credibility, the high ranking of
McMaster.com shows that a site can win credibility on its own merits.
The second notable result is the mediocre ranking of eBay, a bit of a puzzling outcome given that
eBay is widely known and used.
Our third observation is that two e-commerce sites did quite poorly in the credibility rankings:
MTE Nutrition and ThymuSkin. When it comes to establishing Web credibility, these sites might
serve as models about what not to do.
Entertainment Web Sites: Credibility Results

This section presents what we learned about the credibility of entertainment Web sites, starting
with our results about how the credibility comments for entertainment sites differed from the
overall averages, and then presenting the final credibility rankings of the 10 entertainment sites.
Credibility comments about entertainment Web sites: What stands out?

The data tell quite a different story for entertainment sites than for e-commerce sites. In three
areas entertainment sites differed substantially from the overall averages. First, when evaluating
the credibility of entertainment sites, people commented on company motive relatively less
frequently (9.4% vs. 15.5% overall). Next, people also commented much less often about the
identity of site operator (2.6% vs. 8.8%). Finally, people never commented about the customer
service of an entertainment site when evaluating credibility (0% vs. 6.4%).
These data do not say that motive, identity, and customer service do not contribute to the
credibility assessment of entertainment sites. The data simply show that these things may matter
less than for other types of Web sites. In general it could be that the credibility expectations for
entertainment Web sites are lower than for other types of sites. This makes sense, since
entertainment issues are rarely as important to people as issues involving their money or health.
Credibility rankings of entertainment Web sites

Table 3 shows the final ranking of entertainment sites in this study.
Entertainment Web Sites

# of
Final Average
Ranking Score
Ranked
1 Citysearch http://www.citysearch.com +0.58 38
2 Entertainment Weekly http://www.ew.com +0.49 43
Internet Movie
3 http://www.imdb.com +0.38 42
Database
4 Yahoo! Entertainment http://entertainment.yahoo.com +0.18 44
5 Fandango http://www.fandango.com +0.17 48
6 E! Online http://www.eonline.com +0.06 47
7 Moviefone http://www.moviefone.com -0.21 48
8 ChristianAnswers.net http://christiananswers.net/spotlight -0.23 39
9 Ain’t it Cool News http://www.aint-it-cool-news.com -0.50 44
10 Pazsaz.com http://www.pazsaz.com -0.76 49
Table 3: Ranking and scores for entertainment Web sites.
What’s notable about entertainment-site rankings?

Three entertainment Web sites clustered at the top of the credibility rankings: Citysearch,
Entertainment Weekly, and Internet Movie Database. While the comments reveal that
Entertainment Weekly was rated as credible largely for its brand recognition (thanks to the
magazine of the same name), the other two high-ranking entertainment sites were viewed as
credible because of their functionality. Citysearch and Internet Movie Database provided study
participants with reliable information about local events and movies.
The other notable finding in this category is the extremely low score for Pazsaz.com, ranking
even lower than Ain’t It Cool News, a site that we hypothesized would end up being evaluated as
the least credible in the entire study. We were wrong: Pazsaz.com ranked even lower. And the
comments from the participants tell the story: Often the Pazsaz.com site failed to load, giving
users an error instead of content, making it easy for people to rank Pazsaz as less credible than the
site it was paired against. This result indicates that Web users do not forgive this type of technical
failure when evaluating credibility.
Finance Web Sites: Credibility Results

This section presents what we learned about the credibility of finance Web sites, starting with our
results about how the credibility comments for finance sites differed from the overall averages,
and then presenting the final credibility rankings of the 10 finance sites.
Credibility comments about finance Web sites: What stands out?

Five areas in the data suggest how people may evaluate the credibility of finance Web sites
differently from other site categories in our study. When evaluating finance sites, people
commented relatively less often on information focus (18.9% vs. 25.1% overall), information
accuracy (8.0% vs. 14.3%), and advertising (8.3% vs. 13.8%). However, people commented
more frequently in three areas: design look (54.6% vs. 46.1%), company motive (21.0% vs.
15.5%) and name recognition and reputation (21.8% vs. 14.1%).
Taken together, the credibility landscape for finance sites shows that people focused relatively
more on issues of trustworthiness (motives and reputation) and focused relatively less on issues of
expertise. It may be that people in our study did not have the expertise to evaluate issues of
information focus or accuracy of finance sites, so they relied more heavily on other areas:
perceived company motivation and reputation and design look.
Credibility rankings of finance Web sites

Table 4 shows the final ranking of finance sites in this study.
Finance Web Sites

# of
Final Average
Ranking Score
Ranked
1 Schwab http://www.schwab.com +0.57 89
2 Fidelity http://www.fidelity.com +0.28 67
3 ShareBuilder http://www.sharebuilder.com +0.20 75
4 Merrill Lynch http://www.ml.com +0.15 75
Domini Social
5 http://www.domini.com +0.12 68
Investments
6 E-Trade http://us.etrade.com/e/t/home -0.16 76
7 Scottrade http://www.scottrade.com/ -0.20 80
8 ChoicePicks http://www.choicepicks.com -0.22 64
Christian Brothers
9 http://www.cbis-fsc.com/index.asp -0.39 72
Investment Services
10 Stocks at Bottom http://www.stocksatbottom.com/ -0.43 80
Table 4: Ranking and scores for finance Web sites.
What’s notable about finance-site rankings?

Schwab emerged as the clear credibility leader in rankings of the finance category, with a final
score significantly better than the #2 site, Fidelity.
In our view, however, the real story in the finance category is the site that came in at number
three: ShareBuilder. What is ShareBuilder? And how did its Web site compete successfully
against sites with big brand names? As in the case of McMaster-Carr in the e-commerce category,
the success of ShareBuilder suggests that a Web site can be designed to win credibility on its own
merits. While not the focus on this report, an analysis of the ShareBuilder.com site leads our
research team to propose this site is doing many things right to build credibility, from disclosure
issues to design details. At #3 in the rankings, the ShareBuilder Web site may represent the most
significant credibility achievement in the finance category.
Health Web Sites: Credibility Results

This section presents what we learned about the credibility of health Web sites, starting with our
results about how the credibility comments for health sites differed from the overall averages, and
then presenting the final credibility rankings of the 10 health sites.
Credibility comments about health Web Sites: What stands out?

When it comes to health Web sites, the data in our study show that this category stands out in four
ways. First, when evaluating the credibility of health sites, people commented much more often
on issues of information focus (33.0% vs. 25.1% overall) and information usefulness (20.5% vs.
14.8%). Looking more deeply at the data (not presented in this study), we find that these
additional comments tended to be positive. In other words, there were a substantial number of
comments saying that health Web sites won credibility by providing information that people
found to be useful and to have a good focus. The data also show that people commented more
often about advertising on health Web sites, compared to people who evaluated sites in other
categories (21.3% vs. 13.8%). But customer service on health Web sites was rarely mentioned
(0.4% vs. 6.4%).
When viewed as a whole, the data suggest that people evaluating the credibility of health Web
sites pay relatively more attention to the focus and usefulness of the information. What people
find in these areas apparently becomes a significant indicator of the site’s credibility.
Credibility Rankings of Health Web sites

Table 5 shows the final ranking of health sites in this study.
Health Web Sites

# of
Final Average
Ranking Score
Ranked
1 MayoClinic.comvii http://www.mayohealth.org +0.54 61
2 Intelihealth http://www.intelihealth.com +0.44 54
3 NIH http://www.nih.gov +0.43 42
4 MDChoice http://www.mdchoice.com +0.07 73
5 Dr. Koop http://www.drkoop.com +0.03 60
6 WebMD http://www.webmd.com 0.00 50
7 HealthWorld Online http://www.healthy.net -0.36 47
8 Dr. Weil http://www.drweil.com -0.37 51
9 Health Bulletin http://www.healthbulletin.org -0.47 49
Oxygen.com Health
10 http://www.oxygen.com/health -0.51 45
and Fitness
What Average Scores Mean 0.0 = judged as more credible in 50% of pairings
Table 5: Ranking and scores for health Web sites.
What’s notable about health site rankings?

In the current study, three Web sites cluster at the top of the Web credibility rankings:
MayoClinic.com, Intelihealth, and NIH—all highly credible sites. While there may be no
practical difference in credibility among these three top health sites, the sites in this top cluster
are clearly viewed as more credible by our participants than the other seven sites in our study.
These three sites lead the rest of the Web sites by a significant margin.
On the other end of the credibility spectrum, four Web sites cluster at the bottom. However,
what’s more notable is how a third cluster of high-profile dot-com Web sites—MDChoice, Dr.
Koop, and WebMD—landed squarely in the middle of the rankings, viewed as neither high nor
low in credibility compared to the other sites in the study. From a credibility standpoint, the
approach these three companies have taken is not working as well as the approach by Intelihealth.
vii
To distinguish from MayoClinic.org, which is the Web site for the Mayo Foundation and its associated clinics and
hospitals, we refer to MayoClinic.com with the .com succeeding its name throughout this document.
News Web Sites: Credibility Results

This section presents what we learned about the credibility of news Web sites, starting with our
results about how the credibility comments for news sites differed from the overall averages, and
then presenting the final credibility rankings of the 10 news sites.
Credibility comments about news Web sites: What stands out?

News is a Web site category that differed from the overall averages in many areas, suggesting
that people evaluate the credibility of news sites quite differently than other types of sites. The
data show that issues relating to information received relatively more attention when people
evaluated sites in the news category, compared to the overall averages: information focus (31.9%
vs. 25.1% overall), information accuracy (21.7% vs. 14.3%), and—most dramatic of all—
information bias (30.2% vs. 11.6%). Other areas that figured more prominently in people’s
comments about news sites were issues relating to name recognition and reputation (19.1% vs.
14.1%) and writing tone (14.8% vs. 9.0%). Some areas received relatively less attention when
people evaluated the credibility of news Web sites. Design look was mentioned less often (39.6%
vs. 46.1%), and issues relating to customer service were rarely mentioned (0.8% vs. 6.4%).
Interestingly enough, compared to the overall averages, news Web sites received far fewer
comments about underlying company motive (5.9% vs. 15.5%) and fewer comments that
conveyed general suspicion (4.3% vs. 9.4%).
Viewed as a whole, the credibility picture for news sites is intriguing—and perhaps contradictory.
On one hand people evaluating news sites seem quite tuned in to issues of information bias. On
the other hand, people tended not to comment on issues relating to underlying company motive,
an area that would seem to be related to information bias. Perhaps in evaluating the credibility of
these sites, people could have easily commented on information bias but found it harder to
pinpoint or articulate the motives behind the information, leading to fewer comments of this type.
Credibility rankings of news Web sites

Table 6 shows the final ranking of news sites in this study.
News Web Sites

# of
Final Average
Ranking Score
Ranked
1 The New York Times http://www.nytimes.com +0.53 59
2 Yahoo! News http://news.yahoo.com +0.44 50
3 CNN http://www.cnn.com +0.42 55
4 Time http://www.time.com +0.10 49
5 MSNBC http://www.msnbc.com -0.06 47
6 Crosswalk.com http://news.crosswalk.com -0.19 47
7 SF Gate http://www.sfgate.com -0.27 52
8 Drudge Report http://www.drudgereport.com -0.30 57
9 Telluride Gateway http://www.telluridegateway.com -0.36 47
10 Workers World http://www.workers.org/ww -0.45 47
Table 6: Ranking and scores for news Web sites.
What’s notable about news site rankings?

In the news sites credibility rankings, three sites scored much higher than the rest: The New York
Times on the Web, Yahoo! News, and CNN. The surprise here is Yahoo! News. In our opinion,
people often consider The New York Times to be a leader in print news and CNN a leader in TV
news. But Yahoo! is neither based in print nor on TV, and it’s not primarily a news organization.
For these reasons and more, it’s remarkable how well Yahoo! scored, probably winning
credibility points because of the strength of its online brand and the diversity of its news site
offerings.
For our research team, another intriguing result is how poorly MSNBC fared—ending with a
negative credibility score. The high-profile MSNBC is most closely ranked to Crosswalk.com, a
small news site with a religious bias.
Nonprofit Web Sites: Credibility Results

This section presents what we learned about the credibility of nonprofit Web sites, starting with
our results about how the credibility comments for nonprofit sites differed from the overall
averages, and then presenting the final credibility rankings of the 10 nonprofit sites.
Credibility comments about nonprofit Web sites: What stands out?

The data tell an interesting story about nonprofit Web sites, since the comments about these sites
differ from the overall comments in provocative ways. What did people focus on when evaluating
the credibility of nonprofit Web sites? They focused more on the identity of the site operator
(28.9% vs. 8.8% overall) and the underlying company motive (20.2% vs. 15.5%). In addition,
more than for any other site category, people evaluating the credibility of nonprofit sites made
comments that were coded as “general suspicion,” suggesting that the level of suspicion for
nonprofit sites was far above that of any other type of Web site (22.6% for nonprofit vs. 9.4%
overall).
While nonprofit sites are under a high level of scrutiny for trustworthiness, the data suggest they
are not held to high standards in other areas. People evaluating the credibility of nonprofit sites
commented less frequently on issues of design look (39.4% vs. 46.1%), information
design/structure (18.2% vs. 28.5%), and information focus (17.8% vs. 25.1%).
Taken together, the data suggest that when people evaluate the credibility of nonprofit Web sites,
they focus relatively more on issues of who is behind the site and relatively less on how
competent the Web site is in terms of design or information. The data also suggest that despite
having been recruited through nonprofit organizations, people in this study were rather suspicious
of the Web sites they found in this category.
Credibility rankings of nonprofit Web sites

Table 7 shows the final ranking of nonprofit sites in this study.
Nonprofit Web Sites

# of
Final Average
Ranking Score
Ranked
1 American Red Cross http://www.redcross.org +0.72 36
2 Aim High http://www.aimhigh.org +0.23 39
3 Liberty Unites http://www.libertyunites.org +0.17 58
4 MISS Foundation http://www.missfoundation.org +0.14 49
5 Compassion http://www.compassion.com +0.08 48
6 Save the Earth http://www.savetheearth.org +0.05 42
7 The Hunger Site http://www.thehungersite.com -0.06 47
Comic Book Legal
8 http://www.cbldf.org -0.29 45
Defense Fund
9 DogScouts http://www.dogscouts.com -0.32 44
World Peace
10 http://www.worldpeaceendowment.org -0.70 40
Endowment
Table 7: Ranking and scores for nonprofit Web sites.
What’s notable about nonprofit site rankings?

The American Red Cross scored much higher on credibility than other Web sites in the nonprofit
category, dominating its category more than any other site in the entire study. In retrospect,
including the American Red Cross may have been a mistake, because its domination could have
suppressed other interesting results. With the exception of how poorly the World Peace
Endowment was rated on credibility (another example of what not to do), there is nothing
particularly remarkable about the credibility scores of the remaining eight nonprofit Web sites.
Opinion or Review Web Sites: Credibility Results

This section presents what we learned about the credibility of opinion or review Web sites,
starting with our results about how the credibility comments for opinion or review sites differed
from the overall averages, and then presenting the final credibility rankings of the 10 opinion or
review sites.
Credibility comments about opinion or review Web sites: What stands out?
Compared to the overall averages, people commented more frequently on three issues when
evaluating the credibility of the opinion/review sites: information bias (23.8% vs. 11.6% overall),
information accuracy (25.4% vs. 14.3%) and, to a lesser extent, underlying company motive
(22.1% vs. 15.5%). People commented less frequently about the design look of opinion or review
Web sites (38.1% vs. 46.1). Curiously enough, we found no comments about the name
recognition and reputation of these sites (0% vs. 14.1%), even though we included at least one big
brand name in this category: Epinions.com. This last finding is the most striking in this category.
Credibility rankings of opinion or review Web sites

Table 8 shows the final ranking of opinion or review sites in this study.
Opinion or Review Web Sites

# of
Final Average
Ranking Score
Ranked
1 Consumer Search http://www.consumersearch.com +0.44 39
2 Epinions http://www.epinions.com +0.31 52
3 Students Review http://www.studentsreview.com +0.28 36
4 E-complaints http://www.ecomplaints.com +0.26 35
5 Opinion-Pages http://www.opinion-pages.org +0.16 38
6 BizRate http://www.bizrate.com -0.19 37
7 All Star Review http://www.allstarreview.com -0.20 40
8 Review Centre http://www.reviewcentre.com -0.29 45
The Opinionated Beer
9 http://www.tobp.com -0.32 41
Page
10 CardOffers.com http://www.cardoffers.com -0.41 41
Table 8: Ranking and scores for opinion or review Web sites.
What’s notable about opinion or review site rankings?

While most of the other nine categories offer surprises, confirmations, or insights, the results of
the opinion or review sites are largely unremarkable. Unlike the sites in other categories, the sites
here are fairly evenly distributed along the credibility scale, with no notable standouts at the top
or bottom of the heap. Other than having a big brand name Epinions—end up in second place
rather than first, there is nothing particularly notable about the scores or rankings in this category.
Search Engine Web Sites: Credibility Results

This section presents what we learned about the credibility of search engine Web sites, starting
with our results about how the credibility comments for search engine sites differed from the
overall averages and then presenting the final credibility rankings of the 10 search engine sites.
Credibility comments about search engine Web sites: What stands out?
As with news sites, the results for search engine sites differed from the overall averages in many
areas – eleven areas, to be precise. When evaluating the credibility of search engine Web sites,
people commented relatively more often about design look (52.6% vs. 46.1% overall),
information design/structure ( 42.6% vs. 28.5%), performance on a test by user (13.8% vs.
3.6%), site functionality (20.5% vs. 8.6%), advertising (24.6% vs. 13.8%), and past experience
with the site (12.8% vs. 4.6%). On the low end of the scale, people evaluating search engine sites
commented less often on information bias (3.8% vs. 11.6%), information accuracy (7.1% vs.
14.3%), name recognition and reputation (5.1% vs. 14.1%), and customer service (1.0% vs.
6.4%). A further point of distinction for search engine sites is that this category received the
fewest comments coded as “general suspicion” (2.8% vs. 9.4%).
The overall picture for search engines is a complicated one, at least compared to the average
findings for all the categories. For most issues, the percentage of comments differed by more than
5 percent with the average values. It seems clear that people are bringing different expectations to
each Web site category, and that the search engine category seems to be the most unusual
category of the 10.
Credibility rankings of search engine Web sites

Table 9 shows the final ranking of search engine sites in this study.
Search Engine Web Sites

# of
Final Average
Ranking Score
Ranked
1 Google http://www.google.com +0.55 44
2 Yahoo! http://www.yahoo.com +0.24 45
3 About.com http://www.about.com +0.17 36
4 LookSmart http://www.looksmart.com +0.09 53
5 Vivisimo http://www.vivisimo.com +0.08 37
6 Ask.com http://www.ask.com +0.04 48
7 Insider.com http://www.insider.com -0.08 48
8 All the Web http://www.alltheweb.com -0.19 54
9 Overture http://www.overture.com -0.19 52
10 iWon.com http://www.iwon.com -0.63 43
Table 9: Ranking and scores for search engine Web sites.
What’s notable about search engine site rankings?

In the search engine category, the clear credibility leader was Google, followed by Yahoo! in a
distant second place. It’s peculiar that Yahoo!’s outcome in the search engines category is
arguably worse than its outcomes in other categories in this study: travel, news, and sports.
In addition, we found it somewhat surprising how poorly Overture.com was viewed, given that
this site readily discloses information about its corporate operations, including costs of
advertising.
On the bottom of the ranking list in search engines is iWon.com, which scored far below any
other site. Besides lagging so far behind the rest of the pack, iWon’s low ranking is notable
because this site is also reported to be popular,5 demonstrating how in some situations popularity
may not always correlate with high credibility (in other media, the distinction between popularity
and perceived credibility is notable in properties like the National Enquirer and the Jerry
Springer Show).
Sports Web Sites: Credibility Results

This section presents what we learned about the credibility of sports Web sites, starting with our
results about how the credibility comments for sports sites differed from the overall averages, and
then presenting the final credibility rankings of the 10 sports sites.
Credibility comments about sports Web sites: What stands out?

When it comes to sports Web sites in this study, two issues emerged as relatively more prominent
when evaluating credibility: information focus (30.8% vs. 25.1%) and advertising (22.7% vs.
13.8%). Information design was mentioned less often than average (22.3% vs. 28.5%), while
customer service was never mentioned when people evaluated the credibility of sports Web sites
(0% vs. 6.4%).
In general, these findings suggest that compared to other categories, sports Web sites win
credibility by focusing their site in ways people find pleasing and by not overloading the site with
advertising. Oddly enough, issues of information accuracy or bias were not particularly
prominent, perhaps because these topics are not perceived as key issues in sports information.
Credibility rankings of sports Web sites

Table 10 shows the final ranking of sports sites in this study.
Sports Web Sites

# of
Final Average
Ranking Score
Ranked
1 ESPN http://www.espn.com +0.53 51
2 Yahoo! Sports http://sports.yahoo.com +0.43 42
3 CNN Sports Illustrated http://sportsillustrated.cnn.com +0.30 46
4 Sports Network http://www.sportsnetwork.com +0.22 49
5 All Sports http://www.allsports.com +0.08 52
6 SportsJones http://www.sportsjones.com -0.06 49
7 iWon Sports http://sports.iwon.com -0.14 44
8 Stand-Up Sports http://www.standupsports.com -0.45 47
9 Xslant http://www.xslant.com -0.46 41
10 Sports.com http://www.sports.com/en -0.49 51
Table 10: Ranking and scores for sports Web sites.
What’s notable about sports site rankings?

The ranking list for sports Web sites shows two clusters, one at the top and one at the bottom.
Two sites are in the top cluster of credibility: ESPN and Yahoo! Sports. The surprise here—as
with news sites—is how well Yahoo! fared in the rankings. Neither a specialist in news nor
sports, Yahoo! has succeeded in creating an online sports offering that people find to be highly
credible, perhaps even more so than CNN Sports Illustrated.
On the bottom of the rankings are three Web sites—Stand-Up Sports, Xslant, and Sports.com—
all of which fared poorly in comparison to the credibility of other sites. Something about these
sites, perhaps their visual design or their lack of a name brand, is causing people to not view them
as credible sources of information.
Travel Web Sites: Credibility Results

This section presents what we learned about the credibility of travel Web sites, starting with our
results about how the credibility comments for travel sites differed from the overall averages, and
then presenting the final credibility rankings of the 10 travel sites.
Credibility comments about travel Web sites: What stands out?

What makes travel Web sites different from other sites when it comes to credibility evaluations?
The answer seems clear from the data: customer service. People who evaluated the credibility of
travel sites commented on customer service issues far more frequently than the overall average
(18.1% vs. 6.4% overall). On the other end of the spectrum, people evaluating the credibility of
travel sites commented least often on information bias (1.9% vs. 11.6%). Also below average
were the number of comments addressing issues of name recognition and reputation (8.8% vs.
14.1%).
Taken together, these findings show that a category such as travel can be at the extreme for one
quality (customer service) while being at the opposite extreme for another issue (information
bias). Although these differences are in part a result of the Web sites chosen for this study, we
also believe that when people evaluate the credibility of a Web site, they look for different things
in different types of sites. When it comes to travel, people focus relatively more on customer
service as an indicator of credibility, but as the previous section showed, when it comes to sports,
not a single person commented on customer service.
Credibility rankings of travel Web sites

Table 11 shows the final ranking of travel sites in this study.
Travel Web Sites

# of
Final Average
Ranking Score
Ranked
1 Yahoo! Travel http://travel.yahoo.com +0.43 67
2 Expedia http://www.expedia.com +0.32 56
3 Trip.com http://www.trip.com +0.24 63
4 GoNomad http://www.gonomad.com +0.10 58
5 Travel Zone http://www.thetravelzone.com +0.04 52
6 Hotwire http://www.hotwire.com +0.03 64
7 Orbitz http://www.orbitz.com -0.05 61
8 Getaway.com http://www.getaway.com -0.28 69
9 Priceline http://www.priceline.com -0.32 76
10 United Tours & Travel http://www.uttravel.com -0.37 70
Table 11: Ranking and scores for travel Web sites.
What’s notable about travel site rankings?

Our research team found three surprises in the credibility rankings for travel sites. First of all,
Yahoo! Travel received the highest credibility score in this category. This top placement and high
rankings for Yahoo! in other categories suggests that this brand-name company excels at
establishing credibility online.
The next surprise is the mediocre ranking of Orbitz, a Web offering backed by major airlines.
This site ended up being arguably less credible than GoNomad, a site focusing on alternative
travel. The takeaway message here is that an unknown yet well executed Web site can win more
credibility points than a site by established players in the industry.
The third outcome of interest is how poorly Priceline scored in this study, ranking almost at the
bottom. Although Priceline has lots of name recognition and a celebrity personality, the site failed
to convey credibility to people in this study, suggesting that something significant is damaging
Priceline’s credibility.
Concluding Discussion Focusing on Web Site Categories

We conclude this final part of the Results & Discussion section by addressing two issues. First,
we take a critical look at the rankings presented above. Next, we draw a general conclusion about
how site purpose affects credibility evaluations.
Concluding discussion about Web site rankings

Because the research methodology we use in this study is new, one question to ask when viewing
the site rankings is this: Do the Web credibility rankings produced by this study make sense? Our
answer is yes. As a research team we find that the rankings portray a believable picture of how
people view the credibility of the Web sites under study. Of course, there are surprises in the data,
which make a study like this interesting. But for the most part, the ranking results are in line with
what our team (experienced Web users and researchers) would expect to find.
The next question to ask is: How much would these credibility rankings change if our participant
demographics were different? As described earlier, our participants were mostly female (58%
compared to the U.S. average of about 51%6) and older than the U.S. average (39.9 years
compared to U.S. average of 35.57). Yet the most unusual difference about our participants was
their reported use of the Internet, which averaged almost 20 hours per week. This is more than
five times the weekly average reported by Nielsen/NetRatings in August of 2002.8
So how would the credibility rankings differ if our demographics were not skewed toward
slightly older females with considerable Web experience? The answer is impossible to determine
exactly. However, one can get a sense for the impact of our demographic skew— skewed toward
experienced older females—by imagining this study as having the opposite skew —younger
males with little Web experience. Which type of participant is likely to give better credibility
evaluations and contribute quality comments? For our research team, this is an easy choice: We’d
opt for the demographic we have in this study—slightly older females with considerable Web
experience. While the rankings may change with a younger, more male, or less experienced set of
participants, we believe the rankings produced in this study are likely to be more accurate.
One could ask other questions about these rankings, but our last one focuses on a practical issue:
What good are these rankings?
In our view, the best practical use of these rankings—at least for Web site designers—is to
compare the sites with the highest credibility to the sites with lowest credibility. These extreme
credibility differences seem undeniable, regardless of shortcomings in the study method or
participants. From a credibility standpoint, the user experience of the sites that ranked #1 or #2 in
a category is different from the user experience of sites that ranked at the bottom. As the
comments make clear, sometimes these differences have to do with established reputation, such
as in the case of MayoClinic.com or Schwab. But in other cases, most notably with
McMaster.com and ShareBuilder.com, the credibility was apparently won by Web site
performance alone. In sum, a practical use of the rankings is to examine the Web sites at the
extreme ends of the rankings to determine how the highly credible sites differ from sites ranked at
the bottom. This approach can lead to insights about what to do—and what not to do—when
designing for credibility.
While the rankings are interesting to view, we have downplayed their importance in this report
because our other data type—the comments gathered during this research—provide richer insight
as to how people evaluate the credibility of Web sites today.
How Web site topic influences credibility assessments

One overall conclusion from this study, something that has been hypothesized but never clearly
shown, is that people assess credibility differently for different types of Web sites. For example,
our data showed that at times the identity of the site operator mattered very much in determining
credibility (such as with nonprofit sites), and at other times identity mattered very little (as with
entertainment sites). Our data also showed that at times information bias figured prominently into
the credibility evaluations (as with news sites), but it didn’t seem to be prominent in evaluations
of other types of sites (such as travel). Although universal factors that affect Web site credibility
exist—such as design look and information design/structure—people in this study showed that
differences exist as well.
We need more research to understand these differences, as well as research to fully understand
the range of credibility variables that relates to a single category, such as travel Web sites or
search engine sites. The findings from these new studies will have implications not only for
people who design Web sites, but also for the mission of Consumer WebWatch, as it sets forth
additional guidelines for improving people’s experiences on the Web.
The fact that people notice different issues when evaluating site credibility, depending on the type
of site, leads into the next section of the report, which aims to provide readers with a deeper
understanding of how people evaluate credibility.
How to View This Study & Other Web Credibility

Research
To better understand the contributions of the research described in this paper and how these
findings complement previous work in this area, one must have a basic understanding of
Prominence-Interpretation Theory (P-I Theory) (Fogg 2002a).
In brief, P-I Theory posits that two things happen when people assess credibility: a person (1)
notices something (Prominence) and (2) makes a judgment about it (Interpretation). If one or the
other does not happen, then there is no credibility assessment. The process of noticing prominent
elements and interpreting them will typically happen more than once when a person evaluates a
Web site, with new aspects of the site being noticed and interpreted until the person reaches
satisfaction with an overall credibility assessment or reaches a constraint, such as running out of
time.viii
A summary of P-I Theory can be described as follows:
Prominence X Interpretation = Credibility Impact

An element’s likelihood of What value or meaning The impact that element has on
being noticed when people assign to element, credibility assessment.
people evaluate credibility. good or bad.
Affected by involvement Affected by user’s

(motivation & ability), assumptions (culture,
content, task, experience, experience, & heuristics),
individual differences & skill, knowledge, goals &
more. more.
Figure 24: A summary of Prominence-Interpretation Theory.
Previous research on Web credibility has investigated the Interpretation component of this theory
(Finberg, Stone & Lynch, 2001; Fogg, 2000b; Fogg & Tseng, 1999; Fogg et al., 2002; Fogg, Lee,
& Marshall, 2002; Fogg, Marshall, Kameda et al., 2001; Fogg et al, 2000; Fogg et al. 2001; Kim
& Moon, 1998; Princeton, 2002). For example, the study conducted by Princeton Survey
Research Associates and published by Consumer WebWatch (Princeton, 2002) is a study about
Interpretation. In this study, researchers contacted people via the phone and asked them to assess
the importance (their “interpretations”) of various elements of a Web site, such as knowing who
owns the Web site and having a privacy policy. Participants responded to each item, making a
value judgment of those items. The Stanford Persuasive Technology Lab has also done research
that addresses the Interpretation component in P-I Theory, using online questionnaires (Fogg et
al., 2001; Fogg et al. 2002). In these studies, conducted in 1999 and 2002, people were asked how
viii
Presented in more detail elsewhere (Fogg, 2002a), the theory suggests that various factors affect both Prominence
and Interpretation.
different aspects of a Web site would affect the credibility of the site. The questions probed how
people viewed sites that “looked professionally designed,” that had “a broken link,” that “gave a
quick response to a customer service question,” and over 50 other items. These previous studies
were about Interpretation. None of these studies examined the Prominence part of the equation.
In contrast to previous work, the study described in this paper focuses on Prominence. It
investigates what people notice when asked to evaluate the credibility of a Web site.ix And what
do people notice? What is prominent? This research has given us some preliminary answers. For
all 10 categories of Web sites, people noticed the design look. After that, people noticed different
things most often, depending on the content category. For news Web sites, people noticed bias of
information. For nonprofit Web sites, people noticed who was behind the site – the identity of the
site operator. As Prominence-Interpretation Theory suggests, the content and purpose of a Web
site affects what people notice when they evaluate the site’s credibility.
The release of this report is a step forward in the study of Web credibility, because to gain a rich
understanding of credibility impact, one must have information about both Prominence and
Interpretation. Studies that focus on these separate components can be woven together to create a
rich warp-and-woof understanding of Web credibility, an approach that is far richer and—in our
view—more compelling than previous explanations of how people assess the credibility of Web
sites.
Consider how having a privacy-policy statement affects the perceived credibility of a Web site.
Previous research (the various studies that focused on Interpretation) found that people claim to
assign more credibility to sites that have a privacy policy. This makes sense. But what if people
don’t notice the privacy policy? Prominence-Interpretation Theory suggests that if people don’t
notice an element, such as a privacy policy, then it will not have any impact on the overall
credibility assessment. Again, this makes sense: Any site with a privacy policy that does not get
noticed gets no credibility boost from having the privacy policy. Our research shows how this
plays out in real Web sites: Fewer than 1 percent of the comments about the 100 Web sites
mentioned anything about a privacy policy. This element was rarely noticed and, as a result, had
almost no real impact on the credibility assessments people made. The same is true for any other
element, such as a broken link buried deep inside the site. Although previous studies show that a
single broken link will hurt the credibility of a Web site—at least that’s what people reported—
the broken link will have no effect on the credibility assessment if people don’t notice it.
An additional example helps show how previous studies and the current research work together in
understanding Web site credibility evaluations. The Stanford studies on Web credibility elements
found, both in 1999 and 2002, that people claim to assign more credibility to a site that “looks
professionally designed” —that’s an issue of Interpretation. Our current study suggests that
ix
What people notice on a Web site and what they make comments about are not exactly the same things, but we
propose that the comments people made in this study reflect to a substantial degree the things people found most
noticeable.
people frequently notice the design look of the site—which is an issue of Prominence. As P-I
Theory suggests, the combination of high Prominence and favorable Interpretation make
“professional-looking design” a Web site quality that will significantly boost a site’s overall
perceived credibility. An appealing visual design is a pivotal issue in assessing Web site
credibility, since this aspect of a site is likely to be both noticed and interpreted positively.
Even though our current study makes significant steps forward in understanding Prominence, our
research has shortcomings. Because the specific percentages in this study are the result of
variables that can change—the coding categories, the study context, the users who chose to
participate, the 100 Web sites selected for this study—we caution readers against becoming too
attached to these particular values. Although we performed our calculations with care, readers
should view the resulting percentages as approximations, since this study is an early attempt to
measure Prominence. We hope future studies can draw on what we have done in order to enhance
the research method and the data analysis. For example, creating a more precise coding system
will be an ongoing process that will require multiple studies and many debates. Perhaps the
biggest contribution from our study will be to provide an initial set of results that future research
can refine or refute. In other words, we view our study as the opening statement in a new
conversation, not as the final word.
While various questions remain unanswered, one thing is clear: Our collective understanding of
Web credibility assessments will become richer as research continues to give insight in two areas:
(1) what people notice when evaluating Web site credibility, and (2) how people evaluate
different Web site elements or features. Both paths are worthy directions for future research.
Implications for Consumer WebWatch

One purpose of this study was to help Consumer WebWatch fulfill its mission of making the
World Wide Web a place where ordinary people can safely and efficiently find information and
make transactions. Consumer WebWatch has outlined five guidelines involving transparency and
fairness. In brief, these guidelines are as follows (see http://consumerwebwatch.org or Appendix
C for the wording of these guidelines):
1. Identity: Making clear who owns the site and how people can contact them
2. Advertising and Sponsorships: Distinguishing between ads and content and disclosing
relevant business relationships
3. Customer Service: Disclosing costs and policies relevant to consumers
4. Corrections: Correcting past information that was false or misleading
5. Privacy: Disclosing how personal information will be used
The results of our study suggest that ordinary people do not often use the above criteria in
evaluating the credibility of a Web site. None of these categories appeared in more than 15
percent of the comments from our participants; some of them appeared rarely, if ever. Our results
do not mean that the five guidelines are irrelevant to consumers. When asked whether these issues
matter, people say yes (Princeton, 2002). Indeed, if people looked for these features on the Web
sites they visit, they would get a clearer picture about what they can and cannot trust online. But
our data suggest that when actually using the Web, people likely don’t think about these five key
issues. They don’t probe very deeply into issues of identity, sponsorship, corrections, customer
service, or privacy.
The disconnect between what people actually do and what they should do creates two important
opportunities for Consumer WebWatch, one dealing with education, the other with evaluation.
The Need for Education

One role Consumer WebWatch can play in the future is that of an educator about what people
should be looking for as they assess how safe it is to believe information or transact business with
a Web site. If consumers knew they should look for the five markers of transparency and fairness,
they would be more likely to seek and notice those things. And as people start looking for those
things, Web site designers will be under pressure to provide them. Right now there is no standard
way to inspect a site to figure out whether what the site says or does is worthwhile. Consumer
WebWatch is in a good position to create this standard.
The Need for Evaluation Assistance

The second role of Consumer WebWatch is perhaps more realistic than the previous role. (Yes,
changing the way people evaluate Web sites is a worthy goal, but this may be too ambitious a
task.) A more tractable endeavor would be to provide people with assistance in evaluating Web
sites—i.e., to do most of the work in identifying which Web sites are worthy of people’s time and
attention and which sites are not.
This is not a new role for Consumers Union. In fact, it’s the core-value proposition of its leading
publication, Consumer Reports. Ordinary people turn to this publication to find out which
products and services are most reliable and which products and services they should probably
avoid. It’s the trusted guide for people buying washing machines, insurance policies, and mobile-
phone plans. This established role for Consumers Union could—and probably should—be
extended into the online world.
The consumer need for evaluation assistance in the online world may be greater than that need in
the physical world. At least in the physical world, people have years of experience to draw on; in
addition, hundreds of regulations have developed over the years to weed out the worst players.
This is not so in the online world. When we use the Web today, all of us are entering territory
that’s new and constantly changing (consider how dramatically online ads have changed in the
last six months, for example). Because the Web is new and dynamic, even those of us who make
a profession of studying the Web would benefit from the evaluation assistance that Consumer
WebWatch could provide. An organization dedicated to this purpose not only could save people
time and energy, but it also could do the job better than virtually any individual working alone.
References
Benoy, J. W. (1982). The credibility of physically attractive communicators: A review. Journal of

Advertising, 11(3), 15–24.
Berscheid, E. & Walster, E. (1974). Physical attractiveness. In L. Berkowitz (Ed.), Advances in

experimental social psychology, 7 (157–215). New York: Academic Press.
Berscheid, E. (1981) A review of the psychological effects of physical attractiveness. In G. W.

Lucker, K. A. Ribbens, & J. A. McNamara (Eds.), Psychological aspects of facial form (1–23).
Ann Arbor, MI: Center for Human Growth.
Cheskin Research and Studio Archetype/Sapient (1999). Ecommerce Trust Study. Available at
http://www.cheskin.com/think/studies/eCommtrust99.pdf.
Cheskin Research (2000). Trust in the Wired Americas. Available at

http://cheskin.com/think/studies/trustIIrpt.pdf.
Cockburn, A., and McKenzie, B. (2001). What do Web users do? An empirical analysis of Web
use. International Journal of Human-Computer Studies, 54(6), 903–922.
Dion, K. K., Berscheid, E., & Walster, E. (1972). What is beautiful is good. Journal of
Personality and Social Psychology, 24, 285–290.
Eagly, A.H., Ashmore, R.D., Makhijani, M.G., & Longo, L.C. (1991). What is beautiful is good,
but ...: A meta-analytic review of research on the physical attractiveness stereotype.
Psychological Bulletin, 110, 109–128.
Eysenbach, G., & Köhler, C. (2002). How do consumers search for and appraise health
information on the world wide web? Qualitative study using focus groups, usability tests, and in-
depth interviews. British Medical Journal, 324, 573-577.
Finberg, H., Stone, H., & Lynch, D. (2001). Digital Journalism Credibility Study. Available at
http://www.journalists.org/Programs/credibility_study.pdf.
Fogg, B.J. (2002a). Prominence-Interpretation Theory: Explaining How People Assess

Credibility. A Research Report by the Stanford Persuasive Technology Lab, Stanford University.
Available at http://credibility.stanford.edu/pit.html or http://www.webcredibility.org/pit.html.
Fogg, B.J. (2002b). Stanford Guidelines for Web Credibility. A Research Summary from the
Stanford Persuasive Technology Lab, Stanford University. Available at
http://credibility.stanford.edu/guidelines.html or http://www.webcredibility.org/guidelines.html.
Fogg, B.J., & Tseng, H. (1999). The Elements of Computer Credibility. Proceedings of ACM
CHI 99 Conference on Human Factors in Computing Systems 1, 80–87. New York: ACM Press.
Available at http://www.acm.org/pubs/articles/proceedings/chi/302979/p80-fogg/p80-fogg.pdf.
Fogg, B.J., Kameda, T., Boyd, J., Marshall, J., Sethi, R., Sockol, M., & Trowbridge, T. (2002).
Stanford-Makovsky Web Credibility Study 2002: Investigating what makes Web sites credible
today. A Research Report by the Stanford Persuasive Technology Lab in collaboration with
Makvosky & Company. Stanford University. Available at http://www.webcredibility.org.
Fogg, B.J., Lee. E., & Marshall, J. (2002). Interactive Technology and Persuasion. In J. P. Dillard
and M. Pfau (Eds.), The Persuasion Handbook: Developments in Theory and Practice (765–788).
Thousand Oaks, CA: Sage.
Fogg, B.J., Marshall, J. Kameda, T. Solomon, J., Rangnekar, A., Boyd, J., & Brown, B. (2001).
Web Credibility Research: A Method for Online Experiments and Some Early Study Results.
Proceedings of ACM CHI 2001 Conference on Human Factors in Computing Systems. New
York: ACM Press.
Fogg, B.J., Marshall, J., Laraki, O., Osipovich, A., Varma, C., Fang, N., Paul, J., Rangnekar, A.,
Shon, J., Swani, P., & Treinen, M. (2000). Elements that Affect Web Credibility: Early Results
from a Self-Report Study. Proceedings of ACM CHI 2000 Conference on Human Factors in
Computing System. New York: ACM Press.
Fogg, B.J., Marshall, J., Laraki, O., Osipovich, A., Varma, C., Fang, N., Paul, J., Rangnekar, A.,
Shon, J., Swani, P., & Treinen, M. (2001). What Makes A Web Site Credible? A Report on a
Large Quantitative Study. Proceedings of ACM CHI 2001 Conference on Human Factors in
Computing Systems (61-68). New York: ACM Press. Available at
http://www.acm.org/pubs/articles/proceedings/chi/365024/p61-fogg/p61-fogg.pdf
Petty, R.E., & Cacioppo, J.T. (1986). The elaboration likelihood model of persuasion. In L.
Berkowitz (Ed.) Advances in Experimental Social Psychology, 19, 123–205. New York:
Academic Press,
Princeton Survey Research Associates (2002). A Matter of Trust: What Users Want From Web
Sites. Results of a National Survey of Internet Users for Consumer WebWatch. Available online
at http://www.consumerwebwatch.org/news/report1.pdf.
Stanford, J., Tauber, E., Fogg, B., & Marable, L., (2002). Experts vs. Online Consumers: A
Comparative Credibility Study of Health and Finance Web Sites Available online at
www.consumerwebwatch.org.
Tseng, S., & Fogg, B.J. (1999). Credibility and Computing Technology. Communications of the
ACM, 42(5), 39–44. Available at http://www.acm.org/pubs/articles/journals/cacm/1999-42-5/p39-
tseng/p39-tseng.pdf.
Appendix A: Web Sites Selected for this Study

We selected 100 sites for this study, 10 in each of 10 categories. The categories and sites are
listed below in alphabetical order:
E-Commerce
Amazon http://www.amazon.com
Barnes & Noble http://www.bn.com
Best Buy http://www.bestbuy.com
Buy.com http://www.buy.com
Cars.com http://www.cars.com
Dogwise http://www.dogwise.com
eBay http://www.ebay.com
McMaster-Carr http://www.mcmaster.com
MTE Nutrition http://www.mtenutrition.com/index.html
ThymuSkin http://www.thymuskin.com
Entertainment
Ain’t It Cool News http://www.aint-it-cool-news.com

ChristianAnswers.net http://christiananswers.net/spotlight
Citysearch http://www.citysearch.com
E! Online http://www.eonline.com
Entertainment Weekly http://www.ew.com
Fandango http://www.fandango.com
Internet Movie Database http://www.imdb.com
Moviefone http://www.moviefone.com
Pazsaz.com http://www.pazsaz.com
Yahoo! Entertainment http://entertainment.yahoo.com
Finance
ChoicePicks http://www.choicepicks.com
Christian Brothers
Investment Services http://www.cbis-fsc.com/index.asp
Domini Social Investments http://www.domini.com
E-Trade http://us.etrade.com/e/t/home
Fidelity Investments http://www.fidelity.com
Merrill Lynch http://www.ml.com

Charles Schwab http://www.schwab.com
Scottrade http://www.scottrade.com
ShareBuilder http://www.sharebuilder.com
Stocks at Bottom http://www.stocksatbottom.com
Health
Dr. Koop http://www.drkoop.com

Dr. Weil http://www.drweil.com
Health Bulletin http://www.healthbulletin.org
HealthWorld Online http://www.healthy.net
Intelihealth http://www.intelihealth.com
MayoClinic.com http://www.mayohealth.org*
MDChoice http://www.mdchoice.com
National Institutes of
Health http://www.nih.gov
Oxygen.com Health and
Fitness http://www.oxygen.com/health
WebMD http://www.webmd.com
*Note that the Web address we used for MayoClinic.com, www.mayohealth.org, automatically redirects to
www.mayoclinic.com. We refer to this site as MayoClinic.com throughout this paper to distinguish from MayoClinic.org,
which is the Web site for the Mayo Foundation and its associated clinics and hospitals.
News
CNN http://www.cnn.com
Crosswalk.com http://news.crosswalk.com/
Drudge Report http://www.drudgereport.com
MSNBC http://www.msnbc.com
The New York Times on the
Web http://www.nytimes.com
SF Gate http://www.sfgate.com
Telluride Gateway http://www.telluridegateway.com
Time http://www.time.com
Workers World http://www.workers.org/ww
Yahoo! News http://news.yahoo.com
Nonprofit
Aim High http://www.aimhigh.org

Comic Book Legal Defense
Fund http://www.cbldf.org
Compassion http://www.compassion.com
DogScouts http://www.dogscouts.com
Liberty Unites http://www.libertyunites.org
MISS Foundation http://www.missfoundation.org
American Red Cross http://www.redcross.org
Save the Earth http://www.savetheearth.org
The Hunger Site http://www.thehungersite.com
World Peace Endowment http://www.worldpeaceendowment.org
Opinion or Review
All Star Review http://www.allstarreview.com

BizRate http://www.bizrate.com
CardOffers.com http://www.cardoffers.com
Consumer Search http://www.consumersearch.com
E-complaints http://www.ecomplaints.com
Epinions http://www.epinions.com
Opinion-Pages http://www.opinion-pages.org
Review Centre http://www.reviewcentre.com
Students Review http://www.studentsreview.com
The Opinionated Beer Page http://www.tobp.com
Search Engines
About.com http://www.about.com
All the Web http://www.alltheweb.com
Ask.com http://www.ask.com
Google http://www.google.com
Insider.com http://www.insider.com
iWon.com http://www.iwon.com
LookSmart http://www.looksmart.com
Overture http://www.overture.com
Vivisimo http://www.vivisimo.com
Yahoo! http://www.yahoo.com
Sports
All Sports http://www.allsports.com

ESPN http://www.espn.com
iWon Sports http://sports.iwon.com
CNN Sports Illustrated http://sportsillustrated.cnn.com
Sports Network http://www.sportsnetwork.com
Sports.com http://www.sports.com/en
SportsJones http://www.sportsjones.com
Stand-Up Sports http://www.standupsports.com
Xslant http://www.xslant.com
Yahoo! Sports http://sports.yahoo.com
Travel
Expedia http://www.expedia.com
Getaway.com http://www.getaway.com
GoNomad http://www.gonomad.com
Hotwire http://www.hotwire.com
Orbitz http://www.orbitz.com
Priceline http://www.priceline.com
Travel Zone http://www.thetravelzone.com
Trip.com http://www.trip.com
United Tours and Travel http://www.uttravel.com
Yahoo! Travel http://travel.yahoo.com
Appendix B: Coding Categories

The following table shows the coding categories we used in this study, along with examples taken
from the data.
Issue & Codes Examples from the consumer data

Explanation Used (Note: For some comments we’ve corrected typos and punctuation.)
Identity IP • This Web site has clear information about its company and
Comments relating to Identity policies.
Consumer WebWatch positive
Guideline #1, which addresses • I could easily find contact information, a physical address,
identity issues. and phone number, which would help me to verify their
legitimacy.
• Very transparent and informative, not only about

investment possibilities but also about philosophy.
IN • I do not know enough about them to feel safe giving them

Identity my money.
negative
• There was no phone number to call.
• They weren’t really specific about what they do, other

than some broad generalities.
Sponsorship SP • Tells who the sponsoring organizations are.

Comments relating to Sponsorship
Consumer WebWatch positive • This search engine makes it clear what is paid advertising
Guideline #2, which addresses and what is not.
issues of sponsorship.
SN • The “sponsors” and ads made me a bit skeptical.
Sponsorship
negative • Too difficult to tell sponsored ads from other hits
retrieved!
Customer Service CSP • They take pains to let you know how their service works.
Comments relating to Customer
Consumer WebWatch service positive • I like the business being up-front about costs, etc.
Guideline #3, which addresses
issues of customer service. • Useful to the point of defining customer service issues.
Also, comments relating to how
And consumer complaints.
an organization operated were
coded in this category.
CSN • This site seemed to have less accountability to its
Customer customers on the items that can be purchased.
service negative
• I don’t like sites where you can’t see exactly what you are
paying for.
• I would like to find something that will tell me that the

quality of services is high.
Corrections COR [There were no comments on this issue.]

Comments relating to
Consumer WebWatch
issues of correcting false
information.
Privacy PP • Had a privacy statement.

Comments relating to Privacy positive
Consumer WebWatch • Liked the Privacy Policy.
issues of privacy. • I liked the large “Security” button with info re: shared
user data.
PN • No privacy policy evident.

Privacy
negative • Various cookies materialized.
• I hate having to log in and give information when I’m

searching for tickets.
Design Look DLP • I like the look of the Web site. Looks professional.
Comments relating to the look Design look
of the site. positive • The layout is cleaner.
• At first I thought that this was going to be the more

credible site, with its layout and design, but after reading
a few reviews I discovered that it is not.
DLN • Looks very unprofessional.

Design look
negative • Seems flashy, like a used-car salesman.
• Graphics are of an earlier generation of e-commerce UI

[User Interface].
Information Design IDP • It is well organized and has a straightforward design—you

Comments relating to how the Info design understand it within seconds.
information is structured or positive
organized on the site. • This is more organized.
• Everything the Web site visitor needed was right on the

front page.
IDN • Core information (such as contact details) can take a

Info design little while to find.
negative
• Too difficult to navigate.
• Information badly presented. They try to put everything

on the front page.
Information Focus IFP • Somehow looks as if more information is being offered at

Comments relating to the scope Info focus this site.
or focus of the site. positive
• Well focused.
• This site was totally devoted to news and the

dissemination of information.
IFN • This site was interesting, but too broad.

Info focus
negative • There was not a whole lot of information on these pages.
• Too much at a glance.
Information IAP • Information sounds credible and corroborates what little I

Accuracy Info accuracy know about this area of work.
positive
Comments relating to the
accuracy of the information on • This site listed more facts.
the site that do not mention
information source. • The information is generally solid.
IAN • Not necessarily providing me with the most accurate or

Info accuracy best information.
negative
• The data is harder to verify.
• I don’t believe in market timing that is advocated on this

site.
Information Bias IB • I feel their view is colored by their desire to boost their
Comments relating to the Information advertising revenue.
perceived bias of information biased
on the site. • Obvious slant in types of stories covered as well as
headlines.
• Sentimentalist jingoistic crap.
IU • It is credible because the opinions contained therein are

Information based on unbiased research.
unbiased
• Students speaking in an uncensored forum.
• Heavily political, represents the left point of view, but

reports fairly and opinion is reasonable.
Information IUP • This Web site provided useful and interesting knowledge
Usefulness Info usefulness about events in sports.
positive
usefulness of the information • I found this site very helpful and informative. I will visit it
on the site. again to get information I need for myself and my young
son.
• This site is great if you’re going to travel. You can get the
“lay of the land.”
IUN • There appears to be no useful information on this Web

Info usefulness site.
negative
• I didn’t see what sense the page was unless it was purely
for fun.
• This site was not very useful other than the bulletin board.
Information Clarity ICP • Clear, concise information on home page.

Comments relating to how the Info clarity
site’s content is (or is not) clear positive • This Web site is great! It really spells things out for a
and understandable. rookie investor. Everything was easy to find and
understand.
• Presents information clearly and cleanly.
ICN • I found this site unclear and confusing.

Info clarity
negative • Nothing on the site but computer language.
• The Web site talks about making things easy for investors
but then speaks over most people’s heads.
Readability ETR • The layout and the graphics are easy to read.
Comments relating to the site’s Easy to read
readability–-how easy or hard it • The format was easier for me to read and follow along
was to read what was on the with the stories.
pages.
• Easy to read.
HTR • First blush rates it better on credibility, but hard-to-find

Hard to read and hard-to-read critics ratings make it appear a poor use
of my time.
• Type too small. Ad smack in the middle of page, with

editorial material, was repugnant. Headline over picture of
Tiger Woods should have extended over type, too, or type
should have been moved up.
• The page is not easily readable. The font “Courier”

contributed to this.
Currency of CRP • Their information seems to be more up-to-date.

Information Currency
positive • This one would appear to be updated more often and kept
Comments relating to how
current (up to date) the current.
information is on the site.
• Contains date of update.
CRN • The photos of the children were from the mid ‘90s; they
Currency should be more up to date.
negative
• No update date.
• Somewhat dated material, it seems, which gives more

time for spinning news and event reports.
Writing Tone TWP • The headlines were dry and to the point. The Web site
Comments relating to the tone Tone of writing had so much to it that it felt “trustworthy.”
or attitude conveyed by the positive
site’s content. • They have a friendly, no-nonsense, straightforward tone
to their page and their letter.
• Restrained language and lack of buzz and flash make this

a more credible site.
TWN • There was too much sarcasm when it came to headlines.

Tone of writing
negative • “Holy Crap” and other slang or poor language harms
credibility. Credible people tend to understate.
• Seems more optimistically oriented, more like advertising

than actual review.
Site Functionality FP • Downloads quickly.

Comments relating to how the Functionality
site functions, both technical positive • Always uses Inktomi, therefore more consistent, therefore
performance and the services more credible.
the site offers.
• It was the most believable because of the little movie blip
they had on the site.
FN • I find the site more credible in theory but it’s awfully

Functionality buggy. When I entered the city of Rome, Italy it gave me
negative hotel info about Nice, France.
• The site was down for site maintenance.
• The news stories section did not show any stories even
though I tried it twice.
Performance on PTP • I searched for MRI and was able to view images and
Test by User Performed test relevant Web pages.
positive
Comments that tell about a test
the user performed to evaluate • Did 2 searches. Both yielded most relevant sites for the
the site’s credibility. subject at top of list.
• This site is interesting, I found a fare on this site that was

actually cheaper than what I found on the Web site of the
airline itself! I bookmarked this one and will use it again.
PTN • I attempted to access information on shingles at this site

Performed test and no information was available.
negative
• I looked for Anaplastic Astrocytoma. The site had only a
couple sites to look at, and they didn’t give a whole lot of
info.
• It was the least believable because it could not find a coat

of armor for my last name.
Past Experience PEP • I’ve used it so much that I know and trust this
with Site Past experience organization.
positive
Comments relating to previous
experiences people had with • I can ALWAYS find anything I need at this site! Have
the site under evaluation never had difficulties here.
• I have used this Web site in the past and have found it to
have sound information.
PEN • I am familiar with them and usually find their critiques to

Past experience be accurate but not portrayed in the best way.
negative
• I’ve used this site before and it did not meet my
expectations.
Name Recognition NRP/RP • Maybe because it is a well-known outfit it looks to me to

and Reputation Name be more credible.
recognition
Comments relating to name
positive/ • This is a more credible site. Not because of simple name
recognition of the site or the
Reputation recognition but because having a local branch gives this
reputation of the operator.
positive
company an image of being more real.
• Name recognition gives this site the edge. Otherwise my

appraisal was almost equal.
NRN/RN • This site is less credible because the name is unfamiliar.

Name
recognition • Both look credible, but this isn’t a brand name, the other
negative/ is.
Reputation
negative
• I simply don’t know anything about this company. I would
have to do more research on the company before I
trusted it with my business.

Advertising AP/AV • Site did not have pop-ups.

Comments relating to how Advertising
users perceive advertising on positive • Banner ads on the side are more discreet and help make
the site. the site feel independent.
• There are ads along the page but they don’t all jump out
at you and if you are really interested in them you can go
check them out, but you don’t lose track of why you came
to the site.
AN • Site had pop-ups and spam in my face. This makes it less

Advertising appealing and less credible.
negative
• This site is full of ads, the home page is dominated by the
Need Cash promo. Advertising is bound to affect
objectivity.
• Sold out their site to advertising.
Motive of CMP • 501C3 status makes this organization appear more

Organization Company credible.
motive positive
perceived motive—good or • This site is just about opinions and isn’t trying to sell
bad—of the organization anything. Much more of a chance that people are giving
behind the site. honest opinions here since money isn’t involved.
• I would trust this site because it’s run by a religious

denomination whose aim is socially responsible investing.
CMN • Doesn’t seem credible when they give a product a good

Company review and give you a link to order it, too.
motive negative
• This is an entertainment Web site? More like an ad-dollar
driven method of driving people into lame venues. Not
likely.
• The fact that this one is done by an insurance company

makes it seem less credible.
Affiliations ASP • Affiliation with a prestigious university adds to a sense of

Comments relating to the site’s Association objectivity.
affiliates, supporters, or positive
partners. • Backed by CNN
• Links to other major sites. Amazon, Honda. Would these

sites lend their credibility to a scam site? Not likely.
ASN • Lately, anything related to religion and money sends up a

Association red flag.
negative
• I tend to distrust anyone or thing that brags about any
awards right off the get-go. The J.D. Powers awards seem
to be passed out like Halloween candy.
• I don’t believe much that GW Bush says, so his presence

on the Web site detracts from its credibility.
General Suspicion JPS • Sounds almost too good to be true.

General comments about being Just plain
suspicious of the site or its suspicious • I wouldn’t put either in the “credible” category. . . . I
operators. would be suspect of both.
• Because it involves buying and selling I tend to believe

that their system is bound to have faults.

General Dislike JPD • About as cheap and nasty as any form of publication in
General comments about not Just plain any media can be, crass and corporate, avoid like the
liking the site or its operators. dislike plague.
• I don’t like Web sites that exploit women’s bodies.
• The American “entertainment industry” generally fails to

entertain me.
Appendix C: Consumer WebWatch Guidelines

These guidelines were retrieved from http://consumerWebWatch.org on August 17, 2002.
CONSUMER WEBWATCH GUIDELINES
1 Identity:
Web sites should clearly disclose the physical location where they are produced, including an
address, a telephone number or e-mail address.
Sites should clearly disclose their ownership, private or public, naming their parent company.
Sites should clearly disclose their purpose and mission.
2 Advertising and Sponsorships:

Sites should clearly distinguish advertising from news and information, using labels or other
visual means. This includes “in-house” advertising or cross-corporate ad sponsorships. Search
engines, shopping tools and portals should clearly disclose paid result-placement advertising, so
consumers may distinguish between objective search results and paid ads.
Sites should clearly disclose relevant business relationships, including sponsored links to other
sites. For example: A site that directs a reader to another site to buy a book should clearly
disclose any financial relationship between the two sites.
Sites should identify sponsors. The site’s sponsorship policies should be clearly noted in
accompanying text or on an “About Us” or “Site Center” page.
3 Customer Service:
Sites engaged in consumer transactions should clearly disclose relevant financial relationships
with other sites, particularly when these relationships affect the cost to a consumer.
Sites should clearly disclose all fees charged, including service, transaction and handling fees,
and shipping costs. This information should be disclosed before the ordering process begins.
Sites should clearly state and enforce policies for returning unwanted items or canceling
transactions or reservations.

4 Corrections:
Sites should diligently seek to correct false, misleading or incorrect information.
Sites should prominently display a page or section of the site where incorrect information is
corrected or clarified.
Sites should strive to mark content with its published date when failing to do so could mislead
consumers.
Sites should clearly state their policy on a consumer’s rights if a purchase is made based on
incorrect information on the site.
5 Privacy:
Site privacy policies should be easy to find and clearly, simply stated.
Sites should clearly disclose how personal data from site visitors and customers will be used.
Personal data includes name, address, phone number and credit card number.
Sites should disclose whether they use browser-tracking mechanisms such as “cookies,” and
other technologies such as Web beacons, bugs and robots.
Sites should explain how data collected from them will be used.
Sites should notify customers of changes to privacy policies, and provide an easy opt-out
alternative.

Endnotes
1
While the most basic question about Web credibility is difficult to answer in brief, two recent studies have
examined this issue in different ways and reached somewhat different conclusions. See:
Princeton Survey Research Associates (2002). A Matter of Trust: What Users Want From Web Sites. Results
of a National Survey of Internet Users for Consumer WebWatch. Available online at
http://www.consumerwebwatch.org/news/report1.pdf.
Finberg, H., Stone, H., and Lynch, D. (2001). Digital Journalism Credibility Study. Available at
http://www.journalists.org/Programs/credibility_study.pdf
2
For statistics on Internet access in many countries, see
http://cyberatlas.internet.com/big_picture/geographics/article/0,1323,5911_151151,00.html
3
Some of the research examining issues relating to Web credibility include the following:
Cheskin Research (2000). Trust in the Wired Americas. Available at
http://cheskin.com/think/studies/trustIIrpt.pdf.
Cheskin Research and Studio Archetype/Sapient (1999). Ecommerce Trust Study. Available at
http://www.cheskin.com/think/studies/eCommtrust99.pdf.
Finberg, H., Stone, H., & Lynch, D. (2001). Digital Journalism Credibility Study. Available at
http://www.journalists.org/Programs/credibility_study.pdf.
Fogg, B.J., & Tseng, H. (1999). The Elements of Computer Credibility. Proceedings of ACM CHI 99
Conference on Human Factors in Computing Systems 1, 80–87. New York: ACM Press. Available at
http://www.acm.org/pubs/articles/proceedings/chi/302979/p80-fogg/p80-fogg.pdf.
Fogg, B.J., Kameda, T., Boyd, J., Marshall, J., Sethi, R., Sockol, M., & Trowbridge, T. (2002). Stanford-
Makovsky Web Credibility Study 2002: Investigating what makes Web sites credible today. A Research
Report by the Stanford Persuasive Technology Lab in collaboration with Makvosky & Company. Stanford
University. Available at http://www.webcredibility.org
Fogg, B.J., Lee. E., & Marshall, J. (2002). Interactive Technology and Persuasion. In J. P. Dillard and M.
Pfau (Eds.), The Persuasion Handbook: Developments in Theory and Practice (765–788). Thousand Oaks,
CA: Sage.
Fogg, B.J., Marshall, J., Laraki, O., Osipovich, A., Varma, C., Fang, N., Paul, J., Rangnekar, A., Shon, J.,
Swani, P., & Treinen, M. (2001). What Makes A Web Site Credible? A Report on a Large Quantitative
Study. Proceedings of ACM CHI 2001 Conference on Human Factors in Computing Systems (61-68). New
York: ACM Press. Available at http://www.acm.org/pubs/articles/proceedings/chi/365024/p61-fogg/p61-
fogg.pdf.
Princeton Survey Research Associates (2002). A Matter of Trust: What Users Want From Web Sites. Results
of a National Survey of Internet Users for Consumer WebWatch. Available online at
http://www.consumerWebwatch.org/news/report1.pdf
Tseng, S., & Fogg, B.J. (1999). Credibility and Computing Technology. Communications of the ACM, 42(5),
39–44. Available at http://www.acm.org/pubs/articles/journals/cacm/1999-42-5/p39-tseng/p39-tseng.pdf

4
Stanford, J., Tauber, E., Fogg, B., & Marable, L., (2002). Experts vs. Online Consumers: A Comparative Credibility
Study of Health and Finance Web Sites. Consumer WebWatch Available online at www.consumerwebwatch.org.
5
iWon is reported to be among the top 100 most popular Web sites (see http://www.trafficranking.com, as
an example).
6
http://www.census.gov/Press-Release/www/2001/cb01cn67.html
7
http://www.census.gov/Press-Release/www/2001/cb01cn67.html
8
http://reports.netratings.com/nnpm/owa/NRpublicreports.usageweekly

How Users Evaluate Website Credibility

Uploaded by

How Users Evaluate Website Credibility

Uploaded by

How Do People Evaluate a Web Site’s Credibility?

Results from a Large Study

B.J. Fogg, Ph.D., Cathy Soohoo, David Danielson

Julianne Stanford and Ellen R. Tauber

Report Released: October 29, 2002

Study term: June 15, 2002, to August 15, 2002

For more information about this report, contact bjfogg@stanford.edu or brenbe@consumer.org

About Consumer WebWatch ....................................................................... 7

Background & Methods .............................................................................. 12

Results & Discussion .................................................................................. 22

Part 1: Results & Discussion – Overall Analysis of Credibility Comments ... 23

Overall discussion on comments ...................................................................58

Part 2: Results & Discussion – A Focus on Individual Web Site Categories 61

E-Commerce Web Sites: Credibility Results ....................................................63

Concluding Discussion Focusing on Web Site Categories...................................83

Implications for Consumer WebWatch ....................................................... 88

Appendix A: Web Sites Selected for this Study ........................................... 92

Appendix B: Coding Categories .................................................................. 96

Appendix C: Consumer WebWatch Guidelines .......................................... 102

Endnotes .................................................................................................. 104

About these joint studies

Topics our analysis did not find

Insights from Prominence-Interpretation Theory

About Consumer WebWatch

About the Stanford Persuasive Technology Lab

About Sliced Bread Design, LLC

How Do People Evaluate a Web Site’s Credibility?

How to read this report

Background & Methods

Choosing 10 content categories

The categories we selected were:

Choosing 10 Web sites in each category

How we recruited study participants

Who participated in this study

• Female: 58.1%; Male: 41.9%

What people did in the study

5. Reviewed the two sites assigned to them

A more detailed description of the participants’ experience follows:

Figure 1: The first page participants saw in the study.

Figure 3: Participants viewed two Web sites in the same category.

Figure 5: Screen for demographics and for nonprofit donations.

Cleaning the data

How we coded and analyzed the comments

How we calculated the rankings

Results & Discussion

Part 1: Results & Discussion – An Overall Analysis of Credibility Comments

Part 1: Results & Discussion – Overall Analysis of Credibility Comments

Percent Comment Topics

1. 46.1% Design Look

1. Design Look – 46.1% Overall

• More pleasing graphics, higher-quality look and feel — F, 52, Tennessee

• Just looks more credible. —M, 24, New Jersey

Design Look comments, by site category

Search Engines 52.6%

All Sites 46.1%

Opinion or Review 38.1%

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0%

Percentage of Site Evaluations

Figure 6: Percentage of comments related to design look, by category

What we find notable about Design Look

2. Information Design/Structure – 28.5% Overall

Information Design/Structure comments, by site category

Percentage of Comments Related

Search Engines 42.6%

All Sites 28.5%

Opinion or Review 24.3%

Figure 7: Percentage of comments related to information design, by category.

What we find notable about Information Design/Structure

3. Information Focus – 25.1% Overall

• Credible because of the breadth of information available. —M, 35, California

Information Focus comments, by site category

Percentage of Comments Related

All Sites 25.1%

E-C ommerce 24.7%

Search Engines 24.5%

Opinion or Review 22.7%