Wikipedia:Research help/Proposal

This page documents a project proposal from the Wikipedia Library team about test links in reference sections to a "Research help" page (see WP:Research Help) to help Wikipedia readers, especially readers that use Wikipedia in research environments like schools and libraries, get a more nuanced understanding of how to use Wikipedia in research. This basic literacy portal is in part a response to the Wikipedia Library's outreach audience, and documents a number of assumptions held by the Wikimedia community but not often fully understood by outside audiences.

What are we trying to solve?

Wikipedia's massive size and scope gives it an incredibly powerful place in research for the public, academics and the press, yet not everyone who uses Wikipedia as part of their research process understands how our knowledge is different than other source types.

Students and researchers in academic contexts use Wikipedia at various stages in their research, but many have never been taught an effective strategy to using Wikipedia as part of that research process. Frequently students and other types of researchers think of Wikipedia a starting point, but this is a learned skill: for example, one study from Rutgers asked if students ever used Wikipedia's references as a starting place for research, and one student responded “i've [sic] never even considered doing this”.

Moreover, even though some experts have begun to teach Wikipedia as a research starting point, they don't necessarily teach it with an informed knowledge of the Wikipedia community and its policies. For a significant window of time, librarians and the press criticized Wikipedia, often without understanding our community and project. This misinformation alongside larger assumptions about internet resources, has created a range of misunderstandings, from the very simple to the very complex: for example, a study by Rutgers University found that some students assumed the first references in the reference section were the best. Furthermore, university-educated readers are a small portion of our readership: what about Wikipedia readers that don't have college level training for research literacy, especially when we see growth of impact in less wealthy countries that come on the internet?

Editors often assume community members and readers have sufficient research skills to follow the sources: Verifiability's core assumption is that public readers understand the connection between referencing, the knowledge in the article and the materials being referenced. But this is not a universal skill, and, in part, Wikipedia has the responsibility to teach the skill, when it's a core assumption about improving the accuracy of our content.

Furthermore, significant parts of our readership have demonstrated mixed literacy about our process for creating knowledge on Wikipedia and how to use it. A January 2016 Twitter Survey, found that even amongst almost 900 of Wikipedia social media followers (thus fans of the site), over a third almost never checked the references in Wikipedia articles. Two themes from the Wikimedia Foundations' 2015 public strategy consultation's responses highlight the missing knowledge from Wikipedia readers about Wikipedia's content process:

• Content quality (accuracy): These comments emphasized the importance of content accuracy, trustworthiness, and reliability. Comments focused on citation quality, the use of expert editors, and even restricting editing (so that “not everyone can edit”). Most (73%) of comments in this category were from anonymous and new users, signaling an opportunity to talk to readers about the accuracy and trustworthiness of content.
• Wikipedia's effect on education: These comments reflected both a concern about the perception of Wikipedia as a (non)credible source for education, and also recognition of the growing opportunity for Wikimedia to extend its global presence into online education.

These worries about quality and credibility focus readers away from contributing to Wikipedia and more on criticizing Wikipedia's accuracy or usefulness. Our hope is that providing a well explained, community-inflected understanding of quality and credibility will transform the shared story of readers.

The core value of Verifiability makes improving references one of the most valued skills for readers to start contributing as new users, especially when those individuals have research expertise. However, on English Wikipedia (and many other Wikipedias), the only communications of this policy comes from warning templates, tools that direct readers to full community policies -- not simplified introductions to the community's processes for improving the content. With this barrage of legalistic policy, we don't break down the social barriers of contributing Wikipedia or meaningfully inform readers how to edit. The vast majority of content pages—which are middling in quality but don't include warning templates—would benefit from the inclusion of better research from trained researchers, yet don't invite them to participate.

The research portal has the opportunity to highlight how valuable researcher and expert skills can be used to improve Wikipedia. If readers don't understand the research process that creates Wikipedia, they won't recognize the existence of a community, much less join it. Our hope is to create a portal that emphasizes the value of research and referencing to Wikimedia's community, directing more readers and contributors to the resources/tools supported by the Wikimedia community and the Wikipedia Library for those contributors.

References

Who is affected?

All Wikipedia users are affected, because this portal could become one of the most visible introductions to Wikipedia if it were included on every page. We developed it with two audiences in mind:

Readers

Most readers do not fully understand how Wikipedia articles are created. This is often the first barrier for a typical reader to contributing to the community. Further, thousands of reference professionals and educators are teaching students and the public how to use Wikipedia as a research tool (or applying blanket challenges to Wikipedia, such as "it should never be used for real research").

The portal provides a user-friendly understanding of

• Wikipedia's policies around referencing
• how to get access to sources through libraries and web discovery tools when using Wikipedia as a research starting point
• how researchers can contribute on-wiki in an appropriate way

Because researchers, GLAM professionals, teachers and librarians are attentive to referencing, this will provide an excellent opportunity to engage more of these expert communities. The links along the right-hand side and the main Q&A of the page proposal will help those experts:

• teach about using Wikipedia as part of the research process in their field,
• systematically improve references in their topical focus
• participate in programs focused on improving quality content, like the Education Program and GLAM.

We don't expect the links in reference sections to appeal to every reader; however, by catching the eye of the most engaged readers (readers who have arrived at a References section as part of their research process, including educators, secondary/university students, and experts), we hope to have a ripple effect, where those readers are more able to communicate Wikipedia's content to others. In particular, as teachers and librarians discover the tool, we expect it will become a common explanation of Wikipedia for a number of audiences. Initial read-throughs of the page by librarians has been very positive, with comments such as "Every student should read this".

Editors

• We know many Wikipedia editors begin contributing without fully understanding how to do effective research, use reference tools, or apply these skills within Wikipedia's best practices.
• Many experienced editors do not know there are significant research support tools available within the community which we have been trying to index at The Wikipedia Library.

We hope that the page will help guide editors to the community tools that can empower them to contribute more research, and communicate Wikipedia's research purpose to interested readers or outreach communities.

How do we plan to solve it?

The Wikipedia Library team has created a research portal to provide research learning opportunities for these audiences, to teach basic information and research literacy, to direct users to best practices for accessing research materials identified on Wikipedia, and to guide them in contributing to Wikipedia to improve access to these resources for future editors. By creating a welcoming and simple approach to understand Wikipedia's place in public research, we hope to dispel myths about using Wikipedia, more clearly explain the processes that create Wikipedia's research materials, and create an entry point for new editors.

We invite community members to tweak the page, or propose major revisions on the talk page.

Project steps

1. Build Alpha portal  Done WP:Research Help
2. Create an initial proposal for a limited Alpha test of the concept, to decide it this is something the community wants  Done (this page)
3. Do a small batch alpha test with initial design (with Template:Research help) by targeting a subset of articles, like Military History or Medicine, where we can rely on a) significant research being available on articles and b) a significant public interest in researching these topics  Doing...

   Proposal for WikiProject Medicine is at this discussion (Archived here)

   1. Implemented on 50 articles - December 17, 2015
   2. Implemented on 500 more articles - February 3, 2016
   3. Implemented on 1000 articles - February 18, 2016
   4. Implemented on 5000 articles - March 7, 2016
   5. Implemented on larger batch of [X] articles

   Proposal for WikiProject Military history is at this discussion (Archive here) Consensus was to implement it primarily on WWI and WWII articles:

   1. Implemented on 50 articles - December 17. 2015
   2. Implemented on 500 more articles - February 3, 2016
   3. Implemented on 1000 articles - February 18, 2016
   4. Implemented on 5000 articles - March 7, 2016
   5. Implemented on larger batch of [X] articles

   Bot approval: Wikipedia:Bots/Requests for approval/Hazard-Bot 33  Done conditional on 10,000 articles total max.

   can be found here

4. Remove transclusion of the template through (on or before 4/7) and assess outcomes, refining the design with audience focus groups and feedback from alpha pilot.
5. Experiment with other workflows experiment with different entry points (through Reference tooltips for example) and possibly designs based on feedback.
6. Continue collecting data" solicit academic research on portal and other literacy tools, and refine the process while expanding the number of articles affected by the pilot through select WikiProjects and/or other community venues – three academics have committed to doing targeted focus-group research Spring/Summer/Fall 2016, to improve the page itself WP:Research help, that will expand upon existing scholarship.
7. Run a larger RFC via Village Pump representing the Alpha results, other testing to create the project, etc, to move towards a Beta that can be applied to all English pages
8. Disseminate experience to other language communities

Frequently Asked Questions

Why the Wikipedia Library?

We decided to develop this page for a pilot, with the hope that it can become a community-owned project, to support Wikipedia's two major communities of researchers – editors and readers.

• We wanted to provide the community's library partners a tool to explain the relationship between Wikipedia and research while also correctly reflecting our community. At a number of different Library events and talking with librarian networks, we've heard librarians say that they regret how they taught students to use Wikipedia five to ten years ago, when it was still emerging and there was major questions about its reliability. Now the legacy of that conversation, creates a discrepancy between reader understanding and the community's best advice about using Wikipedia.
• We wanted to help direct librarians, newer Wikipedia editors, and other researchers to the range of research services provided by the Wikipedia community. The Wikipedia Library's home page exposes Wikipedians to a number of tools within the community, but doesn't expose some of our best community resources to the public (such as Reference desk).

Is this too much clutter for the reference section?

Maybe? There is pretty good evidence that most of our readers don't go far beyond the lead section of Wikipedia articles, and that most readers when they enter lower sections of the Wikipedia article, they are doing research or reading deeply to investigate a topic more. This is exactly where we want the page to catch attention of our readers: where they are doing research. It might reach a wider audience in another part of the UI/UX for Wikipedia, but, the page is designed for people in the midst of a specific behaviour: research. The survey offered to readers of WP:Research help solicits their feedback on where the page should be linked from.

Why are you using the template Template:Research help instead of embedding the link in the sidebar or {{reflist}}?

We created the template {{Research help}} for several reasons:

• We wanted something that wasn't buried deep in a Lua module, and could be tweaked (and/or noincluded or blanked avoiding the effort needed in modifying a core template and with minimal need for an admin to be "on call") -- we realize that community might reach a point in conversation where we need to put a full stop on the experimentation and we want to be able to do this nimbly.
• We wanted to be able to nimbly change which redirects traffic goes through, to measure the impact of the template on different types of articles. {{Research help}} documents the different variables we have used so far. We are hypothesizing that different audience types will click through with different frequencies. We wanted to measure that impact -- a template added by bot seemed like the most targeted way to do this.

We acknowledge that this isn't a long term solution, if the initial test is successful, we plan to seek community consensus on one of the following:

• Including {{Research help}} in {{reflist}} or <references/>
• Including the link in the left sidebar
• Integration into Reference tooltips
• Including the link in other as yet undecided point

Proposed design for links on article pages

We have created a number of demos, which can be found at WP:Research help/Demo and are open to proposed revisions for the templates. But for the initial pilot phase, we plan to use the following (created with Template:Research_help, because of its relative visibility in the References section:

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean pellentesque, nulla a tempus aliquam, enim nulla posuere purus, id ullamcorper dolor eros et diam. Suspendisse id volutpat orci, at gravida dui. Donec porta felis quis leo tincidunt, porttitor vulputate quam fermentum. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas.

References

---