Academics’ Opinions on Wikipedia and Open Access Publishing

Introduction

When it comes to the decision of where to submit academic research papers, relevance to

practice is considered important, as is scholarly rigor (Klobasa and Clydec, 2010). However,

social and technological advances have brought significant changes in publication, particularly

via a shift to electronic or online media (Rowlands, 2007). The idea of open-access publishing is

predicated upon that shift, which enables its works to be freely available online, and has received

significant attention (e.g., Rowlands, 2007; Suber, 2012). There are in general three models in

the current movement towards open-access academic publishing: pushing traditional journals

towards open access by changing policies (e.g., British Medical Journal and College & Research

Libraries); creating open-access journals (e.g., First Monday, Journal of Computer-Mediated

Communication, Journal of Community Informatics); and using existing online open-access

venue Wikipedia (e.g., Black, 2008; Xiao & Askin, 2012).

Most interest in this topic has focused on the first two models. For example, Suber (2007)

discusses making traditional journals open access, while Peters (2007) examines the conversion

process for a single publisher. Researchers from multiple institutions developed an open-source

system to create and manage open-access journals (

http://pkp.sfu.ca/

), and Solomon (2008)

issued a practical guide to developing an open-access journal. The third model, on the other

hand, has been little explored. Although there have been numerous studies about Wikipedia

publishing, they have mainly focused on the quality of Wikipedia articles (e.g., Yaari, 2011;

Dooley, 2010; Stvilia et al., 2005), technical elements of Wikipedia (e.g.,

Völkel et al. 2006;

Viégas et al., 2007

) ; demographics, particularly the “gender gap” (e.g., Yasseri et al., 2012; Lam

et al., 2011); and Wikipedia as a pedagogical tool (e.g., Forte and Bruckman, 2006; Moy et al.,

2010). Only a few studies examined the possibility of using Wikipedia for academic publishing.

Xiao and Askin (2012) conducted a conceptual analysis to compare the Wikipedia and open-

access publishing models, and identified benefits and challenges of using Wikipedia for

academic publishing. Black (2008) suggests that the Wikipedia model would improve on the

existing peer-review processes of traditional journals. However, there has been little examination

of academia’s views on the matter.

Addressing the gap in the literature, we surveyed academics to understand their

perspectives on using Wikipedia for academic publishing in comparison with open-access

journals. We hypothesized that the researchers’ perspectives would be affected by their

knowledge of Wikipedia’s peer-review process, their experiences with Wikipedia and open-

access journals, and their academic environment and status. The research questions in our study

are:

RQ1: What are the perceived advantages and disadvantages of using Wikipedia for

academic publishing?

RQ2: Do the researchers’ experiences with Wikipedia and open-access journals affect

their perspectives on using Wikipedia for academic publishing? If so, how?

RQ3: Do the researchers’ academic environments and statuses affect their perspectives

on using Wikipedia for academic publishing? If so, how?

RQ4: Does the researchers’ knowledge of Wikipedia’s peer-review process affect their

perspectives on using Wikipedia for academic publishing? If so, how?

Related Work

In designing this study, we examined literature on both open-access journals and

Wikipedia publishing. In regards to the open-access model, Rowlands (2007) noted that it

represents a “tectonic shift” in scholarly communication. Bjørk (2004) suggested that the current

emphasis on journal prestige is a major barrier to adoption of the open-access model. Papin-

Ramcharan and Dawe (2005) suggest that academics encounter barriers in open-access

publishing, including cost and infrastructure, particularly in the developing world. Nicolas and

colleagues (2005) argue that the solution is a market structure for academic publishing. Peter

Suber’s book Open Access (2012) provides a concise yet comprehensive introduction to open

access, including its benefits to research and authors, copyright issues, and future directions.

Open Access @ UNT (

https://openaccess.unt.edu

) has been maintaining resources related to

open access, e.g., international open access, libraries and open access, etc.

There have been discussions about using Wikipedia for academic publishing, although

the possibility has not been explored in detail. As early as 2006, Frishauf suggested that

Wikipedia might make traditional academic journals obsolete. Forte and Bruckman (2006)

considered Wikipedia as an early publication venue for students. Black (2008) examined

Wikipedia's editing model and concluded that Wikipedia could become a venue for academic

publication. In terms of academic perspectives Eijkman (2010) considered faculty viewpoints on

Wikipedia in the context of Web 2.0 technologies, suggesting that its cautious use by the

scholarly community represents a reformation of the conventional knowledge hierarchy.

Taraborelli and colleagues (2011) examined barriers to expert participation in Wikipedia. Park

(2011) considered the extent to which Wikipedia is cited by academic papers. Chen (2010)

examined the perception of Wikipedia as a student resource among faculty members. Dooley

(2010) found that academics already use Wikipedia as a resource for both teaching and research.

The findings of these studies provided a foundation for our research.

Research Method and Analysis

We designed the questionnaire based on our interests of answering our research questions

as well as Xiao and Askin’s (2012) conceptual analysis. In comparing Wikipedia and open-

access journal publishing, they focused on five aspects: authorship, cost, reliability, timeliness,

and peer review. We wrote questionnaire items to compare views on these aspects. The other

three sections of our questionnaire were: demographics, such as age, gender, academic unit and

position, and institutional emphasis on research; experiences, defining terms and querying

participants’ experience with Wikipedia and open-access journals; and feedback, soliciting

respondents’ opinions on Wikipedia’s web structure with respect to supporting academic

publishing. As Wikipedia has a unique web structure, we sought respondents’ feedback on its

design and the advantages or disadvantages of using Wikipedia for academic publishing. We

ended the survey by asking “What’s your biggest concern of using Wikipedia as a venue for

academic publishing? Why?”

Our survey was administered online. We used two techniques to reach potential

respondents. We first randomly selected 41 universities from USA, Canada, China, and India

from rankings for top world universities. We then emailed information about the project to 198

secretaries of six selected departments: three in the natural sciences (chemistry, biology, and

physics) and three in the social sciences (education, psychology and sociology). Of 198 emails

sent, 17 responded with support for forwarding our invitation with the survey URL to the

members of their departments, i.e., potential respondents. The rest did not respond or declined to

participate. Through this technique, we solicited 51 responses in three months, all from Canada

or USA. Because the number of responses was too limited, we sent our email invitation to nine

listserv mailing lists to reach more potential respondents. The mailing lists were

[e-mail], [e-mail], [e-mail], edtech@h-

net.msu.edu, [e-mail], [e-mail], [e-mail],

[e-mail], and [e-mail]. We were

only able to confirm that it was received by subscribers to the last three lists; furthermore, only

four of the lists (chi-announcements, ciresearchers, edtech and air-l) provided their subscriber

count (2249, 800, “approximately 3500” and “over 2000” respectively). Through this technique,

we solicited 69 respondents in four months. We thus had 120 responses submitted online. For

each full response (i.e., all survey items were completed) we donated CAD$2 to United Way.

Our questionnaire data are mainly nominal. We calculated the descriptive statistics using

frequency counts and conducted chi-square tests to examine relationships between the responses

of different items. The software program SPSS 16 was used in the analysis.

Results

Demographics and Academic Background of Respondents

Of the 120 respondents, 65 were male, 49 female, and 6 respondents declined to identify

their gender. Most participants were older: 30% were between 36 and 45, 22% between 46 and

55, and 19% over 55. Only 28% self-identified as being under 35.

Most respondents were tenured or had an academic rank higher than assistant professor:

24% were professors, 17% were associate professors, and 8% tenured assistant professors. Non-

tenured assistant professors comprised only 21%. The majority had research-oriented positions

(70%). 77% of respondents either agreed or strongly agreed that research publications were

important in performance evaluations. As shown in Table 1, respondents represented a range of

departments and faculties.

Table 1: Respondents' academic units.

Faculty/department

Number of

respondents

Natural sciences (chemistry, biology, physics, etc)

36

Social sciences (sociology, psychology, etc)

27

Computer sciences (information, technology, etc)

36

Other

8

No answer

13

In terms of academic publishing, the results show a trend towards collaboration (see

Table 2).

Table 2: Survey responses on authorship

Question: How many articles in peer-reviewed

journals do you have?

Answer

Single author

(number of

responses)

Co-author

(number of

responses)

<3

70

36

3-6

17

17

7-10

13

23

11-15

4

9

>15

13

34

No answer

3

1

RQ1: What are the perceived advantages and disadvantages of using Wikipedia for

academic publishing?

Advantages and Disadvantages Compared to Open-Access Journals

Respondents reported advantages and disadvantages of using Wikipedia for academic

publishing, when asked to compare it with open-access journal publishing. Advantages reported

include cost reductions (40%), timely review (19%), post-publication corrections (52%), and

making articles available before validation (27%). 8% of respondents provided additional

advantages, most related to reaching a wider audience, such as “more widely read” and “true

peer review”. 26% of respondents did not see any advantage.

Disadvantages of Wikipedia compared to open-access journals included questionable

stability (86%), absence of integration with libraries and scholarly search engines (55%), lower

quality (43%), less credibility (57%), less academic acceptance (78%), and less impact on

academia (56%). 6% of respondents noted other disadvantages concerning impact of general

readership on content (e.g., “primarily for the general public”, “less detailed”), and Wikipedia’s

editorial policies (e.g., “arbitrary anti-intellectual editorial policies”). 3% of respondents did not

note any disadvantages.

Each Wikipedia article also has an attached discussion page, allowing for changes to be

made in response to feedback without readers editing directly. 56% of respondents reported they

would like to have such discussion pages for their published academic papers.

Given that Wikipedia has been very successful as a volunteer-driven online community,

it was surprising that only 38% of respondents acknowledged the associated reduction in costs,

and 28% of respondents chose “neither agree nor disagree” when asked if Wikipedia publishing

costs less than an open-access journal. Most respondents also felt that the author’s knowledge

and intellectual contribution could not be demonstrated via Wikipedia.

Other Perceived Impact

We collected respondents’ opinions on perceived advantages of Wikipedia for academic

publishing based on Xiao & Askin’s conceptual analysis (2012). As seen in Table 3, the results

showed some agreement with their analysis, but the advantages are not strongly perceived by the

respondents. A relatively high percentage of respondents chose “neither disagree nor agree”.

Table 3 R

espondents’ opinions on various perceived advantages of Wikipedia for academic

publishing

Questionnaire Items

Percentage of the Responses (N = 120)

Likert Scale (from “Strongly Disagree” to
“Strongly Agree”

Agree or

Strongly

Disagree or

Strongly

Neither

Disagree or

No

response

Agree

Disagree

Agree

I’d like to have a discussion page or forum for
each of my published academic papers for
anyone to comment on my work.

58%

19%

19%

4%

An author’s knowledge can be demonstrated
through Wikipedia as an academic venue,
compared to a scholarly journal.

23%

46%

23%

8%

An author’s intellectual contribution can be
demonstrated through Wikipedia as an academic
venue, compared to a scholarly journal.

21%

44%

27%

8%

An author’s writing skill can be demonstrated
through Wikipedia as an academic venue,
compared to a scholarly journal.

36%

33%

21%

10%

A Wikipedia article can become a reliable source
after it’s peer-review process.

42%

33%

22%

3%

The peer-review process of a Wikipedia featured
article is of comparable rigorousness to that of an
open access journal.

8%

51%

24%

17%

The Wikipedia model increases the diversity in
points of view compared to the articles of open
access journals.

35%

20%

32%

13%

Wikipedia’s quick turnaround in its peer-review
process is the biggest advantage of the model
over open access journals.

22%

22%

35%

21%

Wikipedia’s quick turnaround on post-publication
corrections of an article’s content is the biggest
advantage of the model over open access
journals.

29%

16%

36%

19%

Wikipedia allows articles to be edited by anyone. A majority of participants (90 out of

120) do not feel comfortable having other researchers edit their papers-in-progress even if the

researchers are in the same community. 28 participants explained that they prefer to have

authorial control (“my name but no control of content”, “if it’s my article, it’s my article”), while

some offered other misgivings (“annoying and intrusive gatekeeping”, “rivalries…could lead to

sabotage”). One of the strengths of Wikipedia is its open collaboration (Stvilia et al., 2008).

Despite Wikipedia contributors usually having never met in person, there is the potential for

effective collaboration. As with traditional research partnerships in academia, contributors to

Wikipedia may have developed relationships that enabled them to trust each other’s research

expertise. Furthermore, the culture of Wikipedia encourages cooperative editing and provides

venues for discussion of pages being developed.

RQ2: Do the researchers’ experiences in Wikipedia and open-access journal

publishing affect their perspectives on using Wikipedia for academic publishing? If

so, how?

Respondents’ Experiences with Wikipedia and Open-Access Journals

77% of respondents reported reading Wikipedia both for pleasure and for academic use.

However, as shown in Table 4, respondents’ experiences are limited. 57% of respondents had no

experience with Wikipedia other than reading the articles, and no respondent identified

him/herself as a Wikipedia administrator (a user who can delete pages and block editors). On

another survey item, thirteen respondents reported having cited Wikipedia articles in peer-

reviewed papers.

Compared to Wikipedia, fewer respondents had reading experiences with open-access

journals (see Table 4). 23% reported that they read open-access journals regularly, while 43%

read them but not regularly. But respondents had more experience in editing and publishing in

open-access journals: twenty-eight reported that they had published an article in open-access

journals, and seven had been an editor. 41% had cited open-access journal articles in their peer-

reviewed papers. On another survey item, only 3% of the respondents didn’t believe in the

quality of open-access journals.

Table 4 Respondents’ experiences of using Wikipedia and open-access journals

Questionnaire Item

Percentage of the Responses

(N = 120)

Have you ever had other experiences with Wikipedia? Choose all that apply:


Editing as a non-registered user

21%



Editing as a registered user

23%



Writing an article

15%



Administrator

0



Other (bureaucrat, steward, etc)

1%



None of the above

57%

Have you ever had other experiences with open-access journals? Choose all that apply:



Publishing an article

23%



Editor

6%



Administrator

0



None of the above

68%

In performance evaluations, researchers are often credited for reviewing academic as

service for professional communities. However, there is much less recognition for reviewing

Wikipedia articles: although 32 out of 120 respondents reported that faculty members in their

department were credited for time spent reviewing articles in open-access journals, only four

respondents reported that faculty members were credited for time spent reviewing Wikipedia

articles related to their academic careers.

Another interesting finding is that the researchers’ experiences with open-access journals

are correlated with their Wikipedia experiences, e.g., those who have not had any open-access

journal experiences are more likely to have not had any Wikipedia experience (N = 120, DF = 1,

Pearson chi-square statistics = 12.81, p < .001). Those who have not read open-access journals

are less likely to have edited Wikipedia as a non-registered (N = 109, DF = 1, Pearson chi-square

statistics = 11.69, p = .001) or a registered user (N = 109, DF = 1, Pearson chi-square statistics =

8.42, p = .004). They are more likely to report no Wikipedia experiences (N = 109, DF = 1,

Pearson chi-square statistics = 14.14, p < .001).

On the other hand, those who have edited Wikipedia as non-registered users are more

likely to have cited an open-access journal in a peer-reviewed article (N = 109, DF = 1, Pearson

chi-square statistics = 4.19, p = .041). Those who have cited Wikipedia are more likely than

expected to have cited open-access journals (N = 106, DF = 1, Pearson chi-square statistics =

5.35, p = .021). Interestingly, those who have published articles in open-access journals are more

likely to disagree that the Wikipedia model increases diversity in points of view compared to

open-access journals, whereas those who have not published in open-access journals are more

likely to think the opposite (N = 97, DF = 1, Pearson chi-square statistics = 5.20, p = .023).

Effects of Respondents’ Wikipedia Experiences

There are statistically significant relationships between researchers’ experiences with

Wikipedia and their opinions on the advantages of Wikipedia over open-access journals:

previous experience with Wikipedia seems to lead to more positive responses. Researchers who

have edited Wikipedia are more likely to believe that Wikipedia has the advantage of making

articles available before validation (Non-registered: N = 120, DF = 1, Pearson chi-statistics =

4.85, p = .028, Registered: N = 120, DF = 1, Pearson chi-statistics = 5.63, p = .018), and making

post-publication corrections (Non-registered: N = 120, DF = 1, Pearson chi-statistics = 5.23, p =

.022, Registered: N = 120, DF = 1, Pearson chi-statistics = 7.01, p = .008) . Researchers who

edited as registered users are also more likely to believe that Wikipedia has the advantage of

offering a wide variety of articles (N = 120, DF = 1, Pearson chi-square statistics = 4.45, p =

.035) and being less expensive (N = 120, DF = 1, Pearson chi-square statistics = 8.28, p = .004).

They are also less likely to believe that Wikipedia has no advantage compared to open-access

journals (N = 120, DF = 1, Pearson chi-square statistics = 3.94, p = .047). People who have

written an article on Wikipedia are more likely to believe that Wikipedia has the advantage of

making post-publication corrections (N = 120, DF = 1, Pearson chi-square statistics = 5.78, p =

.016). Those who have cited Wikipedia are more likely to believe that a Wikipedia article can

become a reliable source (N = 111, DF = 2, Pearson chi-square statistics = 8.91, p = .012).

However, there were two cells with expected count less than 5 for the case of having cited

Wikipedia but disagreeing on it being reliable.

Those who have had no experience with Wikipedia are more likely to believe that

Wikipedia has no advantage compared to open-access journals (N = 120, DF = 1, Pearson chi-

square statistics = 5.23, p = .022). They are also more likely to believe that Wikipedia has

questionable stability because of changes by anonymous or pseudonymous readers (N = 120, DF

= 1, Pearson chi-square statistics = 5.99, p = .014), and is lower quality (N = 120, DF = 1,

Pearson chi-square statistics = 5.90, p = .015).

Researchers’ Wikipedia experiences are also related to their authoring practices and/or

preferences. Those who have had experiences with Wikipedia are more likely to say that they

want an open discussion page for their published papers (N = 115, DF = 2, Pearson chi-square

statistics = 12.62, p = .002).

Effects of Respondents’ Open-Access Journal Experiences

Researchers’ experiences with open-access journals also play a role in their opinions of

Wikipedia. Those who read articles published in open-access journals in their discipline are more

likely to believe that Wikipedia has the advantage of being timelier (N = 109, DF = 1, Pearson

chi-square statistics = 4.37, p = .037) and reducing cost of publication (N = 109, DF = 1, Pearson

chi-square statistics = 4.04, p = .045). Those who have published an open-access journal article

are more likely to believe that publishing in Wikipedia has less impact on academia (N = 120,

DF = 1, Pearson chi-square statistics = 5.44, p = .02) and doesn’t have more diversity in points of

view (N = 104, DF = 2, Pearson chi-square statistics = 10.56, p = .005). In terms of citations,

those who have cited Wikipedia in their peer-reviewed articles are more likely than expected to

have also cited open-access journals (N = 106, DF = 1, Pearson chi-square statistics = 5.35, p =

.021).

Researchers who have little or no experience with open-access journals seem to have

mixed opinions on Wikipedia publishing. Those with no experience with open-access journals

are more likely to believe that Wikipedia has a wider readership (N = 120, DF = 1, Pearson chi-

square statistics = 5.13, p = .023) and is timelier (N = 120, DF = 1, Pearson chi-square statistics

= 4.57, p = .033). They are most likely to neither agree nor disagree that Wikipedia increases

diversity of points of view (N = 120, DF = 1, Pearson chi-square statistics = 10.09, p = .006).

Those who have not published in an open-access journal also tend to believe that publishing on

Wikipedia will be timelier (N = 120, DF = 1, Pearson chi-square statistics = 3.98, p = .046).

However, those who do not read open-access journals are more likely to believe that writing skill

cannot be demonstrated through Wikipedia (N = 99, DF = 2, Pearson chi-square statistics =

10.17, p = .006). When respondents have negative opinions on the peer-review process in open-

access journals, they tend to have negative opinions on Wikipedia publishing. For example, those

who do not believe that open-access journals have a peer-review process of reasonable quality

are more likely to believe that academic publishing in Wikipedia is even less credible (N = 120,

DF = 1, Pearson chi-square statistic = 6.36, p = .041). Those who chose not to comment on the

quality of open-access journal’s peer-review process are more likely than expected to believe

that academic publishing in Wikipedia is less credible. They also tend to believe that intellectual

contribution and writing skills cannot be demonstrated through Wikipedia (N = 110, DF = 4,

Pearson chi-square statistic = 10.38, p = .034; N = 108, DF = 4, Pearson chi-square statistics =

9.57, p = .048). But as there are cells that have expected counts less than 5 in these chi-square

tests, the results cannot be considered valid.

The researchers’ experiences with open-access journals are correlated with their opinions

on the medium. Those who have some experiences with the journals (e.g., read articles, write

articles) are more likely to agree that open-access journals have a peer-review process of

reasonable quality (N = 89, DF = 1, Pearson chi-square statistics = 5.59, p = .018). They are

more likely have cited an open-access journal (N = 109, DF = 1, Pearson chi-statistics = 36.92, p

< .001).

We found that lack of active Wikipedia experiences (editing, etc) could affect opinions

about open-access journal publishing. For example, those who don’t have any active Wikipedia

experiences are less likely to think that open-access journals have a peer-review process of

reasonable quality (N = 89, DF = 1, Pearson chi-square statistics = 5.76, p = .016).

In summary, these results suggest that researchers’ lack of experiences in open publishing

– either through Wikipedia or open-access journals – contribute to their negative opinions on

these models. We also note that this relationship is correlated not causal. In other words, it could

also be that the researchers’ negative opinions on open-access publishing made them choose not

to have any involvement. The key finding here is that experiences contribute to positive opinions

not negative opinions. This is consistent with the work of Rowlands and colleagues (2004), who

in a survey study found a relationship between greater knowledge or experience of open-access

journals and positive attitudes towards these journals.

RQ3: Do the researchers’ academic environments and statuses affect their

perspectives on using Wikipedia for academic publishing? If so, how?

Effects of Respondents’ Academic Environment

The researchers’ academic environment seems to have some impact on their opinions

about Wikipedia publishing. It seems that those whose departments give credit for writing

Wikipedia articles most strongly believe that Wikipedia can demonstrate an author’s knowledge.

However, as there were only 4 respondents whose departments do so, the chi-square test’s

criteria were violated (3 cells have count less than 5). More data are needed.

The academic environment may have an impact on respondents’ experiences with open-

access publishing. Those who reported that their faculty are credited for time spent reviewing

open-access journal articles are more likely to have had experiences with open-access journals

(N = 120, DF = 2, Pearson chi-square statistics =12.07, p= .002), e.g., read (N = 109, DF = 2,

Pearson chi-square statistics =7.52, p = .023) or written an article in an open-access journal (N =

120, DF = 2, Pearson chi-square statistics = 19.24, p < .001). Those who chose not to comment

on their faculty’s policy are more likely to have not had experience with open-access journals.

Effects of Respondents’ Academic Status

We asked respondents to report their current professional status. Chi-square tests showed

that tenured professors are more likely to think that their departments credit faculty members for

time spent on reviewing open-access journal articles, while non-tenured professors are more

likely to think the opposite, and researchers of other positions (e.g., instructor or other in the

questionnaire) are more likely to choose not to answer this question (N = 119, DF = 4, Pearson

chi-square statistics=25.11, p < .001). One explanation is that with lower and less secure job

status people have more conservative feelings about service in the performance evaluation.

Compared to tenure-track faculty members, instructors and ‘other’ academics are more

likely to believe that academic publishing in Wikipedia will be more widely read (N = 119, DF =

2, Pearson chi-square statistics = 8.41, p = .015) and more timely (N = 119, DF = 2, Pearson chi-

square statistics = 8.44, p = .015). They are less likely to think that Wikipedia has “no

advantage” over open-access journals (N = 119, DF = 2, Pearson chi-square statistics = 8.39, p =

.015). We asked respondents to explain any other advantages they believe Wikipedia has over

open-access journals. Of the seven respondents who left comments, all but one held non-tenure

track positions (e.g., instructors). These comments are mainly related to Wikipedia’s larger and

broader readership.

RQ4: Does the researchers’ knowledge of Wikipedia’s peer-review process affect

their perspectives on using Wikipedia for academic publishing? If so, how?

Respondents’ Knowledge of Wikipedia’s Peer-Review Process

Wikipedia has several peer-review processes, including Good Article Nominations, Peer

Review, and A-Class Review; the most significant and well-established review process is

Featured Article Candidates (FAC). Approximately 0.1% of articles on Wikipedia (3,825

articles) have successfully undergone the FAC process

(

http://en.wikipedia.org/wiki/Wikipedia:Featured_article_candidates

), which evaluates articles

based on their prose, coverage, neutrality, sourcing, images, and adherence to style guidelines

(

http://en.wikipedia.org/wiki/Wikipedia:Featured_article_criteria

). 54% of respondents were

aware that Wikipedia had a peer-review process; of these, 35% considered it to be less rigorous

than that of scholarly journals. When asked if Wikipedia’s FAC process was of comparable

rigorousness to the review process of an open-access journal, 51% did not think so, 17% did not

give their opinion, and 24% neither agreed nor disagreed. However, their written explanations of

the process demonstrate lack of familiarity and misconceptions with the Wikipedia model. In an

open-ended question, we asked respondents to explain the peer-review process that Wikipedia

uses. 53 responded to this question. Six of them indicated that they didn’t know or could not give

specifics, while others were vague or incorrect (e.g., “watchdog process”, “country review”).

Most reviewers of featured articles edit under pseudonyms rather than real names. We

wondered whether this would be perceived by the academic community as analogous to a

“double-blinded” review process, given that the identities of both authors and reviewers are

unknown. However, only 25% of respondents considered it to be so.

Only 24 respondents felt comfortable with having their research papers reviewed by

Wikipedia users. Of the 50 respondents who provided reasons why they did not feel comfortable,

34 expressed concerns about reviewers’ background (“not at all clear what their credentials

are”). There were other concerns as well, such as “content can be used for political purposes”

and “a lot of people will start doing review in a territorial basis”.

We ran chi-square tests between the questionnaire items about the respondents’

knowledge of Wikipedia’s peer-review process and those about their perspectives on using

Wikipedia for academic publishing. We found that those who, prior to this survey, were aware

that Wikipedia had a peer-review process were less likely to think that academic publishing on

Wikipedia would be less credible than open-access journals (N = 117, DF = 2, Pearson chi-

square = 10.212, p = .006) and more likely to think that an author’s knowledge can be

demonstrated through Wikipedia (N = 90, DF = 2, Pearson chi-square = 5.929, p = .052). Those

who agreed that a Wikipedia article can become a reliable source after being reviewed were

more likely to agree that the Wikipedia model increases diversity in points of view compared to

open-access journals (N = 89, DF = 1, Pearson chi-square = 4.621, p = .032), and that

Wikipedia’s quick turnaround on post-publication corrections is its biggest advantage (N = 82,

DF = 1, Pearson chi-square = 6.605, p = .01). Those who tend to agree that FAC is of

comparable rigorousness to the peer-review process of open-access journals were more likely to

think that academic publishing on Wikipedia would have wider readership (N = 74, DF = 1,

Pearson chi-square = 7.796, p = .005).

Discussion

Challenges of Using Wikipedia for Academic Publishing

Our results speak to Fitzpatrick’s (2011)’s viewpoint that changing how we disseminate

research results is more than merely changing the publishing system. Instead, the whole

academic system that influences how we research, write, and review needs to be re-examined.

Our results suggest where some challenges are. It seems that the performance evaluation system

in academia has not adapted well to open-access publishing. A key development would be to

encourage a culture of service to both the community and the public, even among junior faculty

or non-tenure-track instructors. The open-access movement entails the promotion and

promulgation of unrestricted knowledge (Velterop, 2003), which is also one of the primary

motivators cited by Wikipedia editors as their reason for contributing to the project (Glott et al.,

2010).

Wikipedia’s web structure

The Wikipedia search engine does not allow one to search for only featured articles, and

they are not automatically distinguished from non-featured articles in the topic-based category

trees. Readers must browse through a list of all Featured Articles according to broad categories

(

http://en.wikipedia.org/wiki/Wikipedia:Featured_articles

), or search for a title using a search

bar.

The respondents were asked to suggest the features that they think Wikipedia’s web

structure should include; nine features and an open field were provided. 26 respondents made

suggestions. The three most selected features were all related to searching, with the most selected

being “making the promoted peer-reviewed articles searchable from university libraries”. Table

5 provides details of the responses.

Table 5. Respondents’ suggestions on the Wikipedia web structure for academic publishing

… Wikipedia web structure should (please mark all that apply):

No. of the

Responses

Provide separate hierarchical levels for promoted peer-review articles, as opposed to
having them in the same hierarchy as regular non-peer-reviewed articles

14

Provide additional hierarchical levels for different research topics

8

Allow researchers to form their research categories as a collective action to categorize and
classify their papers

13

Make the promoted peer-reviewed articles searchable from university libraries

17

Make the promoted peer-reviewed articles searchable from Google Scholar

14

Have a “Wikipedia Scholars” portal, allowing its promoted peer-reviewed articles to be
searched

14

Protect the promoted peer-reviewed academic papers from being edited by anonymous
readers

11

Protect the promoted peer-reviewed academic papers from being edited by anyone

12

Have a separate Wiki site just for academic papers but preserve the current web structure
of Wikipedia

7

Other challenges

The last but not the least identified challenge is related to the perceived negatives

associated with Wikipedia publishing. The respondents were asked to explain their biggest

concern of using Wikipedia for academic publishing. Recurring themes included: editing by non-

experts (e.g., “ability of non-experts and quacks to edit”); lack of stability (e.g., “If articles are

continually updated, at some point, the article might not look the way it did at the beginning”);

lack of authorial control; lack of rigor and credibility; and lack of recognition by the academic

community. The last theme is related to the challenge of adapting well-established academic

systems to accommodate open-access models, but the others were all related to the researchers’

negative perceptions of Wikipedia publishing. These attitudes persist despite studies indicating

that Wikipedia is comparable in quality to traditional encyclopedias (Giles, 2005) or professional

databases (Rajagopalan et al., 2010). Positively, the fact that the more experiences with

Wikipedia publishing the more positive the academic researchers’ attitudes suggests that

promoting the academic researchers’ participation in Wikipedia can improve their perspectives

on the idea.

Scholarly communication is crucial in the academic community. Various emerging

technologies have developed to support collaboration and scholarly communication through

informal paths (e.g., Mendeley, ResearchGate). However, the formal scholarly communication

model has not changed much between traditional and open-access journal publishing, despite the

known shortcomings of these models, e.g., long review process, cost, etc. Elbeck and

Mandernach (2008) suggest methods to improve open-access journal quality by reducing effort

costs and involving readers as reviewers through an interactive interface. Their suggestions have

some similarities to the Wikipedia model. Wikipedia’s peer review model is different from that

of academic publishing: reviewers sign up for articles to review, during the review process

author(s) can communicate with the reviewers freely and make changes in the article, and the

process is typically completed in weeks as opposed to months or longer. These differences, as

well as the technical interface used and the emphasis on collaborative editing, offer some

potential for updates to the scholarly communication model.

Originality in Wikipedia

One of the primary concerns with the use of Wikipedia as a venue for academic

publishing is the site’s discouragement of original research (per its policy WP:NOR). There are

two obvious potential approaches for academics: they may work on Wikipedia within this policy

– which implies building upon previously cited sources in creating a knowledge base, an

approach covered in the current model of publishing by the literature review or the textbook – or

they may adopt Wikipedia’s model of peer review and structure into an academic venue that

allows original research (a limited form of which already exists in Wikiversity, a partner site to

Wikipedia).

The first approach is advocated by such organizations as the Association for

Psychological Science (APS) and the American Sociological Association (ASA), both of which

have “Wikipedia Initiatives” to encourage members to engage with the site. The APS

(“Wikipedia Initiative”) suggests psychology professors use Wikipedia for coursework,

assigning students to improve psychology-related articles. The ASA

(http://www.asanet.org/footnotes/nov11/wikipedia_1111.html) promotes direct editing by

academics as well as students, advocating a “duty” to improve coverage of sociological topics in

a manner understandable to the general public. Similarly, the Journal of Medical Internet

Research (Heilman et al., 2011) notes that Wikipedia is a “key tool for global public health

promotion”, while PLoS Computational Biology (Logan et al., 2010) encourages those in the

biomedical fields to “share your expertise”. More broadly, the place of genres like the textbook

and literature review is well-established in the academic community (Montuori, 2005), and the

field of “publicly engaged academic work” is growing in prominence (Ellison and Eatman,

2008), both of which suggest the value of this approach.

The second approach would adopt the advantages of the Wikipedia system – particularly

timeliness and transparency – while still incorporating original scholarship by creating a

publication venue based on the Wikipedia model. This approach has already been partially

trialed by some scientific subdisciplines. For example, WikiGenes (http://www.wikigenes.org/)

promotes collaborative editing in the biomedical field (Hoffman, 2008). WikiProteins and

WikiPathways both enable collective curation and annotation of data sets to facilitate correlation

and semantic association (Mons et al., 2008, Pico et al., 2008). The general concept of open peer

review has been trialed by such journals as the British Journal of Psychology and British Journal

of Medicine (Walsh et al., 2000; Smith, 1999). However, no known publisher has yet combined

elements of open peer review with the wiki collaborative software. We advocate further study to

determine which approach would be most amenable to both the scholarly and Wikipedia

communities.

Study Implications

In our study, we found several implications for future work. First, we noted that increased

familiarity with both open-access journals and Wikipedia is associated with more positive

opinions of these venues. We suggest that both venues expand their outreach efforts to reach

more members of the academic community and enhance their position as viable alternatives to

the traditional journal model. We further suggest open-access journals, and even non-open-

access electronic journals, consider adopting elements that respondents identified as advantages

of Wikipedia, particularly simplifying the process of post-publication correction and reader

comment. For their part, Wikipedia could amend their policies to address some of the concerns

raised about the use of Wikipedia as a publishing venue. In particular, they might increase efforts

to enforce rigor and maintain the quality of reviewed articles. It would also be essential to

integrate reviewed Wikipedia articles into scholarly search engines like Google Scholar and into

library search capabilities. In fact, given the findings by Dooley (2010) that faculty already use

Wikipedia in both teaching and research, we advocate integration as a first step. Finally, we

support further study into the place of Wikipedia in the open publishing movement and the

relationship between Wikipedia and academia.

Study Limitations

The major limitation of our study is the sample size. The power analysis using GPower

3.1.shows that, with statistical power at .95, for chi-square tests with DF = 1, 2, or 4, we would

need a sample size of 145, 172, or 207 respectively to detect medium effect, and 52, 62, or 75 to

detect big effect. This means we could only detect big effect in this study at statistical power of

.95. We call for larger studies to further explore this topic.

Concluding Remarks

We administered an online survey from October 2011 to April 2012 to understand the

academic community’s experiences with Wikipedia and open-access journals, and their opinions

on such publishing models. Our survey showed that respondents acknowledged some benefits of

publishing in Wikipedia, but that there are concerns about the qualification of Wikipedia users as

reviewers, the conflict between original research and Wikipedia policy, and the suitability of

academic content for Wikipedia readers. Interestingly, there was no observed complaint about

the unique aspects of the Wikipedia peer-review model, such as its pull style for identifying

reviewers (Wikipedians volunteering to review specific papers as opposed to editors assigning

reviewers), the flexible communication flow between reviewers and author(s), and ambiguous

authorship. These results suggest that such features may be explored in the academic publishing

model regardless of whether Wikipedia would be used as the venue or not.

References

Bjørk, B.C. (2004), “Open access to scientific publications: an analysis of the barriers to change”,

Information Research, Vol. 9 No. 2, available at:

http://InformationR.net/ir/9-2/paper170.html

(accessed

10 March 2013).

Black, E.W. (2008), “Wikipedia and academic peer review: Wikipedia as a recognised medium

for scholarly publication?”, Online Information Review, Vol. 32 No. 1, pp. 73-88.

Chen, H. (2010), “The perspectives of higher education faculty on Wikipedia”, Electronic

Library, Vol. 28 No. 3, pp. 361-373.

Dooley, P.L. (2010), “Wikipedia and the two-faced professoriate”, in Proceedings of the 6

th

International Symposium on Wikis and Open Collaboration in Gdańsk, Poland, 2010,

ACM, USA, doi:

10.1145/1832772.1832803

(accessed 10 March 2013).

Eijkman, H. (2010), “Academics and Wikipedia: reframing web 2.0+ as a disruptor of traditional

academic power-knowledge arrangements”, Campus-Wide Information Systems, Vol. 27

No. 3, pp. 173-185.

Elbeck, M., and Mandernach, J. (2008), “Expanding the value of scholarly open access e-

journals”, Library and Information Science Research, Vol. 30 No. 4, pp. 237-241.

Ellison, J., and Eatman, T.K. (2008), “Scholarship in public: knowledge creation and tenure

policy in the engaged university”, Imagining America, Paper 16, available at:

http://surface.syr.edu/ia/16

(accessed 10 March 2013).

Fitzpatrick, K. (2011), Planned Obsolescence: Publishing, Technology, and the Future of the Academy,

New York University Press, New York, NY.

Forte, A., and Bruckman, A. (2006), “From Wikipedia to the classroom”, in

Proceedings of the 7th

international conference on Learning sciences, ICLS, Bloomington, IN, pp. 182-188.

Frishauf, P. (2006), “Are traditional peer-reviewed medical articles obsolete? A pitch for the

Wikipedia concept”, Medscape General Medicine, Vol. 8 No. 1, p. 5.

Giles, J. (2005), “Internet encyclopaedias go head to head”, Nature, Vol. 438, pp. 900-901.

Glott, R., Schmidt, P., and Ghosh, R. (2010), “Wikipedia survey – overview of results”, UNU-

MERIT, available at:

http://wikipediasurvey.org/

(accessed 10 March 2013).

Heilman, J.M., Kemmann, K., Bonert, M., Chatterjee, A., Ragar, B., Beards, G.M., Iberri, D.J.,

Harvey, M., Thomas, B., Stomp, W., Martone, M.F., Lodge, D.J., Vondracek, A., de

Wolff, J.F., Liber, C., Grover, S.C., Vickers, T.J., Mesko, B., and Laurent, M.R. (2011),

“Wikipedia: a key tool for global public health promotion”, Journal of Medical Internet

Research, Vol. 13 No. 1, p. e14, available at:

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3221335/

(accessed 10 March 2013).

Hoffman, R. (2008), “A wiki for the life sciences where authorship matters”, Nature Genetics,

Vol. 40, pp. 1047-1051.

Klobasa, J.E., and Clydec, L.A. (2010), “Beliefs, attitudes and perceptions about research and

practice in a professional field”, Library & Information Science Research, Vol. 32 No. 4,

pp. 237–245.

Lam, S.K., Uduwage, A., Dong, Z., Sen, S., Musicant, D.R., Terveen, L., and Riedl, J. (2011),

“WP:Clubhouse? An exploration of Wikipedia’s gender imbalance”,

in Proceedings of the

7th International Symposium on Wikis and Open Collaboration in Mountain View, USA, 2011,

ACM, USA, pp. 1-10.

Logan, D.W., Sandal, M., Gardner, P.P., Manske, M., and Bateman, A. (2010), “Ten Simple

Rules for Editing Wikipedia”, PLoS Computational Biology, Vol. No. 9, p. e1000941.

Mons, B., Ashburner, M., Chichester, C., van Mulligen, E., Weeber, M., den Dunnen, J., van

Ommen, G.J., Musen, M., Cockerill, M., Hermjakob, H., Mons, A., Packer, A., Pacheco,

R., Lewis, S., Berkeley, A., Melton, W., Barris, N., Wales, J., Meijssen, G., Moeller, E.,

Roes, P.J., Borner, K., and Bairoch, A. (2008), “Calling on a million minds for

community annotation in WikiProteins”, Genome Biology, Vol. 9 No. 5, p. R89.

Montuori, A. (2005), “Literature review as creative inquiry: reframing scholarship as a creative

process”, Journal of Transformative Education, Vol. 3 No. 4, pp. 374-393.

Moy, C.L., Locke, J.R., Coppola, B.P., and McNeil, A.J. (2010), “Improving science education

and understanding through editing Wikipedia”, Journal of Chemical Education, Vol. 87

No. 11, pp. 1159-1162.

Nicolas, D., Huntington, P., Dobrowolski, T., and Rowlands, I. (2006), “Ideas on creating a

consumer market for scholarly journals”, Learned Publishing, Vol. 19 No. 4, pp. 245-

249.

Papin-Ramcharan, J., and Dawe, R.A. (2006), “The other side of the coin for open access

publishing: a developing country view”, Libri, Vol. 56 No. 1, pp. 16-27

Park, T.K. (2011), “The visibility of Wikipedia in scholarly publications”, First Monday, Vol 16

No. 8, available at:

http://firstmonday.org/ojs/index.php/fm/article/viewArticle/3492

(accessed 20 May 2013).

Peters, P. (2007), “Going all the way: how Hindawi became an open access publisher”, Learned

Publishing, Vol. 20 No. 3, pp. 191-195.

Pico, A.R., Kelder, T., van Iersel, M.P., Hanspers, K., Conklin, B.R., and Evelo, C. (2008),

“WikiPathways: Pathway Editing for the People”, PLoS Biology, Vol. 6 No. 7, p. e184.

Rajagopalan, M., Khanna, V., and Stott, M. (2010), “Accuracy of cancer information on the

internet”, Journal of Clinical Oncology, Vol. 28 No. 15, p. 6058.

Rowlands, I. (2007), “Electronic journals and user behavior: a review of recent research”,

Library & Information Science Research, Vol. 29 No. 3, pp. 369-396.

Rowlands, I., Nicholas, D., Huntington, P. (2004), “Scholarly communication in the digital

environment: what do authors want?”, Learned Publishing, Vol. 17 No. 4, pp. 261-273.

Smith, R. (1999), “Opening up BMJ peer review”, British Medical Journal, Vol. 318, available

at:

http://www.bmj.com/content/318/7175/4

(accessed 10 March 2013).

Solomon, D.J. (2008), Developing Open Access Journals: A Practical Guide, Chandos

Publishing, Cambridge, UK.

Stvilia, B., Twidale, M.B., Smith, L.C., and Gasser, L. (2005), “Assessing information quality of

a community-based encyclopedia”, in Naumann, F., Gertz, M., Mednick, S. (eds.),

Proceedings of the International Conference on Information Quality, MITIQ, Cambridge,

MA, pp. 442-454.

Stvilia, B., Twidale, M.B., Smith, L.C., and Gasser, L. (2008), “Information quality work

organization in Wikipedia”, Journal of the American Society for Information Science and

Technology, Vol. 59 No. 6, pp. 983-1001.

Suber, P. (2007), “Flipping a journal to open access”, SPARC Open Access Newsletter, 114,

available at:

http://dash.harvard.edu/bitstream/handle/1/4322572/suber_flipping.html

(accessed 20 May 2013).

Suber, P. (2012), Open Access, MIT Press, Cambridge, MA.

Taraborelli, D., Mletchen, D., Alevlzou, P., and Gill, A.J. (2011), “Expert participation on

Wikipedia: Barriers and opportunities”, in Wikimania 2011, Haifa, Israel, Wikimedia

Foundation, San Francisco, CA, available at:

http://upload.wikimedia.org/wikipedia/commons/4/4f/Expert_Participation_Survey_-

_Wikimania_2011.pdf

(accessed 10 March 2013).

Velterop, J. (2003), “Should scholarly societies embrace open access (or is it the kiss of death)?”, Learned

Publishing, Vol. 16 No. 3, pp. 167-169.

Viégas, F.B., Wattenberg, M., McKeon, M.M. (2007), “The hidden order of Wikipedia”, Online

Communities and Social Computing, Vol. 4564, pp. 445-454.

Völkel, M., Krötzsch, M., Vrandecic, D., Haller, H., and Studer, R. (2006), “Semantic Wikipedia”, in

Proceedings of the 15th international conference on World Wide Web, ACM, USA, pp. 585-594.

Walsh, E., Rooney, M., Appleby, L., and Wilkinson, G. (2000). “Open peer review: a

randomized controlled trial”, The British Journal of Psychiatry, Vol. 176, pp. 47-51.

“Wikipedia Initiative”, Association for Psychological Science, available at:

http://www.psychologicalscience.org/index.php/members/aps-wikipedia-initiative

(accessed 10 March 2013).

Xiao, L., and Askin, N. (2012), “Wikipedia for academic publishing: advantages and

challenges”, Online Information Review, Vol. 36 No. 3, pp. 359 – 373.

Yaari, E. (2011), “Information quality assessment of community generated content: a user study of

Wikipedia”, Journal of Information Science, Vol. 375, pp. 487–498.

Yasseri, T., Sumi, R., and Kertész, J. (2012), “Circadian Patterns of Wikipedia Editorial Activity: A

Demographic Analysis”, PLoS ONE, Vol. 7 No. 1, p. e30091.