Judging the Quality and Credibility of Informationin Internet Discussion ForumsReijo SavolainenSchool of Information Sciences, FIN-33014, University of Tampere, Finland. E-mail: Reijo.Savolainen@uta.fiThis exploratory study contributes to research on rele-vance assessment by specifying criteria that are usedin the judgment of information quality and credibility inInternet discussion forums. To this end, 4,739 messagesposted to 160 Finnish discussion threads were analyzed.Of the messages, 20.5% contained explicit judgments ofthe quality of information and credibility in other mes-sages. In the judgments, the forum participants employedboth positive criteria such as validity of information andnegative criteria such as dishonesty in argumentation. Inthe evaluation of the quality of the message’s informationcontent, the most frequently used criteria pertained to theusefulness, correctness, and specificity of information.In the judgment of information credibility, the main crite-ria included the reputation, expertise, and honesty of theauthor of the message. Since Internet discussion forumstend to emphasize the role of disputational discoursequestioning rather than accepting the views presentedby others, mainly negative criteria were used in the judg-ments. The generality of our claims is limited because wechose forums that focused on sensitive and value-ladentopics; future work could explore credibility and qualityjudgment in other forums and forumlike venues such asquestion and answer sites as well as exploring how qual-ity and credibility judgments interact with other aspectsof forum use .IntroductionThe issues of information quality and credibility are gain-ing importance, particularly in the World Wide Web (WWW)context. The WWW provides a unique information-seekingenvironment, but it often lacks quality-control mechanisms.For example, online discussion forums tend to provide mes-sages that draw on vague and conflicting information sources.The questions of information quality and credibility areoften examined in the context of user-generated relevancecriteria. In a major review of relevance studies, Saracevic(2007) emphasized the significance of research that wouldfocus on “real users, in real situations, dealing with real issuesReceived January 12, 2011; revised March 14, 2011; accepted March 15,2011© 2011 ASIS&T•Published online in Wiley Online Library(wileyonlinelibrary.com). DOI: 10.1002/asi.21546of relevance” (p. 2141). The present article contributes to thecontextualist relevance research by focusing on judgmentsof information quality and credibility made in an Internetdiscussion forum. This context is interesting since the judg-ments of information quality and credibility are made innaturalistic settings. Many of the earlier studies on this topichave drawn on assigned search tasks (e.g., Rieh, 2002). Thepresent study is unique since no corresponding investigationshave been made in the context of open online discussionforums. However, the variety of Internet discussion forumsdefies all attempts to draw a statistically representativepicture of information behavior at such arenas. Therefore,an exploratory approach focusing on specific online forumsis justified at the current state of research on informationquality and credibility in the Internet.The present article builds on the solid ground provided bystudies on information quality and credibility (Hilligoss &Rieh, 2008; Metzger, Flanagin, Eyal, Lemus, & McCann,2003; Rieh, 2002; Rieh, 2010; Rieh & Danielson, 2007;Rieh & Hilligoss, 2008). The main research goal of thisstudy is to learn what kind of criteria are used while judgingthe information quality and credibility of messages posted toonline discussion forums. To this end, an empirical study wasconducted by focusing on messages available in a Finnishdiscussion board. The criteria of information quality andcredibility were examined by concentrating on two discus-sion topics: use of natural products (or health food) and issuesof racism. In these areas, a total of 160 discussion threadscontaining 4,739 messages were analyzed. Since the primaryinterest lies in the analysis of the criteria used in the judgmentof information quality and credibility, the issues debated inthese two discussion threads are of secondary importance.Still, this work represents a useful first look at these judg-ments in naturalistic discussion settings and paves the wayfor a variety of future work around other topics and discussionvenues that can build upon these findings.BackgroundA review of earlier research will place the present studyin a broader context. Studies characterizing user-generatedJOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGYrelevance criteria will be briefly discussed first, followed bythe review of studies on information quality and credibility.User-Generated Relevance CriteriaThe first empirical studies on relevance assessment inreal-life settings were made in the 1990s. Barry (1994) pio-neered this field by exploring the criteria mentioned by usersevaluating the information within documents as it relatedto the users’ information need situations. She identified23 categories of relevance criteria, including, for exam-ple, depth/scope, recency, and subjective accuracy/validity.Schamber examined criteria mentioned by occupational usersof weather information sources in real-life information seek-ing and use situations (for the main findings of the study,see Barry & Schamber, 1998, pp. 224–225). Ten summarycategories of criteria were identified, including, for example,currency and specificity of information, and the reliabilityor reputation of the source of information. The pioneeringstudies demonstrated that even though the number of crite-rion categories is rather high, they can be crystallized intoa finite range of user-generated relevance criteria that areshared across users and situations. Schamber and Bateman(1996) identified five major categories of this kind: clarity,currency, credibility, availability, and aboutness. Later stud-ies have provided support for this conclusion (e.g., Crystal &Greenberg, 2006; Maglaughlin & Sonnenwald, 2002).Since the late 1990s, the user-generated relevance criteriahave been explored primarily in the context of web search-ing. These studies have focused on diverse groups such aschildren (Hirsh, 1999), scholars (Rieh, 2002), and universitystudents (Tombros, Ruthven, & Jose, 2005). For