Introduction Publication of the Review 1 2 3 2 2 Retractions The National Library of Medicine (NLM) is a leader in the bibliographic handling of retractions. The Medical Subject Headings (MeSH) contain the concept “retracted publication,” which identifies a citation previously published and now retracted through a formal issuance from the author, publisher, or other authorized agent. In January 2005, the PubMed query “Retracted Publication[Publication Type] AND 1971:2004[edat]” retrieved 619 retracted citations that entered PubMed between 1971 and 2004. Since the query “1971:2004[edat]” retrieves approximately 12.5 million citations, less than 1 in 10000 publications have been retracted. 4 5 6 The objective of this paper is to document the IHCA review as an event in the history of medical publishing, to identify the factors leading to the publicizing of a retracted publication, and to assess the implications. Methods The objectives of this research called for various study methods. The author employed the following three methods: (1) historical processes of collecting documents about a contemporary event and organizing them thematically; (2) ethnographic processes of author participation in the event, personal communication with other participants in the event, interpretation of communications, and construction of models; (3) content analyses based on bibliographic database and Internet searches, coding of the retrieved documents, and tallying of the code frequencies. 7 Various database and Internet searches were employed to study the impact of the review and to quantify the difference between mass media coverage of the original publication and its retraction. LexisNexis Academic databases of health news and general news were searched, as was Google. The queries were designed in an iterative process that began with keywords from the question to be addressed but refined the query based on study of the query retrieval results. The retrieved results were coded, and the coding language was also developed in an iterative process. First, the obvious codes “about the review” and “about the retraction” were introduced. Each retrieved document was classified into a single code by the author. If the retrieved document was not appropriately described by an existing code, then the coding language was augmented. The Web of Science was also queried to identify academic citations, but no citations were identified (data not shown). Most database and Internet searches were conducted in May 2005. To better understand how special the publicity accorded the IHCA review was, this study was extended to three other publications: 2 of these were retracted publications tagged as “Retracted Publication” (1 Cochrane review, but not eHealth related, and 1 non-Cochrane review, but eHealth related), and 1 was a meta-analysis with a scope similar to that of the IHCA review. These 3 reports were identified through PubMed searches. Results The following qualitative results on the impact of the IHCA review are organized into three main sections: scientist reaction, mass media reaction, and patient reaction. The section on scientist reaction considers Cochrane reviewers' reactions and how eHealth scientists responded to the IHCA review in the comment section of the Cochrane database. The mass media section provides the Cochrane retraction and then explores, via LexisNexis and Google results, the reaction of the mass media to the IHCA review. The patient reaction section shares dialogue from patient-patient online discussions that reveals the reactions of patients to the IHCA review. Scientist Reaction The Cochrane Collaboration allows anyone to submit comments to the published reviews. Two scientists' comments on the IHCA review appeared independently on October 28, 2004. Kummervold and Eysenbach criticized the IHCA review for both its protocol and its coding. 8 8 9 10 11 The April 2005 edition of the Cochrane Systematic Reviews did not mention the IHCA review. Royle, the chief executive officer of the Cochrane Collaboration, said that further review of the revised report was ongoing and no date could be given as to when the review might be published (personal communication, April 25, 2005). Mass Media Reaction 12 Most significantly, the UCL bulletin was circulated to information intermediaries that are considered the main entrance to the world's mass media, including AlphaGalileo and EurekAlert. A search on LexisNexis Academic with the query “Elizabeth Murray AND health” for the period June 2004 to May 2005 revealed a total of 15 relevant press reports, in the following categories: Life Science Weekly Law and Health Weekly Health and Medicine Week. The Daily Telegraph Omaha World Herald Time 13 14 British Broadcasting Corporation US News and World Report MedicalNews The Neuroscience for Kids Newsletter 15 Time NLM indexed the IHCA review and entered the citation for it (including its abstract) in PubMed on October 21, 2004. The “Retracted Publication” tag did not, however, appear in PubMed until March 24, 2005. Patient Reaction 16 Time 17 The typical patient with a chronic disease has no formal medical training and is ill prepared to critique a meta-analysis of clinical trials. However, the typical patient is vulnerable to cultural pressures, as they are partially shaped by and reflected in the mass media. Comparison With Another Cochrane Retraction 18 A search on LexisNexis with the query “Brewster AND antihypertensive” for the period November 2004 to May 2005 retrieved no articles in either the “General News–Major Papers” category or the “Medical and Health News” category. 19 Annals of Internal Medicine Comparison With Other Retracted Articles Related to eHealth 20 21 22 A search on LexisNexis Academic with the query “McKinley and surgical and Internet” for the period 1995 to May 2005 revealed no relevant press reports, neither in the “General News–Major Papers” category (three hits were all not relevant to the McKinley article) or in the “Medical and Health News” category. 22 Thus, the only other retraction of a published article appearing in PubMed similar in topic (the Internet) to the IHCA review had a very different pattern of reactions than the IHCA review. A Similar Meta-Analysis on eHealth 23 What has been the impact of the Wantland et al paper and how does that compare to the impact of the IHCA review? The Wantland et al paper was not announced with a press release in EurekAlert. A search on LexisNexis Academic for newspaper articles about the Wantland et al paper retrieves no articles. The queries performed were similar to those performed for the IHCA review and included “Wantland AND health” for 2004 through 2005 in General News/Major Papers. A search was done on Google for “Wantland health Web” on May 24, 2005. Of the first 200 returns, 182 were not relevant. Of the remaining 18 hits, 15 pages contained academic citations to Wantland et al, 2 announced the appearance of the article, and 1 was a personal blog that commented on the article. Thus, most of the Google returns that gave Wantland et al citations are academic in character and very different from the mass media coverage afforded the IHCA review. Discussion 23 This section next presents a framework based on tiers of response. The first tier is medical scientists. The second tier is the mass media spreading medical press releases. The third tier is the patient community reacting to the mass media and the scientists. First-Order Problem 24 Intention of the work Conduct of the studies Design and interpretation of studies Bias from conflict of interest In the case of the IHCA review, the intention was scientifically appropriate, namely to gain further insight about IHCAs through a systematic review. In the other three categories, fault can be found: The errors in the coding of data should not have been made. The coauthors Nazareth and Tai, who are credited with doing the coding, have good enough credentials to not lay the blame on lack of experience: Nazareth is a Professor at UCL and is Scientific Director of the British Medical Research Council's General Practice and Research Framework, and Tai has coauthored several articles over the past two decades that appeared in refereed medical journals. An explanation for the miscoding in terms of experience of the coders is not apparent. 8 9 The reporting of the work suggests possible bias. The authors and their employers have sensationalized a result that catches the media's attention. For some observers, the review appeared biased in that the authors, who are affiliated with medical institutions, concluded that patients should listen to their doctor, instead of seeking help on the Internet. 24 25 26 27 Second-Order Problem 28 29 Time 28 30 31 Third-Order Problem 32 33 If and when the revised IHCA review is published, what could it say that would undo the effect of the original publication? If the conclusion is that IHCAs result in improved clinical outcomes, then the medical profession will want to closely study the protocol and might have grounds to discredit the conclusion. The media trumpeted the IHCA review conclusion partly because it was counterintuitive but was backed by top-notch institutions. If the conclusion becomes intuitive, then the media are unlikely to be interested in it. The reactions to the IHCA review in patient online discussions highlight the importance of virtual communities in helping patients deal with published information. Simple extensions to Web-based, patient, discussion systems could help patients connect to Web-based publications. For instance, when a patient posts a message to a Web-based discussion board, the Web system could parse the message and provide links from the message to relevant articles on the Web. Patients might follow the links and engage in discourse about the validity and implications of the literature. This might lessen the potential ill effects of publications that are wrong or misleading. Conclusions This special medical publishing event was marked by incorrect coding and a desire for maximum publicity. The IHCA review authors, their employers, and the Cochrane Collaboration were responsible for quality control, and failed. The mass media played their part by widely publicizing a sensational message but not reacting to the notice that that sensational message was false. The false result that patients are clinically harmed by interactive applications was very strongly delivered to patients worldwide. The broad lesson to be re-learned is that potentially sensational results should be carefully scrutinized before being sensationalized.