Identifying issues for qualitative research As the most comprehensive archive of written material representing our world and people's opinions, concerns, and desires (in industrialized countries), the Internet can be used to identify `issues' for qualitative (descriptive) research and to generate hypotheses. Material published on the Internet may be a valuable resource for researchers desiring to understand people and the social and cultural contexts within which they live--outside of experimental settings--with due emphasis on the interpretations, experiences, and views of `real world' people. Reviews of information posted by consumers on the Internet may help to identify health beliefs, common topics, motives, information, and emotional needs of patients, and point to areas where research is needed. Comparing recommendations found on the Web against evidence-based guidelines is one way to identify areas where there is a gap between opinion and evidence, or where there is a need for clinical innovation. The accessibility of information for analysis and the anonymity of the Internet allow researchers to analyse text and narratives on Web sites, to use newsgroups as global focus groups, and to conduct interviews and surveys via e-mail, chat rooms, Web sites, or newsgroups.Topics suited to qualitative research include: Analysis of interactive communications (e.g. e-mail). Study of online communities (virtual self-help groups, newsgroups, mailing lists). Investigation of communication processes between patients and professionals. Study of consumer preferences, patient concerns, and information needs. 1-2 3 Passive analysis: For example, studying information on Web sites or interactions in newsgroups, mailing lists, and chat rooms--without researchers actively involving themselves. Active analysis: Also called participant observation; the researcher participates in the communication process, often without disclosing their identity as researcher. For example, they may ask questions in a patient discussion group implying that she or he is a fellow patient. Such studies often involve elements of deception, unless the researcher is a sufferer him- or herself. Interviews and surveys: See below. 1 Using the Internet for surveys Using the Internet for surveys requires an awareness of methodologies, selection bias, and technical issues. Methodological issues Internet-based surveys may be conducted by means of interactive interviews or by questionnaires designed for self-completion. Electronic one-to-one interviews can be conducted via e-mail or using chat rooms. Questionnaires can be administered by e-mail (e.g. using mailing lists), by posting to newsgroups, and on the Web using fill-in forms. When e-mail is used to administer questionnaires, messages are usually sent to a selected group with a known number of participants, thus allowing calculation of the response rate. Surveys posted to newsgroups may request that the completed questionnaire is posted back to the researcher, but it is impossible to know who and how many people read the questionnaire. If Web-based forms are used, questionnaires can be placed in a password-protected area of a Web site (i.e. participation by invitation or registration only), or alternatively they may be open to the public (i.e. any site visitor can complete the survey). The latter option makes calculation of a response rate more difficult but not impossible: the number of people who access (without necessarily completing) the questionnaire is counted and used as the denominator. Web-based surveys have the advantage that the respondent can remain anonymous (as opposed to e-mail surveys, where the e-mail address of the responder is revealed). Furthermore, they are very convenient for the researcher, as responses can be directly stored in a database where they are immediately accessible for analysis. 4-10 11 12 13 14 15 Guidelines for Web-based surveys Scenarios that may be suitable for a Web-based survey Respondent features: Respondents are already avid Internet users; e-mail addresses known for reminder messages. Respondents are enthusiastic form fillers; will not require monetary incentives. Need for respondents covering a wide geographical area (e.g. rare clinical special ties, diseases). Respondents are known to match non-respondents and even non-Internet users on key variables. Survey features: Need for complex branching, interactive questionnaire or multimedia as part of the survey instrument. Survey content will evolve fast (e.g. Delphi method surveys use repeating rounds of revised questionnaires delivered over a short period, incorporating aggregate results from previous rounds until convergence is achieved). Intent is to document bizarre, rare phenomena whose simple occurrence is of interest. No need for representative results: collecting ideas vs. hypothesis testing. Investigator features: Limited budget for mailing and data processing, but good in-house Web skills. Precautions can be taken against multiple responses by same individual, password sharing. Web survey forms have been piloted with representative participants and demonstrate acceptable validity and reliability with most platform, browser, and Internet access provider combinations. Data is required fast in a readily analysed form. Scenarios that are unsuitable for a Web-based survey Respondent features: Target group is under-represented on Internet; e.g. the underprivileged, elderly people. Target group is concerned, however unreasonably, about privacy aspects. Target group requires substantial incentives to complete the survey. Need for a representative sample. Survey features: Need for very accurate timing data on participants (inaccuracies in the range of seconds are added due to network transmission times, unless JavaScript or Java applets are used; see Glossary) or observational data on participants. An existing paper instrument has been carefully validated on target group. Need to capture qualitative data or observations about participants. Wish to reach the same group of participants in the same way months or years later. Investigator features: Limited in-house Web or Java expertise but existing desktop publishing and mailing facility. 16 4 5 17-19 Box 1 20 Selection bias In `open' surveys conducted via the Internet where Web users, newsgroup readers, or mailing list subscribers are invited to participate by completing a questionnaire, selection bias is a major factor limiting the generalizability (external validity) of results. Selection bias occurs due to: The non-representative nature of the Internet population. 21 20 22 Technical issues Box 2 Technical issues in implementing Web-based surveys Use of `cookies' Cookies can assign a unique identifier to every questionnaire viewer, useful for determining response and participation rates, and for filtering out multiple responses by the same person. As cookies may be regarded with suspicion, we recommend that researchers openly state that cookies will be sent (and the reasons for this); set the cookie to expire on the day that data collection ceases; and publish a privacy policy. Measuring response time The time needed to complete a questionnaire can be readily calculated by subtracting the time a form was called up by the browser from the time it was submitted using an automatic time-stamp. The response time may be used to exclude respondents who fill in the questionnaire too quickly: this may identify hoax responses, where respondents don't read the questions. Avoiding missing data Forms can be configured to automatically reject incomplete questionnaires and point out missing or contradictory items. Checks can be made on the client (p. 9) prior to submission, or following submission to the server (where incomplete responses can also be analysed, e.g. during a questionnaire pilot). Maximizing response rate 23 24 25 15 24 Randomizing items Scripting languages may be used to build dynamic questionnaires (as opposed to static forms) that look different for certain user groups or which randomize certain aspects of the questionnaire (e.g. the order of the items). This can be useful to exclude possible systematic influences of the order of the items upon responses. Ethical issues 1 26-31 32 In qualitative research on the Web, informed consent is required when: Data are collected from research participants through any form of communication, interaction, or intervention. 33 34 35 In any case, researchers should make themselves familiar with the virtual community they are approaching; i.e. read the messages in a newsgroup for some time (`lurking'). Under no circumstances should researchers blindly spam (p. 31) or cross-post requests for research participation to various newsgroups. 28 In conducting surveys researchers may obtain informed consent by declaring the purpose of the study; disclosing which institutions are behind the study; explaining how privacy will be assured; and detailing with whom data will be shared and how it will be reported, before participants complete the questionnaire. When reporting results, it is obvious that the total anonymity of research participants needs to be maintained. Researchers have to keep in mind that, by the very process of quoting the exact words of a newsgroup or mailing list participant, the confidentiality of the participant may already be broken as Internet search engines may be able to retrieve the original message, including the e-mail address of the sender. It is essential, therefore, to ask participants whether they agree to be quoted whenever there may be a retrievable archive, pointing out the risk that they may be identifiable. Problems can also potentially arise from just citing the name of the community (e.g. of a newsgroup), which may damage the community being studied. Finding methods, protocols, and instruments For laboratory `bench work', researchers often need a protocol for a specific assay method. In addition to the possibility of searching literature databases, there are also specialized services on the Web that can assist in this research, such as MethodsFinder and the `Technical tips online' database at BioMedNet: MethodsFinder (BIOSIS):  http://www.methodsfinder.org/ BioMedNet:  http://www.bmn.com/ http://www.qolid.org/ Online statistical analysis tools are available at the Simple Interactive Statistical Analysis (SISA) Web site, while background information is available within the online book Statistics at square one: SISA (Daan Uitenbroek):  http://home.clara.net/sisa/ Statistics at square one (British Medical Journal Publishing Group):  http://www.bmj.com/collections/statsbk/ Protocols of clinical trials, which may be useful for researchers developing their own protocols, can be found in some of the clinical trial databases available on the Web, as described below. Clinical trials and the Web The Web is being used to assist in the identification and conduction of clinical trials. Identifying trials 36-39 40 A researcher wants to conduct a randomized controlled trial and wants to know whether anyone else is already running one on the same topic. A physician has a patient who is asking about available trials. A patient is looking for ongoing trials. A researcher is looking for possible participants for his trial. A researcher doing a systematic review is looking for unpublished trials. 41 42 43 44 http://www.nabco.org/ 45 http://ClinicalTrials.gov Commercial enterprises also help researchers to recruit patients, or help patients to find clinical trials. For example: CenterWatch Clinical Trials Listing Service (CenterWatch, Inc.):  http://www.centerwatch.com/ ClinicalTrialFinder.com (Clinical Data Technologies Ltd):  http://www.clinicaltrialfinder.com/ Current Controlled Trials (BioMed Central):  http://www.controlled-trials.com 46 Clinical Trials Register (GlaxoSmithKline):  http://ctr.glaxowellcome.co.uk/ Search for Cures (Pharmaceutical Research and Manufacturers of America):  http://www.phrma.org/searchcures/ Finally, information or databases on ongoing clinical trials can often also be found on disease-specific sites. For example: Canadian HIV Trials Network:  http://www.hivnet.ubc.ca/ctn.html CancerNet (National Cancer Institute):  http://cancernet.nci.nih.gov/ Conducting trials on the Web 47 48 49-52 Pre-publishing and publishing research 53 54 Sending and discussing preliminary results on mailing lists. Publishing drafts of scientific papers on pre-print/e-print sites (p. 239) in order to solicit comments and to improve the manuscript. Publishing data and information in databases; e.g. nucleotide sequences in the EMBL/Genbank databases. 55 Current awareness services Electronic editions of paper journals and `stand alone' e-journals typically offer subscriptions to `TOC alerts', where users receive a table of contents by e-mail as soon as a new issue appears.The more sophisticated systems allow users to specify their interests using a controlled vocabulary, enabling the system to screen each newly published article for certain keywords or citations. Examples of current awareness services include: Customised @lerts (British Medical Journal):  http://bmj.com/cgi/customalert/ JournAlert (Doctors.net.uk):  http://www.doctors.net.uk/ Journal Watch (Massachusetts Medical Society):  http://www.jwatch.org/