Introduction 1 2 3 3 7 1 2 3 3 2 2 3 8 9 8 1 3 1 1 1 κ κ κ 1 10 3 10 1 1 11 1 12 14 15 17 This study aims to assess the feasibility and the validity of the ACA to derive weights for iQoL domains. Furthermore, agreement of the weighting procedures performed by the ACA and the DW will be assessed. Because it would not be feasible to use the JA as well, the ACA was only compared with the DW. Since JA is rarely used and the scientific community has overwhelmingly embraced the DW, despite the lack of data on its validity, we wished to compare ACA and DW. Further, relationships of the resulting iQoL index scores with scores on a VAS for QoL and for iQoL may give more insight into the validity of both weighting methods. Methods Patients To assess the feasibility and validity of the ACA to derive iQoL weights, a convenience sample of outpatients with rheumatoid arthritis or cancer who were treated at the Leiden University Medical Center were asked to participate in the study. We selected patients with rheumatoid arthritis (RA) who received multidisciplinary day treatment or had an appointment with the specialized nurse consultant about their treatment. Patients with cancer were selected if they received curative radiotherapy at the time of the study or had received curative radiotherapy in the 6 months before. The latter patients received a letter at home in which the head of the Department of Radiotherapy asked them to participate in the study. These groups were selected because they presented at the clinic with symptoms (RA) or were known to have side effects (cancer) impacting their quality of life. All patients were only included in the study after they had given their informed consent. The Medical Ethical Committee of the Leiden University Medical Center approved the research protocol. Interview and questionnaire 3 1 18 N n N N n 19 20 Finally, patients filled out a questionnaire that addressed demographic factors such as age, sex, marital status, education, and religion. Computation of iQoL weights and index scores 18 16 19 16 The index score for iQoL is a weighted score, calculated by multiplying the functioning scores for the domains with their corresponding weights as derived by the DW and the ACA method, and summing these. Further, an unweighted index score was calculated by simply summing up the functioning scores and dividing by 5. Feasibility and validity The feasibility of the ACA was assessed by measuring the percentage of patients that were able to finish the task, by measuring the administration time, and by asking the patients how they evaluated the ACA with respect to difficulty and acceptability. We asked patients two quantitative items about the method being confronting (very, somewhat, not) or being unpleasant versus fun (1 = very unpleasant, 5 = much fun). Further, we also coded qualitative statements about the ACA being upsetting (comments such as ‘nasty’, ‘mean’, ‘suicide questions’, ‘I felt like a prisoner’). As a measure of difficulty we also assessed how often patients chose the worst option in a dominant pair, a pair in which one of the scenarios was on all domains better than the other. The validity of the ACA was first studied by assessing the number of inconsistencies in the rank ordering of utilities, that is, the number of pairs in which the utilities for two levels of functioning were ranked opposite to the direction of the levels of functioning. We analyzed whether age, health status, and level of education were related to answers to dominant pairs and the number of inconsistencies by Pearson’s correlation and analysis of variance. Next, we assessed whether patients were willing to trade off a decrease from the best to the second-best functioning level on their most important domain with the largest improvement on their second important domain. This was done by computing the ratio between the difference in utilities for the largest benefit in the second important domain and the difference in utilities for the two highest functioning levels of the most important domain. A value smaller than 1 was taken as indicating that the patient was not willing to trade off decline in the most important domain for any benefit in the second important domain. We similarly assessed whether patients were willing to trade off a decrease from the second to the third functioning level on their most important domain with the highest improvement on their second important domain. Agreement 21 Results Patients n n n 1 Table 1 N  N Sex     Female 24 (51)  Living arrangement     With partner 41 (87)  Education a 19 (40)  Religion     Religious 25 (53)  Diagnosis     Rheumatoid arthritis 20 (43)      Breast cancer 11 (23)      Prostate cancer 11 (23)      Rectal cancer 5 (11)  Place of interview     Hospital 32 (68)     At home 15 (32) a ACA: feasibility and validity 2 Table 2 N 1  N Partner 22 (47) Children 12 (26) Partner and children 13 (28) Family 18 (38) Own health 30 (64) Health of partner 5 (11) Social contacts and friendship 21 (45) Transportation 10 (21) Independence 5 (11) Hobbies and relaxation 23 (49) Work 21 (45) Feelings 5 (11) Activities of daily life 5 (11) Sports and holidays 11 (23) Other 34 (72) Total 235 1 The ACA survey took on average 20 min (range 10–37 min). All patients were able to finish the ACA. Five patients (11%) were in some sense upset about the ACA survey, and a further three (6%) judged the questions as very confronting. An additional patient found it very unpleasant. For example, when a patient had nominated own health and relationship with the partner as domains, the ACA could offer one scenario in which the patient’s health was very good whereas the relationship with the partner was poor, and another scenario in which the patient’s health was very poor whereas the relationship with the partner was very good. Some patients became upset when they had to make a choice between such options. Patients were offered, on average, 2.9 dominant pairs (range 0–6). In such pairs, four times (3%) the worst option was chosen and once (1%) the patient had no preference. The four patients who chose the worst option had the same level of education and were of the same age as the other patients. 3 r P n P Table 3 a  Most important Least important N Domain 1 Domain 2 Domain 3 Domain 4 Domain 5 Number of inconsistencies in rank order utilities out of six pairs per domain Mean (SD) 0.2 (0.6) 0.2 (0.5) 0.8 (1.0) 1.0 (1.2) 1.7 (1.1) 3.9 (2.1)  N N N N N N No inconsistencies 41 (87) 41 (87) 25 (53) 23 (49) 8 (17) 138 (59) One inconsistency 2 (4) 4 (9) 7 (15) 9 (19) 11 (23) 33 (14) Two inconsistencies 4 (9) 2 (4) 13 (28) 10 (21) 17 (36) 46 (20) Three inconsistencies   2 (4) 4 (9) 9 (19) 15 (6) Four inconsistencies    1 (2) 2 (4) 3 (1) a One out of 43 patients was not willing to trade off a decline from the best to second-best functioning level on the most important domain for the largest benefit on the second important domain. Further, all patients were willing to trade off a decline from the second to the third level of the most important domain for the largest benefit on the second important domain. Agreement between DW weights and ACA weights 4 Table 4 Absolute differences between DW weights and ACA weights  a a N Domain 1 Domain 2 Domain 3 Domain 4 Domain 5 Mean (SD) 7.5 (5.3) 4.7 (4.2) 4.4 (3.5) 4.6 (4.1) 5.9 (4.9) P Absolute difference between ACA and DW  N N N N N N Less than 5 points 17 (36) 31 (66) 28 (60) 31 (66) 26 (55) 133 (57) 5–10 points 13 (28) 12 (26) 16 (34) 13 (28) 12 (26) 66 (28) More than 10 points 17 (36) 4 (8) 3 (6) 3 (6) 9 (19) 36 (15) a  5 Table 5 Agreement between DW weights and ACA weights  ACA–DW linear correlation ACA–DW intraclass agreement r P b P Domain of individual quality of life a     Most important domain 0.27 0.06 0.23 0.06     Domain 2 0.30 0.04 0.28 0.03     Domain 3 0.43 0.003 0.33 0.01     Domain 4 0.33 0.02 0.28 0.03     Least important domain 0.22 0.14 0.18 0.11 ACA, adaptive conjoint analysis; DW, direct weighting a b f Consequences of weighting method P 6 Table 6 Impact of weighting procedure on index score for individual quality of life, and on correlations with global quality of life   DW index ACA index Unweighted index r r r Index score for individual quality of life DW-index score 1.0 0.95** 0.92** ACA-index score 0.95** 1.0 0.89** Unweighted index score 0.92** 0.89** 1.0 SEIQoL VAS 0.62** 0.54** 0.63** QoL VAS 0.40* 0.36* 0.33* P P DW, direct weighting; ACA, adaptive conjoint analysis; SEIQoL, schedule for evaluation of individual quality of life; VAS, visual analogue scale; QoL, quality of life Discussion 22 23 ACA was to some extent feasible, because the ACA took on average 20 min and all patients were able to finish it. However, one in five patients judged the ACA task as upsetting, very confronting, or very unpleasant. The paired comparison task, despite working well for some domains, turned out not to be appropriate for some others. Especially choosing between two domains that are dear to the patient turned out not to be feasible. Sometimes, the computer offered a dominant pair, mostly resulting from the fact that the utilities of the functioning levels were almost equal. Only seldom was the worst option chosen. This finding shows that almost all patients understood the task of paired comparisons and were able to make a valid choice. A limitation of our procedure was that patients did not rate scenarios with four or five domains. Although the use of such scenarios would have provided for more precise estimates, this had to be balanced against feasibility. Using all five attributes in the pairwise comparisons would also have allowed for the evaluation of full profiles (health states). The goal of this first study on the use of ACA for the SEIQoL was merely to assess its feasibility and to compare ACA weights with DW weights, not to assess full profiles. We therefore preferred to opt for the more feasible approach, which we deemed sufficiently difficult already. Many patients gave inconsistent answers, but these inconsistencies mostly occurred on domains 3 and lower, and especially on the least important domain. For the two most important domains, the large majority of patients had utilities ordered in the same direction as the corresponding levels of functioning. In the case of less important domains, the differences between utilities of successive functioning levels are probably small, which leads to inconsistent answers. 17 1 24 28 28 29 Our study has some limitations. The number of patients was relatively small, due to the qualitative character and semistructured design of the SEIQoL interview. However, larger numbers would not have changed the conclusion of our paper. A problem in measuring the validity of the SEIQoL is that a gold standard for iQoL is lacking. JA has been considered the standard weighting method, but due to its complex nature has been replaced in the field by DW. Unfortunately, it was not possible to include both the JA and the ACA, because of the cognitive burden imposed on the patients. Despite the patients’ reluctance to perform the ACA, our study gave clear insight into the problems of deriving weights for iQoL domains. Our findings show that weighting has almost no effect on the association between the SEIQoL and global iQoL, although incorporating weights for domain functioning led to slightly higher iQoL index scores than the unweighted index score. Selecting and weighting domains are clearly confounded. Because of the high correlations between the weighted and unweighted index scores, it seems sufficient to use the unweighted index score as a measure for global iQoL.