Introduction 2005 1991 2001 1994 1991 2001 1 Table 1 An overview of the Child Behaviour Checklist (CBCL), Conners Parent Rating Scale-Revised:Short version (CPRS-R:S), and the Diagnostic and Statistical Manual of Mental Disorders-4th edition symptoms Scale Symptom CBCL Attention Problems Acts too young for his/her age Can’t concentrate, can’t pay attention for long Can’t sit still, restless, or hyperactive Confused or seems to be in a fog Daydreams or gets lost in his/her thoughts Impulsive or acts without thinking Nervous, high-strung, or tense Nervous movements or twitching Poor school work Poorly coordinated or clumsy Stares blankly CPRS-R:S ADHD-index Inattentive, easily distracted Short attention span Fidgets with hands or feet or squirms in seat Messy or disorganized at home or school Only attends if it is something he/she is very interested in Distractibility or attention span a problem Avoids, expresses reluctance about, or has difficulties engaging in tasks that require sustained mental effort (such as schoolwork or homework) Gets distracted when given instructions to do something Has trouble concentrating in class Leaves seat in classroom or in other situations in which remaining seated is expected Does not follow through on instructions and fails to finish schoolwork, chores, or duties in the workplace (not due to oppositional behavior or failure to understand directions) Easily frustrated in efforts DSM-IV ADHD Inattention Often fails to give close attention to details or makes careless mistakes in schoolwork, work, or other activities Often has difficulty sustaining attention in tasks or play activities Often does not seem to listen when spoken to directly Often does not follow through on instructions and fails to finish schoolwork, chores, or duties in the workplace (not due to oppositional behavior or failure to understand instructions) Often has difficulty organizing tasks and activities Often avoids, dislikes, or is reluctant to engage in tasks that require sustained mental effort (such as schoolwork or homework) Often loses things necessary for tasks or activities (e.g., toys, school assignments, pencils, books, or tools) Is often easily distracted by extraneous stimuli Is often forgetful in daily activities Hyperactivity Often fidgets with hands or feet or squirms in seat Often leaves seat in classroom or in other situations in which remaining seated is expected Often runs about or climbs excessively in situations in which it is inappropriate Often has difficulty playing or engaging in leisure activities quietly Is often “on the go” or often acts as if “driven by a motor” Often talks excessively Often blurts out answers before questions have been completed Often has difficult awaiting turn Often interrupts or intrudes on others (e.g., butts into conversations or games) 1993 1994 2000 2001 2002 2004 2006 2001 2001 2001 1996 1998 1997 1997 2003 2000 1996 2001 1997 2003 1998 1976 1986 2005 2003 2002 1998 2001 2004 1997 2001 2001 The purpose of the present paper is to investigate the construct validity of CBCL-AP, CPRS-R:S ADHD-I, and DSM-IV ADHD. Three questions are addressed. First, what are the phenotypic correlations between the three instruments? Second, do the instruments reflect a common underlying factor? Third, what are the genetic and environmental influences on the common and the instrument-specific factors? Methods Subjects 2002 2006 2007 Questionnaires were sent to all families that agreed to participate with the research of the Netherlands Twin Registry when the children were born (N = 7,828 families; birth cohorts 1989–1994) at the ages 7, 10, and 12 years. At least one measurement is available for 10,916 twins from 5,458 families, so the response rate is 70%. CBCL ratings were available in 10,018 twins at age 7, 6,565 twins at age 10, and 5,780 twins at age 12. CPRS-R:S ratings were available for 4,887 twins at age 12, and DSM-IV interviews were available for 1,006 twins. Complete data were available in 740 twins. The fact that the number of CPRS-R:S ratings is lower than the number of CBCL ratings, can be explained by the fact that the CPRS-R:S was not included for children born before 1991. The number of available questionnaires decreases over time as a result of the longitudinal character of the study (i.e., a number of children in the study had yet to reach the age of 12). 2000 Selection for the diagnostic interview 2006 Measures 1991 1996 2001 1998 1994 2001 1993 1998 Statistical analyses Transformation to categorical data 2004 1998 The CBCL AP score was calculated by summing the responses on the 11 items which resulted in a sum score with a possible maximum of 22. The four categories consisted of a score of 0, 1–2, 3–5, and 6 or higher, respectively. The CPRS-R:S ADHD-I score was calculated by summing the responses on the 12 items, which resulted in a sum score with a possible maximum of 36. The four categories consisted of a score of 0–1, 2–5, 6–11, and 12, or higher, respectively. The DISC sumscore with a range of 0 to 18 was transformed into an ordinal variable with four categories. The four categories were: (i) not affected (0 symptoms); (ii) mildly affected (1–2 symptoms); (iii) moderately affected (3–5 symptoms); and (iv) highly affected (more than 6 symptoms). The use of this four category variable provides greater resolution, and so better statistical power than the use of a dichotomous variable (ADHD absent versus ADHD present). Correcting for the selection 2002 Prevalences 2 Genetic modeling 2003 2001 2 2 2 2 2 1986 1992 Because the number of twins for whom interview data are available is relatively small, and sex differences in heritability are usually not found, the data from male and female twins were combined in the analyses. To allow for prevalence differences between boys and girls, sex was included as a covariate on the thresholds. The type-I error rate of all statistical tests was set at .05. Results Descriptives 3 Twin correlations 2 Table 2 Polychoric correlations in monozygotic (above diagonal) and dizygotic (below diagonal) twins   First-born Second-born CBCL 7  CBCL 10  CBCL 12  CPRS  DSM  CBCL 7  CBCL 10  CBCL 12  CPRS DSM First-born CBCL age 7 1 .66 .62 .51 .59 .76 .54 .49 .45 .45 CBCL age 10 .70 1 .69 .61 .59 .56 .77 .58 .53 .48 CBCL age 12 .63 .74 1 .71 .57 .48 .54 .75 .58 .53 CPRS-R:S  .56 .68 .77 1 .60 .46 .55 .62 .84 .51 DSM .51 .55 .59 .68 1 .34 .41 .46 .46 .64 Second-born CBCL age 7 .31 .22 .18 .15 .04 1 .66 .63 .52 .46 CBCL age 10 .22 .35 .22 .21 .01 .66 1 .71 .64 .59 CBCL age 12 .21 .28 .34 .24 .13 .60 .72 1 .75 .58 CPRS-R:S  .22 .27 .28 .38 .08 .49 .64 .74 1 .60 DSM .11 .16 .11 .07 .13 .45 .63 .67 .58 1 Note Genetic analyses 2 P 3 2 P 2 P 2 P 2 P 2 P 1 Table 3 Multivariate model fitting of maternal ratings on CBCL, CPRS-R:S and DSM-IV ratings on attention problems and ADHD in 7-year-old children  −2 log LL N par With model d.f. 2 P 1. Fully saturated 63020.52 210 – – – – 2. Thresholds MZ/DZ free, thresholds boys/girls equated  63123.54 126 1 84 103.02 .08 2a. Thresholds CBCL age 7 equated in MZ/DZ 63124.66 125 2 1 1.11 .29 2b. Thresholds CBCL age 10 equated in MZ/DZ 63123.98 125 2 1 .43 .51 2c. Thresholds CBCL age 12 equated in MZ/DZ 63123.68 125 2 1 .14 .71 2d. Thresholds Conners age 12 equated in MZ/DZ 63123.60 125 2 1 .06 .81 2e. Thresholds DSM age 12 equated in MZ/DZ 63126.34 125 3 1 2.80 .09 3. Thresholds boys/girls free, thresholds MZ/DZ equated  63108.18 126 1 84 87.66 .37 3a. Thresholds CBCL age 7 equated 63423.19 125 3 1 315.01 <.001 3b. Thresholds CBCL age 10 equated 63395.18 125 3 1 287.00 <.001 3c. Thresholds CBCL age 12 equated 63321.32 125 3 1 213.14 <.001 3d. Thresholds Conners age 12 equated 63388.24 125 3 1 280.06 <.001 3e. Thresholds DSM age 12 equated 63137.03 125 3 1 28.85 <.001 4. Cholesky decomposition ADE 63147.41 102 1 50 59.03 .18 4a. Independent pathway model D; Cholesky decomposition AE 63149.40 97 4 5 1.99 .85 4b. Independent pathway model A; Cholesky decomposition DE 63151.88 97 4 5 4.47 .48 4c. Independent pathway model E; Cholesky decomposition AD 63170.06 97 4 5 22.65 <.001 4d. Independent pathway AD; Cholesky decomposition E 63163.86 92 4 10 16.45 .09 4e. Independent pathway model ADE 63189.83 87 4 15 42.42 <.001 4f. Independent pathway AD; Cholesky decomposition E, instrument-specific A factors dropped 63255.66 87 4d 5 91.80 <.001 4g. Independent pathway AD; Cholesky decomposition E, instrument-specific D factors dropped 63164.92 87 4d 5 1.06 .96 5. Common pathway model 63406.53 79 4 23 259.12 <.001 Fig. 1 A graphical representation of the unstandardized additive genetic (A) and dominant genetic (D) effects on five measurements of Attention Problems and ADHD. In this figure, a graphical representation of the best-fitting model and the estimated factor loadings is provided for one individual twin. Additive genetic effects correlate 1 in MZ twins and .5 in DZ twins. Dominant genetic effects correlate 1 in MZ twins and .25 in DZ twins. To identify the model, the variances of the five categorical measurements are constrained at 1. CBCL7 = Child Behavior Checklist at age 7; CBCL10 = Child Behavior Checklist at age 10; CBCL12 = Child Behavior Checklist at age 12; CPRS-R:S = Conners Parental Rating Scale-Revised:Short version at age 12; DSM = DISC-IV ADHD at a mean age of 12 years  1 2 2 2 2 2 4 4 Table 4 Standardized genetic and environmental influences on the variances and covariances of five ratings of ADHD and attention problems  A D E CBCL 7 CBCL 10 CBCL 12 CPRS-R:S DSM CBCL 7 CBCL 10 CBCL 12 CPRS-R:S DSM CBCL 7 CBCL 10 CBCL 12 CPRS-R:S DSM CBCL age 7 .41     .36     .23     CBCL age 10 .39 .53     .45 .25    .16 .22    CBCL age 12 .53 .62 .68    .26 .19 .07   .21 .19 .25   CPRS-R:S .62 .69 .74 .79   .26 .17 .08 .05  .13 .14 .18 .16  DSM .51 .57 .68 .67 .56 .25 .17 .09 .07 .04 .24 .26 .23 .25 .40 Note 5 Table 5 Genetic and environmental correlations of five ratings of ADHD and attention problems  A D E CBCL 7 CBCL 10 CBCL 12 CPRS-R:S DSM CBCL 7 CBCL 10 CBCL 12 CPRS-R:S DSM CBCL 7 CBCL 10 CBCL 12 CPRS-R:S DSM CBCL age 7 1.0     1.0     1.0     CBCL age 10 .57 1.0    1.0 1.0    .50 1.0    CBCL age 12 .62 .74 1.0   1.0 1.0 1.0   .56 .60 1.0   CPRS-R:S .58 .69 .76 1.0  1.0 1.0 1.0 1.0  .34 .49 .68 1.0  DSM .52 .62 .68 .63 1.0 1.0 1.0 1.0 1.0 1.0 .39 .52 .45 .62 1.0 Note Discussion The aim of this study was to determine the extent to which three different instruments, which are commonly used to assess ADHD, attention problems, and hyperactivity, measure a common construct. The instruments considered are two scales based on items from questionnaires (CBCL-AP, and CPRS-R:S ADHD-I), and a DSM-IV ADHD interview. First, we considered the phenotypic correlations. Second, we tested if the variance in the different instruments reflects one common underlying factor. Third, we estimated the genetic and environmental influences on individual differences in ADHD. This is the first study that includes multivariate genetic analyses of behavior rating scales and DSM-IV interview data collected in a large sample of twins of approximately the same age. The CBCL scores collected at age 7 and 10 years were included only to correct for the selection. In the discussion, we focus mainly on the CBCL, CPRS-R:S and DSM interview data, which were collected at a mean age of 12 years. The phenotypic correlation between CBCL-AP and the CPRS-R:S ADHD-I was high (r = .75). The correlations between the CBCL and the DSM and between the CPRS-R:S and the DSM were slightly lower (r = .62). These lower correlations can both be the result of the different time-points at which the behavior checklists and the DSM interview data were collected (the mean time-span between measurement occasions was 10 months), the differences in the time frame for the assessment of the items (e.g., 1 month for the CPRS-R:S, 6 months for the CBCL, and 1 year for the DSM), and of instrument or method variance (e.g., interview versus behavior checklists). The genetic analyses show that the covariance between CBCL and CPRS is for 82% explained by genetic factors while the covariance between CBCL and DSM was for 75% explained by genetic factors. Therefore, the higher phenotypic correlation between CBCL and CPRS is not caused by a relatively higher genetic covariance. As noted, the AP scale of the CBCL questions relate to both inattention and hyperactivity/impulsivity. The fact that the correlation between the CPRS-R:S ADHD-I and DSM-IV ADHD is identical to the correlation between CBCL-AP and DSM-IV ADHD implies that the CPRS-R:S and the CBCL measure ADHD equally well. The description of the eleven item CBCL scale as an inattention scale seems to be too limited, because both the item content and the current results suggest that the CBCL also signals problems related to hyperactivity/impulsivity. 1997 1997 2003 2000 2001 The poor fit of the common factor model suggests that the construct validity of the instruments is not perfect. However, it is interesting to consider the implications of the overlap between the sets of genes that explained variance in the three instruments. High genetic correlations imply that the detection of the specific genes that play a role for ADHD, does not depend much on the instrument that is used. At age 12, the additive genetic correlations of the CBCL, CPRS-R:S, and DSM varied between .63 and .76, while the dominant genetic correlations could be constrained at 1. The non-shared environmental correlations are also quite high, and vary between .45 and .68. The dominant genetic correlations of 1 suggest that there is a subset of genes whose effect is not instrument or age dependent. In contrast, the correlations of the additive genetic effects are high but less than perfect. This suggests that the influence of most genes with an additive effect are not sensitive to the particular instrument that is used, although there are some genes that explain variance only in a particular measurement (e.g., CBCL), but not in another (e.g., DSM). P 2003 2003 2004 2006 2005 2003 2003 2004 2006 2005 Limitations The results of this study should be interpreted bearing in mind the following limitations. First, further study is required to investigate if the results of the current study, which was based on a Dutch population sample, generalize to population samples outside the Netherlands. Second, clinical diagnoses were based on structured diagnostic interviews with the mother. The results may be different when the assessment of ADHD is based on expert clinical diagnoses. Third, no distinction was made between problems related to inattention and problems related to hyperactivity. Since the CBCL does not distinguish between inattention and hyperactivity (and probably the number of items is too small to reliably measure these two factors) we did not distinguish between the subscales. Fourth, we did not allow for sex differences in the genetic and environmental influences based on the results of univariate studies. Because of the increased statistical power in the multivariate model, it is possible that sex differences do exist. However, due to the categorical nature of the data, and the fact that some of the cells in the contingency tables contain few individuals in a two-group analysis, statistical problems will arise in a four-group analysis. Fifth, as a result of the categorical nature of the data, computational limitations prohibited inclusion of confidence intervals. Clinical implications Two general approaches towards the measurement of ADHD can be distinguished. In the DSM-IV framework, ADHD is viewed as a categorical trait. Using behavior checklists, children can show variation in a continuum from not affected at all to severely affected. The current study shows that variance in DSM-IV symptoms, the CBCL-AP scale, and the CPRS-R:S ADHD-I is explained mostly by genetic effects. The correlations between the genetic influences on variance in these three measurements of ADHD are high. This implies that different measurements tap the same genetic liability.