Introduction 1 2 3 + + 4 9 10 11 12 13 14 15 25 26 27 The main aim of the “CIMT monitoring panel” is to harmonize and optimize the monitoring of antigen-specific T-cells among the participating laboratories, based on objective rationales with respect to the testing procedure, the analysis and the interpretation of results. Important requirements for an immunological test are sensitivity, applicability to large amounts of clinical material and feasibility at reasonable cost. The results generated by the tests should be reproducible and sensitive, independently of the place where they have been performed. After the first meeting of the working group, a series of inter-laboratory testing projects was initiated, in which individual laboratories could compare their performance, express their needs and exchange experience in order to improve their local assays. Here we report the results of the first two phases of the CIMT monitoring panel, with 13 participating centers from six European countries. Materials and methods Preparation and screening of PBMC samples l l 6 38 Synthetic peptides and HLA-tetramers http://www.syfpeithi.de 12 + Participating centers Twelve centers from five European countries participated in the first phase of the monitoring panel. As one of the investigators moved to another institution during the study a 13th center from a 6th European country was added to the group in the second phase of the panel. Participation in the panel was open to all interested laboratories with a focus on T-cell monitoring, independently of membership in the Association for Immunotherapy of Cancer. Reagent distribution and assay guidelines Coded PBMC samples, synthetic peptides and HLA-A*0201 tetramers were shipped on dry ice to the participants. Additionally, guidelines for the two T-cell assays were distributed for each phase: Phase I/2005 6 For the functional assays, synthetic peptides were diluted at 1 mg/ml in PBS as a stock solution. Concentrations in further tests were 1–10 μg/ml, left to the choice of the participants. There were no recommendation which functional test should be performed, so that each group could choose the test either routinely used, or to be implemented for its own needs. In this first phase, 11/12 laboratories chose the IFNγ ELISPOT assay, one lab (Z10) a FACS-based intracellular IFNγ staining and one lab performed both assays (Z7). Spot counting was performed locally. Phase II/2006 Results Collection and analysis of results + + + +, − + Results t The raw data from both panel phases will be provided to interested readers upon request. Results Phase I/2005 of the interlaboratory testing project—general aspects Coded PBMC samples from four HLA-A*0201-positive and one HLA-A*0201-negative healthy donor (D1–D5) were included in this first testing phase. The thawing procedure for PBMC samples in the test centers was not standardized and the recovery of viable cells varied greatly between 45 and 102% (mean 73%) in the 12 labs. However, the number of cells recovered was in all cases sufficient to perform the required analyses. When all the data from the tetramer staining and functional tests were combined it became clear that subjects D1 and D5 had responded to the HLA-A*0201 restricted CMV-derived peptide, consistent with their CMV seropositive-status, and that subjects D1, D2, D3, and D5 had responded to influenza. In total, each laboratory should in theory have been able to measure six positive (2× CMV and 4× influenza) responses. Detection of antigen-specific T-cells by tetramer staining and IFNγ ELISPOT + + 1 + + 1 1 + + + + + +  Fig. 1 + + + upper panel y x +  Table 1 Overview of the tetramer results from phase I/2005 of the CIMT monitoring panel  D1 CMV D1 Flu D2 Flu D3 Flu D5 CMV D5 Flu  Min 448 20,000 6,667 1,818 106 1,786 (I) Mean 141 8,095 1,909 1,014 80 1,106 Max 35 3,774 526 595 26 588 Z1 129 3,774 635 870 106 758 (II) Z2 112 – – 1,818 70 1,770 Z3 n.d – – 701 n.d – Z4 97 – – 854 106 1,342 Z5 161 – – 909 88 606 Z6 139 6,896 1,316 1,000 91 651 Z7 448 20,000 6,667 1,266 82 1,724 Z8 100 5,128 545 595 87 877 Z9 n.d – 1,316 1,274 n.d 1,786 Z10 94 – 2,597 1,360 86 1,439 Z11 99 4,675 1,667 877 58 588 Z12 35 – 526 645 26 625 Detected by 10/10 5/12 8/12 12/12 10/10 11/12 (III) Detected % 100 42 67 100 100 92 (I) Minimum, mean and maximum frequencies of antigen-specific T-cells x + (III) Number and percentage of centers which detected a given reactivity in donors D1, D2, D3 and D5 2 Materials and methods Table 2 Overview of the IFNγ ELISPOT results from phase I/2005 of the CIMT monitoring panel  D1 CMV D1 Flu D2 Flu D3 Flu D5 CMV D5 Flu  Min 3,061 62,500 55,555 33,333 11,428 50,000 (I) Mean 1,855 38,141 43,589 17,547 4,405 30,811 Max 888 10,256 30,769 8,571 1,039 14,705 Z1 1,006 – – 8,571 – – (II) Z2 2,439 – – – 2,816 – Z3 1,295 62,500 – 22,727 1,412 27,727 Z4 1,312 – – 10,909 1,980 – Z5 – – – – – – Z6 233* – – 1,960* 253* – Z7 1,895 10,256 44,444 33,333 11,428 50,000 Z8 3,061 – – – 6,896 – Z9 888 41,666 55,555 12,195 1,039 14,705 Z10 ND ND ND ND ND ND Z11 1,769 – 30,769 – – – Z12 3,030 – – – 5,263 – Detected by 10/11 3/11 3/11 6/11 8/11 3/11 (III) Detected % 91 27 27 55 73 27 (I) Minimum, mean and maximum frequencies of antigen-specific cells + Results from Z6 were not included for calculation of the mean frequency of antigen-specific T-cells in D1, D3 and D5 (III) Number and percentage of centers that detected a given response in donors D1, D2, D3 and D5 ND + Materials and methods + 4 4 + + 1 + + 2 + + + Fig. 2 a  Bars bars open bars grey bars 4 + black bars 4 + boxes bar asterisk P b  bars open bar Grey bars black bars boxes asterisks P  + 6 ELISPOT assays are heterogeneous and require standardization The ELISPOT analyses were performed according to 11 more or less different protocols. The most discernible differences that were observed in these protocols concerned (1) the different types of multi-screen plates, (2) the serum origin, (3) the use of duplicates, triplicates or quadruplicates, (4) the use of allogeneic APC, (5) the inclusion of a resting phase after thawing the PBMC, (6) the number of PBMC per well, (7) the type of antibodies used, (8) the type of spot-reader, and the (9) enzyme and substrate for staining of the spots. Each center also used a different plate protocol (distribution of the wells, number of replicates, control tests). The influence of each of these parameters on the number of positive responses was studied by further analysis in which the laboratories were divided into two subgroups. As a result, several criteria were identified which could help to improve the sensitivity and comparability of detection. t Materials and methods t 2 2 2 5 5 5 2 + 5 Phase II/2006 of the interlaboratory testing project—general aspects + + + + + + n + n + + 3 + + + 3 +  +  +  + + + Table 3 Overview of tetramer results from phase II/2006 of the CIMT monitoring panel  D1 Flu D2 CMV D2 Flu D4 Flu D5 CMV D6 Flu D7 Flu D8 CMV  Min 7,143 77 10,000 10,000 60 3,333 869 42 (I) Medium 3,739 45 3,573 5,278 37 1,116 347 19 Max 1,538 28 1,250 2,500 30 571 202 8 Z1 3,333  45 3,333 5,000 39 769 294 10 (II) Z2 4,000 30 3,333 5,000 30 1,100 250 21 Z3 4,000 47 – 5,000 45 588 270 20 Z4 1,538 71 2,857 6,666 33 769 263 42 Z5 6,666 54 2,500 – 43 1,428 377 24 Z6 5,000 77 – 10,000 60 1,666 869 27 Z7 2,857 47 3,333 6,666 38 833 290 22 Z8 2,000 35 3,333 3,333 31 666 244 8 Z9 3,333 31 10,000 3,333 32 3,333 625 20 Z10 3,333 28 2,222 – 30 952 202 13 Z11 1,666 36 1,250 2,500 32 571 215 15 Z12 7,413 37 – – 35 714 260 8 Z13 – 53 – – 34 – – 20 Detected by 12/13 13/13 9/13 9/13 13/13 12/13 12/13 13/13 (III) Detected % 92 100 69 69 100 92 92 100 (I) Minimum, mean and maximum frequencies of antigen-specific T-cells x + (III) Number and percentage of centers that detected each of the eight possible responses + 4 4 Table 4 Overview of IFNγ ELISPOT results from phase II/2006 of the CIMT monitoring panel  D1 Flu D2 CMV D2 Flu D4 Flu D5 CMV D6 Flu D7 Flu D8 CMV  Min 44,118 1,791 33,803 58,824 1,745 48,387 14,720 1,698 (I) Medium 28,823 1,088 16,395 49,960 999 14,265 4,669 1,023 Max 10,345 396 1,231 41,096 391 4,458 1,706 269 Z1 44,118 596 – – 596 48,387 14,720 318 (II) Z2 – 1,732 – – 1,745 4,739 2,222 1,698 Z3 – 1,333 – – 1,117 – 2,982 1,292 Z4 – – – – 774 – 2,273 447 Z5 – 997 – – 1,157 17,647 3,827 1,209 Z6 10,345 1,031 14,151 – 1,006 – 1,706 1,005 Z7 – 1,317 – – 1,061 10,000 6,000 1,661 Z8 32,258 396 – 58,824 391 4,458 3,623 269 Z9 28,571 966 – 41,096 912 14,423 8,955 1,081 Z10 – 847 1,231 – 806 7,426 3,052 696 Z11 – 1,044 33,803 – 1,087 20,339 3,670 1,273 Z12 – 1,791 – – 1,387 10,619 3,057 1,583 Z13 – 1,008 – – 949 4,615 4,615 770 Detected by 4/13 12/13 3/13 2/13 13/13 10/13 13/13 13/13 (III) Detected % 31 92 23 15 100 77 100 100 (I) Minimum, mean and maximum frequencies of antigen-specific cells x (III) Number and percentage of centers that detected each of the eight possible responses Comparison of the results obtained in both phases 3 + + Fig. 3 a b filled circle filled line open circle open line y x + x  y y 4 Fig. 4 a  b  filled squares open squares x x + a x b X y y broken lines  + + 4 4 + + 5 + + + +  5 Fig. 5 a  bars + bars + open bars grey bars 4 + filled bars 4 + b  bars open bars grey bars filled bars  5 5 Experience does not equal performance Among the 13 centers that had participated in phase II, tetramer stainings had been performed for 1–8 years. Similarly, the experience in the ELISPOT technology varied between 1 and 10 years. For both techniques, we could not find any correlation between the years of experience and the ability to detect T-cell responses, not even among the subgroups of moderate or low T-cell responses (not shown). Discussion 28 31 32 33 36 3 4 37 y + + 39 40 39 40 In the ELISPOT assay, the background spot numbers obtained by the different participants varied greatly, but we were unable to correlate this finding to a distinct variable. Since the spontaneous cytokine secretion impacts significantly on the sensitivity of this assay, factors that especially influence the non-specific spot production, possibly the medium type or serum source, will need to be systematically analyzed in a separate study. + In addition to the systematic identification of variables that correlate with sensitivity/insensitivity of various assays, inter-laboratory testing projects also allow the rapid evaluation of individual performance among a group. Interestingly, the finding that experienced laboratories did not perform better than laboratories which recently applied these techniques strongly suggests that non-optimal protocols, once established in a lab, can commonly be maintained for several years. Periodic comparison of local protocols with those of other centers is recommended. Even if a new staff member uses an established protocol, it is recommended to have them participate in inter-laboratory testing/teaching exercise. Regular participation in multi-center comparisons could thereby help to optimize and validate participants’ performance over time and to maintain sensitive protocols or minimal standards. This is of great importance when material from expensive clinical trials has to be analyzed. All data from the CMV-serology, from the pre-testing experiments and from the results generated by the participating laboratories in ELISPOT and tetramer staining were taken together for each donor in order to qualitatively validate the presence of CMV- and influenza-specific T-cells. To estimate the quantity, i.e. the frequency of specific T-cells in each donor, we calculated the average of all qualitatively positive results, as well as the standard deviations. This procedure constitutes only an approximation of the real number of antigen-specific cells present in a given sample, and cannot be taken as a method for determining absolute T-cell frequencies. Cell samples that contain pre-defined numbers of antigen-specific T-cells (e.g. spiked T-cell clones), especially tumor-reactive T-cells, are not easily available for use in multi-center comparisons, although such standard samples are urgently needed. We see this as one major bottle-neck for the optimization and standardization of immunomonitoring techniques. Methods to generate such standard samples for broader use will therefore be elucidated with high priority in the near future for the next phases of this international collaboration. Another big challenge will be to define accepted rules for the settings of the equipment used in these analyses (flow cytometer or ELISPOT reader) in order to uniformly process and analyze the raw data. Ten from eleven laboratories that performed the ELISPOT assay in the first phase used an ELISPOT reader for spot counting. It is known that spot counts between centers can differ significantly and this may be explained by the use of different reading machines, different settings for the same type of machine or by the experience of the operator. Within this group, four different commercially available reading systems were used (supplementary Table S2). We were not able to identify differences between the types of ELISPOT readers. A new ELISPOT panel phase is currently in preparation, that will specifically focus on the performances of different ELISPOT readers and try to introduce tools to control inter center variation. In addition, none of the participant reported on the use of live/dead cell discrimination on thawed PBMC samples for the FACS-based experiments. Whether the combination of staining with Ab/HLA-tetramers and vital dyes or with a resting phase is beneficial for increasing the sensitivity of the tetramer staining assay could be addressed in future testing actions. 41 Last but not least, we would like to stress that even the best guidelines and protocols alone cannot guarantee good performance. Monitoring of antigen-specific T-cell responses requires skills as well as experience. Participation in immunomonitoring panels cannot compensate for the need to constantly educate and train staff and to develop specific expertise for covering individual needs. Nevertheless, we strongly believe that by organizing further two-step inter-laboratory testing projects, the CIMT monitoring panel will be able to improve the sensitivity of the assays used for immunomonitoring as well as to actively participate in the harmonization of these assays, which is required to enable the comparison of immunotherapeutic trials performed in different centers. Electronic supplementary material Below is the link to the electronic supplementary material.
 ESM (PPT 82 kB)