Introduction 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 13 14 20 21 22 4 23 24 25 26 9 27 If recalibration is not sufficient to improve the performance of the prognostic model, the only alternative is to develop a new model that takes into account the results of studies done since the original model was developed. This means incorporating missing variables that have been shown to affect outcome, minimizing problems with the application of the model, and reducing the possibility of other confounders. The objective of the SAPS 3 project was to cope with the above-stated problems by developing a new model for improved risk adjustment in critically ill patients. Another important goal was to make the new model available free of charge for use in the scientific community. In the SAPS 3 study (which took place at the end of 2002), data about risk factors and outcomes in an international multicentric cohort of critically ill patients were prospectively collected so that a high-quality database would be available for further analysis of the associations between risks and outcomes in our patients. Materials and methods Project Organization http://www.esicm.org During the data collection phase, a coordination and communications centre (CCC) was installed. The CCC was responsible for the management and control of the project. This included the administration of all project tasks and implementation of actions and activities as necessary; communication between project partners (e.g., centres, researchers and institutions) through sampling and distribution of necessary information; and pooling and administration of the data provided by project participants. In addition, the CCC provided almost around-the-clock service to answer urgent questions and resolve problems during the phase of data collection. In each country, a country coordinator was responsible for operational management and direct communication with the participating ICUs in that country, including giving specific help when necessary. The country coordinator was responsible for ensuring completion of the various tasks required of the participating ICUs. The list of country coordinators can be found in Appendix E of the ESM. At the ICU level, an ICU coordinator was responsible for local activities, such as obtaining approval from the local ethics or data-protection committees where applicable. In addition, the ICU coordinator was responsible for supervising the daily data collection, problem management, controlling the completeness of the data, data quality control, training medical and nonmedical staff for data collection, management of the data, and transmission of the data to the CCC or country coordinator. The list of ICU coordinators can be found in Appendix F of the ESM. Data collection  i soft 2 n In addition, each ICU received a questionnaire with detailed questions about ICU structures and about resources available in other areas of the hospital. Data were collected at ICU admission, on days 1, 2 and 3, and on the last day of the ICU stay. Data from the day of admission (aside from sociodemographic data such as age and sex) were categorized into different levels: (i) data about the condition of the patient before ICU admission, such as chronic conditions and medical diseases; (ii) data about the patient’s condition at ICU admission, such as the reason for admission, infection at admission, and surgical status; and (iii) data about the patient’s physiologic derangement at ICU admission. These data were collected within an hour before or after ICU admission. 5 28 cute medical disease  Surgical status  reason for admission http://www.saps3.org Database Data were collected and pooled by the CCC. The final database file was then imported into the SAS system, Version 8e (SAS Institute Inc., Cary, USA). Data cleaning was accomplished through the application of a variety of additional plausibility controls and cross-checking of information between redundant data fields. n n n n  Basic SAPS 3 Cohort n n  SAPS 3 Hospital Outcome Cohort Because the study was an observational study and no additional interventions were performed, the need for informed consent was waived by the institutional review board. Each ICU, however, was made responsible for obtaining local permissions as necessary. Data quality Recorded data were evaluated for completeness of the documentation and reliability. Interrater quality control was performed through rescoring of the data from day 0 (the day of ICU admission) for three randomly selected patients in each ICU. From the rescored data, kappa coefficients and intra-class correlation coefficients were calculated, as appropriate. Availability of the variables necessary to calculate the SAPS II was used as an indicator for the completeness of the data. Statistical analysis  P 29 30 31 Results Data quality Four hundred eighty-three rescored patients could be identified and linked to their original counterparts (2.5% of admitted patients). Data quality was found to be excellent, with the majority of coefficients being >0.85. Only two of the more than 50 tested variables had coefficients <0.80 (body weight, 0.79; positive end-expiratory pressure, 0.72), and only one was <0.70 (leukocytes [maximum], 0.57). For a detailed list of coefficients see Table E1 in the ESM. Data completeness was also found to be satisfactory, with 1 [0–3] SAPS II parameter missing per patient. Description of ICUs The Basic SAPS 3 cohort includes 307 ICUs from 35 countries. On average each ICU contributed 50 (27–78) patients to the cohort. To assess heterogeneity of results between different geographic regions, seven regions were defined: Australasia, Central and South America, Central and Western Europe, Eastern Europe, North America, Northern Europe, and Southern Europe and Mediterranean countries. The allocation of countries to these regions can be seen from Table E10 of the ESM. n Description of patients 1 1 Table 1 Basic cohort:  HO cohort:  n:  Basic cohort HO cohort n % n % Number of patients 19,577  16,784 100.0 Gender       Female 7,678 39.2 6,610 39.4   Male 11,881 60.7 10,161 60.5    Missing 18 0.1 13 0.1 Age, years (median, quartiles) 63 49-74 64 49-74 Origin       Home 2,810 14.4 2,343 14.0   Same hospital 13,926 71.1 12,063 71.9   Chronic care facility 74 0.4 64 0.4   Public place 519 2.7 432 2.6   Other hospital 2,125 10.9 1,791 10.7   Other 80 0.4 59 0.4   Missing 43 0.2 32 0.2 Intra-hospital location before ICU admission       Emergency room 5,419 27.7 4,630 27.6   Intermediate care unit/High dependency unit 562 2.9 475 2.8   Operating room 7,537 38.5 6,449 38.4   Other 552 2.8 413 2.5   Other ICU 698 3.6 611 3.6   Recovery room 482 2.5 400 2.4   Ward 3,411 17.4 3,036 18.1   Missing 916 4.7 770 4.6 ICU admission status       Planned admission 6,750 34.5 5,598 33.4   Unplanned admission 12,338 63.0 10,801 64.4   Missing 489 2.5 385 2.3 Acute Infection at ICU admission       No infection 15,254 77.9 12,968 77.3   Clinically improbable/colonization 342 1.7 298 1.8   Clinically probable/documented 2,761 14.1 2,422 14.4   Microbiologically documented 1206 6.2 1,083 6.5   Missing 13 0.1 13 0.1 Surgical status       No surgical procedure 8,437 43.1 7,305 43.5   Scheduled surgery 6,800 34.7 5,700 34.0   Emergency surgery 3,321 17.0 2,930 17.5   Missing 1,019 5.2 849 5.1 Fig. 1 n  Columns:  squares:  Cardiovascular, respiratory and neurologic diseases were the most frequent organ-specific reasons for admission (Table E4, ESM). The acute medical diseases necessitating ICU admission included a broad spectrum of diagnoses (Table E5, ESM). Approximately one half of the patients underwent surgery before ICU admission, with abdominal, cardiac and vascular surgery being the most frequent procedures (Table E6, ESM). 2 3 Table 2 Basic cohort:  HO cohort:  n: 
ICU LOS:  IMCU/HDU:  Q1, Q3:  Basic cohort HO cohort n % n % Number of patients 19,577 100.0 16,784 100.0 ICU LOS, days (median, quartiles) 2 1–6 2 1–6 ICU discharge—destination       Home 438 2.2 361 2.2   Same hospital 14,946 76.3 12,477 74.3   Other hospital 1,029 5.3 852 5.1   Missing 3,164 16.2 3,094 18.4 Intrahospital discharge       Emergency room 58 0.3 50 0.3   IMCU/HDU 2,222 11.4 1,873 11.2   Other 303 1.5 257 1.5   Other ICU 583 3.0 479 2.9   Recovery room 306 1.6 218 1.3   Ward 12,250 62.6 10,291 61.3   Missing 3,855 19.7 3,616 21.5 ICU discharge—status       Planned discharge 14,872 76.0 12,262 73.1   Unplanned discharge 1,595 8.1 1,467 8.7   Missing 3,110 15.9 3,055 18.2 Risk adjustment       SAPS II score (median, Q1–Q3) 30 20–42 31 21–43   SOFA score (median, Q1–Q3) 9 6–11 9 6–11 Outcome       ICU mortality (%)  15.2  17.7 Table 3  n  Australasia Central & South America Eastern Europe Central and Western Europe Northern Europe Southern Europe and Mediterranean countries North America Number of patients 2,235   2,540   1,084   4,712   355   7,854   797   Females, % 38.0   44.6   42.2   40.1   42.5   36.9   38.0   Age, years (median, quartiles) 59 45–71 31 46–74 62 48–72 65 52–75 66 51–76 64 59–74 64 49–75 SAPS II score (median, Q1-Q3) 28 19–40 30 20–42 27 18–43 29 21–40 35 25–46 32 22–44 29 20–39 SOFA score (median, Q1-Q3) 8 6–10 8 6–11 9 7–11 8 5–11 9 7–11 9 7–11 9 6–11 ICU mortality, % 12.7   17.4   16.9   10.8   20.6   18.1   8.5   Performance of the SAPS II 5 n  p Discussion To the best of our knowledge, the SAPS 3 study is the largest prospective epidemiologic multicentre, multinational study conducted in health services and outcomes research in intensive care medicine to date. 32 33 3 32 34 3 35 19 36 11 12 16 37 38 We did, however, experience problems with the cohort of rescored patients: First, we had to exclude all rescored patients for whom the original counterpart was also excluded due to the application of any of the exclusion criteria. Second, in some cases the original patient identification was either missing or documented in such a way that a unique allocation was not possible. Both of these exclusions reduced the number of rescored patients available for analysis. Two strategies to build up a cohort are available: first, to recruit only patients who meet well-documented inclusion criteria (such as documented vital status at hospital discharge) or, second, to document all patients and then exclude patients based on a predefined set of exclusion criteria. For the SAPS 3 study we chose the second option—to form two different cohorts—because we needed to provide a basic cohort for all further analyses of the SAPS 3 database. Since some studies will focus on different outcomes (e.g., ICU outcome rather than hospital outcome), we decided to use missing ICU outcome (and not hospital outcome) as an exclusion criterion for the basic cohort. 1 2 We conclude that the SAPS 3 database is within the above discussed limits of high quality and reflects the heterogeneity of current intensive care provision. As such, it provides an excellent basis for the development of a new risk adjustment system. Electronic Supplementary Material (PDF 794 KB)