Models of preoperative risk stratification in cardiac surgery: the prediction of mortality or of survival?

BACKGROUND: The need to assess the quality of heart surgery outcomes stimulated the development of pre-surgical risk stratification models in order to predict outcome on the basis of patient characteristics. The aim of the study was to compare the predictive accuracy of hospital mortality according to the following three models: Parsonnet (NBI Score), Higgins (CCF Score) and Roques (French Score), in a setting totally independent from the one in which the models were derived. METHODS: For each of the 516 patients undergoing heart surgery at our institution between January 1992 and December 1993, we calculated the pre-surgical risk according to the three models. Then we compared the predicted mortality against the observed mortality by means of the Shannon accuracy index, the ROC curve analysis and the overestimation histogram. RESULTS: Overall observed mortality (1.5%) was similar to the predicted mortality by the NBI Score (1.5 +/- 2.1%, p = ns), the CCF Score (1.7 +/- 2.0%, p = ns) and the French Score (1.9 +/- 2.5%, p = ns). The predictive accuracy of global mortality is very high and equal with the three models, and it is very low in the 8 patients who died (NBI Score = 0.06 +/- 0.06; CCF Score = 0.125 +/- 0.067; French Score = 0.102 +/- 0.07, p = ns). The area under the ROC curve is identical in the 3 models. CONCLUSIONS: The predicted mortality obtained by the three models is not significantly different from the observed mortality and therefore, the global accuracy is similar and very high, while it is very low for patients who will die. The models for pre-surgical risk stratification are useful for comparing the results among different institutions or different surgeons, or for monitoring the results over time in the same institution, but they cannot be used to accurately predict the individual risk of hospital mortality.