The aim of this study was to validate a case-mix adjustment tool (neural network) for the audit of postoperative outcomes. We tested its calibration and discrimination on two unseen groups of patients being treated for squamous cell carcinoma (SCC) of the head and neck and compared observed complication rates with predicted rates. A total of 196 patients who were treated at two UK NHS institutions between 2016 and 2018 were audited. Preoperative data pertaining to risk (T classification, complexity of operation, and "high-risk" status) were collected, together with data on postoperative complications. Diagnostic test statistics and receiver operating curves (ROC) were used to test the performance of the tool. The score was well calibrated (predicted and observed complication rates both 43%), but discrimination suggested only fair accuracy (ROC 0.66 - 0.68). Adjustment of case mix for the audit of postoperative complications is difficult, although our model suggests that departmental audit is possible, and its accuracy is equivalent to that of other national audits. Further work may elucidate key variables that have not yet been assessed.
