In a forensic psychiatric department Mudde et al. investigated if scores on the Historical Clinical Risk Management-20 (hcr-20) predicted aggressive behaviour. Risk assessment may be used to protect the hospital staff. Of 102 patients about 42% (43 patients) had caused one or more aggressive incidents as recorded by the Staff Observation Aggression Scale-Revised (soas-r).
On average HCR-scores differed 4 points (scale range 0-40) between aggressive versus non-aggressive patients. The authors calculated the area under the curve (auc)-value and in addition performed a “logistic aggression analysis” (something added in translation here). However, this is the world upside-down. The auc-value is a relative prediction assessment tool used as a general measure for the goodness-of-fit of a logistic regression model. It is not a useful measure of how well the model classifies individuals. And the model in this "aggression analysis" didn't perfom well.
HRC-20 scores resulted in an auc-value of .70 and 64% of patients were correctly classified. But 64% is not much more than the 58% of patients who did not engage in violent behaviour. In a fishing expedition the authors found two items that resulted in 73% of patients correctly classified. They suggest that these HCR-subscale items may be able to predict aggressive behaviour fairly accurately. However, auc-value and classification percentage are overestimated because the model and model-fit assessment are based on the same dataset (Read more here >>). The authors acknowledge that the model is not useful for clinical practice. Although model-fit and prediction power are low, they conclude that the HCR-20 can predict “to a certain extent” which patients will engage in violent behaviour. The authors argue that their results correspond to those of similar earlier investigations. But many wrongs don't make a right.
N. Mudde, H. Nijman, W. van der Hulst, J. van den Bout (2011). Predicting aggression during the treatment of forensic psychiatric patients by means of the hcr-20, Tijdschrift voor Psychiatrie, 53, 11:705-713