Schoevaerts and collegeas (2013) estimate the incidence and evolution of compulsory admissions in Belgium and the Netherlands by pooling and analysing available register data. The epidemiological exploration shows an increase in the number of involuntary psychiatric admissions. That increase is without dispute. However, some question are raised by the presentation of the statistics and the interpretation of the results.
The authors point at an increase by 47% and 25% in Belgium and the Netherlands respectively, but these proportions concern different time periods. Over the comparable period of 2002 to 2008 the increase is about 23 to 24% in both countries. Schoevaerts et al. intended ‘to look into the pitfalls in analysing and comparing these data’, but overlooked this one. And more.
Not a real test
For the Doubting Thomas not completely convinced by these growth percentages, Chi-square tests for linearity were conducted and regression lines were calculated and plotted together with 5 digits R-squared statistics (proportion of explained variance). However, these statistics assume normal distributions and uncorrelated errors, while counts and proportions are not distributed normally and some autocorrelation will occur as it is not uncommon for patients to be involuntarily admitted more than once. Apart from that, there is no real test of the null-hypothesis that states that the increase or decrease is exactly 0. That is shown by the Chi-square for linear trend formula, (N-1)*r. The authors looked at ‘raw differences in incidence’, but then for 40 to 50 compulsory admissions per 100.000 inhabitants yearly for a period of 13 years a significant outcome will be attained for a correlation coefficient as low as r = 0,007. In other words: for these numbers any increase or decrease will be statistically significant.
Potential effects missed
Next, Schoevaerts et al. point at several factors that contribute to the increase of the number of compulsory admissions, but relatively little attention is given to factors to prevent the use of constrains. The authors link the increase to a social and professional culture of risk avoidance that exchanges freedom for safety. However, this is a negative perspective on compulsory admission. In contrast, focus has shifted to a treatment perspective which at local level is reflected in changes in characteristics of patient involuntarily admitted (more severely ill patients and limited social support), the shift in the dangerousness criterion from ‘danger to others’ to ‘self neglect’, longer hospitalisation and improved continuity of care. Schoevaerts et al. refer to limited effects of better integrated healthcare, but missed the potential effect on the number of compulsory admissions of primary care cooperation aimed at early detection of problematic situations and continuity of care (Wierdsma 2008).
More than registration
Final conclusion by Schoevaerts et al. is that standardisation of the registration of compulsory admissions is needed to improve comparability of hospitalisation data. No one can object to that. However, priority should lie elsewhere. Tilburg (2012) previously emphasized that compulsory measures should be viewed in the context of a comprehensive policy aimed at preventing dangerousness. This implies that we need to develop regional information systems on the use of mental health care together with neighbouring areas. In a network of such information systems new prevention initiatives can be tested on a large scale. The focus should not be on improved registration, but on the development and evaluation of interventions that could prevent compulsory admissions.
Schoevaerts K, Bruffaerts R, Mulder CL, Vandenberghe J. An increase of compulsory admissions in Belgium and the Netherlands: an epidemiological exploration. [Stijging van het aantal gedwongen opnames in België en Nederland: een epidemiologische analyse]. Tijdschr Psychiatr 2013; 55: 45-55.
Tilburg W van. Onderzoek naar separatie: een goed begin, maar nog wel half werk. Tijdschr Psychiatr 2012; 54: 222-4.
Wierdsma AI. Follow-up after involuntary mental healthcare: Who cares? Emergency compulsory admission and continuity of care in Rotterdam, the Netherlands. Rotterdam: Erasmus Universiteit; 2008.
Reply to Wierdsma
We thank our colleague Wierdsma for his comments and will gladly respond to his critique.
First concerning the differences in the incremental proportions of the number of compulsory admissions in Belgium and the Netherlands. These proportions cannot be compared in strict sense because the periods are different. Therefore these incremental proportions were not compared In our paper: nowhere do we state that the increase in compulsory admissions was larger in Belgium than in the Netherlands. The periods were thus chosen for the availability of data comparable within a country over the years. The incremental proportion of the number of compulsory admissions in Belgium was 42% between 1999 en 2008 (not 47% as Wierdsma states), in the Netherlands it was 25% between 2002 and 2009. A look at the incidence numbers for 2002 and 2008 shows an equivalent increase: 21% in Belgium and 20% in the Netherlands.
In the paper we focused on comparing the incidence rates in the two countries. It appeared that the incidence in the Netherlands was clearly higher than in Belgium. In 2008 77 per 100.000 inhabitants were admitted involuntarily per year in the Netherlands, compared to 47 in Belgium. The fact that incremental proportions can only effectively be compared when increments cover a similar period goes without saying (Schoevaerts e.a. 2012).
Secondly the statistics employed. Indeed we used a relatively ‘simple’ statistical method, such as Chi-square linked to the assumption that the data are normally distributed. However, we deemed the use of complex statistical models not necessary in view of our research questions. Indeed the aim of our study was to explore the incidence of compulsory admissions in the lower countries. This is part of descriptive epidemiology. In contrast with analytic epidemiology the descriptive variety generates hypotheses instead of testing them. Hypothesis testing is not addressed in descriptive epidemiological research (Rothman et al. 2008). Therefore the significance tests could be interpreted as illustrative and inviting for all readers rather than solely aimed at conclusions and convictions of doubting Thomasses.
Apart from this a possibly thorny question remains open: in our approach we indeed made some implicit assumptions. Wierdsma’s argument that we assumed that data are normally distributed, is important. However, his argument holds especially if the number of cases is relatively small. However, in large studies (data not normally distributed) some parameters, such as confidence intervals, are aligned, so that they can be estimated adequately using a normal distribution (Flanders & Kleinbaum 1995). Likewise argument can be found to employ linear regression techniques with nonnormally distributed data. Using population data with large numbers of cases and non-cases one bases the statistical methods on the central limit theorem. This assumes that linear techniques are used with variables that do not satisfy the condition of a normal distribution (Lumley et al. 2002).
Above arguments do not imply that no multivariate and/or non-linear statistical methods are needed to answer our research questions. These arguments indicate, however, that it is hard to state a priori that this or that technique using the normal distribution is ‘de facto’ wrong.
Thirdly the factors that can contribute to a decrease in the use of constrains. Although this topic was beyond the scope of our paper, one of us indirectly addressed it previously in Tijdschrift voor Psychiatrie (Verbrugghe et al. 2008). In this context Wierdsma raises some interesting elements. Open access and low threshold psychiatric services, early detection and continuity of care, with attention to self-determination and autonomy, naturally need to be explored as possible routes to reduce the number of compulsory admissions.
However, a likely pitfall would be that because of more assertive (outreaching) care some severely ill psychiatric patients will be more readily admitted involuntarily because of a risk avoiding culture. Thus longitudinal research is paramount to map the long term effects of changes in psychiatric care on the pattern of compulsory admission (Schoevaerts et al. in press).
Wierdsma also mentions a shift in the dangerousness criterion from ‘danger to other’ to ‘self neglect’. However, this is true only for the Netherlands, as we described in the discussion section. Finally, in his last remark Wierdsma pleads the development and evaluation of interventions to prevent compulsory admissions. He implies that our final conclusion that standardisation of the registration of compulsory admissions is needed to improve the comparability of the data is too limited. So he ignores precisely that part of our discussion where we point at the necessity of measures to prevent compulsory admissions. However, reliable data are necessary in the development and monitoring of an innovative prevention policy in an European context.
Flanders WD, Kleinbaum DG. Basic models for disease occurrence in epidemiology. Int J Epidemiology 1995; 24: 1-7.
Lumley Th, Diehr P, Emerson S, Chen L. The importance of the normality assumption in large public health data sets. Ann Rev Public Health 2002; 23: 151-69.
Rothman KJ, Greenland S, Lash TL. Types of epidemiological studies. In: Rothman KJ, Greenland S, Lash TL, Modern Epidemiology. Philadelphia: Lippincot Williams & Wilkins; 2008. pp. 87-99.
Schoevaerts K, Bruffaerts R, Vandenberghe J. Wetenschappelijke validatie van gegevens en cijfers met betrekking tot gedwongen opname in Vlaanderen (2007-2010). Zorginspectie; 2012.
Schoevaerts K, Bruffaerts R, Vandenberghe J. Gedwongen opname in Vlaanderen 2007-2010. Gent: Academia Press Gent; ter perse.
Verbrugghe A, Nys H, Vandenberghe J. Wanneer is een psychose gevaarlijk? Ethische, professionele en juridische afwegingen inzake psychose en gedwongen opname in België. Tijdschr Psychiatr 2008; 50: 149-58.
A note on tests and descriptive epidemiology
Touché. I merged some lines and misread the numbers, but in the end we can all agree that incremental proportions should be compared over a similar period. That does not go without saying when you aim to look into the pitfalls in analysing and comparing these data. Next, everyone will agree that improved registration and prevention of compulsory admissions are important. It is only that I favour regional information systems that cover all types of leverages to help broaden our scope of evaluating new interventions.
However, there can be no rapprochement in the statistical department. We are lectured on types of epidemiological research and the central limit theorem, but this fails to address the issue that differences in annual rates of compulsory admissions should not be tested. Statistical tests cannot generate hypotheses and generally P-values are only impressive and confusing instead of illustrative and inviting for readers. Moreover, in order to satisfy the conditions of the central limit theorem the data have to be independent and identically distributed (i.i.d.). I doubt that these conditions hold. But then, annual numbers of patients admitted to psychiatric hospitals are not samples. They are the real thing. Calculating a Chi-square statistic for ‘raw differences in incidence’ could perhaps be interpreted as a permutation test, but then differences per year in the large numbers of patients that are not involuntarily admitted will contribute to the test statistic. Everything will be very significant statistically, but not very meaningful. There may be large numbers of patients, but rates can be calculated for only a few years.
In this context, including a reference to Kenneth J. Rothman is something of a gotspe. He argued that in data on the entire population there is no sampling error to worry about unless one is interested in the underlying mechanism, such as a biologic norm, in a hypothetical super-population. And Rothman is well known for his battle against the use of P-values. In Epidemiology: an introduction (2002) he wrote: “In nearly all instances, there is no need for any test of statistical significance to be calculated, reported, or relied upon, and we are much better off without them.”