Descheemaeker and colleagues aimed to differentiate an anaclitic and an introjective cluster of “psychological disorders” (PD) in DSM-IV PD-characteristics. In a cross-sectional study (n = 48) they investigated the relationship between subscales of the SCID-II Personality Questionnaire and the Depressive Experiences Questionnaire. Multiple linear-regression analyses were used to control for the severity of the depression - and controlling for the introjective and anaclitic dimension , “respectively” the authors add. But that is somewhat confusing - the model is:

*PQ subscale = B0 + B1(introjective) + B2(anaclitic) + B3(severity) + error*

What is controlling what in this model is a matter of interpreting the regression coefficients “respectively”. However, assumptions for linear-regression analyses were not met and the conclusion are overstated.

Not normally distributed

PQ-subscales are the sum of yes-answers on SCID-II items, ranging from 6 items for schizoid personality (score range: 0-5 ) to 17 items for narcissistic personality disorder (score range: 0-6). Clearly, these sum-scores are not normally distributed - e.g. mean schizoid scores = 1,31, standard deviation (SD) = 1,29. Descheemaeker and colleagues forgot to report tests for normality of the data and apparently did not consider transforming these skewed data. In general, transformed data are more difficult to interpret and tests of normality are not very helpful (Read here why). But simply assuming a normal distribution in an analysis of skewed data is not the way to go. In case the dependent variable is a count, and mean and variance are about equal, Poisson-regression would be a more appropriate type of analyses.

Capitalising on chance

Moreover, the authors performed twelve regression analysis (PQ dimensions) with three independent variables of which either one of the two DEQ subscales could be statistically significant. This is, of course, capitalising on chance.

Model information is hard to interpret

At request, Descheemaeker and colleagues did not assent to re-analyze their data, claiming that patients had not given informed consent to analyses by a third party. So we don’t know how much, if at all, the results were affected by skewed data and multiple testing.

More importantly, it is difficult to interpret the results. Information about model fit, like R-square, is omitted. And the authors reported standardized regression coefficients and effect sizes, but no confidence intervals (which are important given a small sample size of 48 participants). The standardized regression coefficients represent the change in standard deviation PQ units when a DEQ subscale increases one standard deviation. These numbers are not comparable across models and difficult to interpret (especially when data are skewed). Cohen’s f2 effect sizes are reported, probably squared partial correlations although the method section is not very clear. These effect sizes indicate that the DEQ-dependency score was associated with dependent and borderline personality disorder and DEQ-self criticism was associated with depression personality disorder. Not very upsetting perhaps, but the authors make a lot more out of this.

Hasty generalizations

In his famous paper “The earth is round, p<.05”, Cohen (1994) pointed at the “reject-H0-confirm-the theory“ error. Descheemaeker and colleagues conclude that “there was evidence for a positive relationship” between personality dimensions and dependency and self criticism scores. The authors feel that they “provide important empirical support” for the difference between the anaclitic and introjective dimension. However, this conclusion is based on significant p-values in a small sample. Some no-effect null hypothesis are rejected, but that is not confirmation of the theory. Effect sizes indicate that some results are trivial and others could be difficult to replicate.