In January 2011 Dr. William Carpenter, editor in chief of Schizophrenia Bulletin, decided negatively on publication of our manuscript entitled "Improved insight affects social outcomes in psychotic patients: a one-year follow-up study". The decision stands, of course, because none of the referees gave the manuscript the highest rating and editors also review the material and reach their own conclusions. However, the negative comments of two anonymous referees clearly showed that they commented on statistical analyses they didn’t fully understand.

Two out of four reviewers were positive: “Hypothesis raised by the authors seemed to me well defined and they used a sound methodological approach with new but complex statistical analyses” (referee 2) and “The study will contribute significantly to the existent literature on this clinically and scientifically relevant topic” (referee 3).

We used mixed models in a longitudinal analysis of the effect of change in illness insight on change in social outcomes. However, referee 1 was confused about the dependent variables used: “Are they the final scores of the social outcome variables or the change from baseline to final score?”. “Are the scores used in the models raw scores or changes in scores from baseline to 12-month?”. Apparently the only analyses this referee is familiar with is the variance-covariance OLS-regression analysis or the analysis of change scores (which is often inappropriate even in a pre-post test analysis). Likewise, referee 4 seemed only at comfort with good old repeated measures ANOVA: “The graphs are inappropriate. It can't and shouldn't show months 2-11, nor have data points (black circles) at these points - they don't have the data to do that.” The fact that we graphically illustrated a statistical model based on 1, 6 and 12 month assessments and even included a reference to support our approach, apperently was beyond the intellectual capability of this anonymous referee.

The editor in chief of Schizophrenia Bulletin reached his own conclusion on our manuscript, but failed to apologize for this substandard review process. It encouraged me to accept a review invitation in future only on the condition that the authors will receive the exact review text including my name. It seems to me that signed reviews are the only way to get rid of the review process that gives a platform to arrogant and unskilled researchers.

 

Post scriptum. The paper was published in Comprehensive psychiatry:

Antoon W B van Baars, André I Wierdsma, Michiel W Hengeveld, Cornelis L Mulder (04/2013). Improved insight affects social outcomes in involuntarily committed psychotic patients: A longitudinal study in the Netherlands. DOI:10.1016/j.comppsych.2013.03.016