Statistical reporting errors in psychology (2016)

Nuijen and colleagues studied reporting errors in a sample of over 250,000 p-values reported in eight major psychology journals. The authors developed the new R package Statcheck. This set of programs was used to retrieve null-hypothesis significance testing (NHST) results. The paper reports that one in eight papers contained a grossly inconsistent p-value that may have affected the statistical conclusion. The prevalence of gross inconsistencies was higher in p-values reported as significant. The authors conclude: "This could indicate a systematic bias in favor of significant results. Possible solutions for the high prevalence of reporting inconsistencies could be to encourage sharing data, to let co-authors check results in a so-called co-pilot model, and to use statcheck to flag possible inconsistencies in one's own manuscript or during the review process."

 

 

The ASA statement on p-values (2016)

The American Statistical Association is moving into a 'post p-value era': six principles regarding the misuse of the p<.05 threshold.

Ronald L. Wasserstein & Nicole A. Lazar (2016): The ASA's statement on p-values: context, process, and purpose, The American Statistician

http://dx.doi.org/10.1080/00031305.2016.1154108

 

Morrens (2013): And the winner is…

In the December issue of “Tijdschrift voor Psychiatrie” Morrens reported on the winners of the prize for best paper in 2013 with a doctor in training in psychiatry as first author. A five-member jury rated a total of 15 papers on theoretics, originality, use of language, structure, and scientific, social and clinical relevancy. The 1250 euro first prize was awarded to the paper “An increase of compulsory admissions in Belgium and the Netherlands: an epidemiological exploration”. According to the jury, the authors demonstrated that the incidence of compulsory admissions per 100,000 inhabitants increased by 42% in Belgium and 25% in the Netherlands. These numbers were singled out eventhough a commentary by me, that was published in the journal’s April issue, pointed out that these percentages cannot be compared because they concern different time periods, 1999-2008 and 2002-2009 respectively. The authors confirmed that the increase in the period 2002-2008 was roughly equal in both countries, i.e. 21% and 20% (Tijdschrift voor Psychiatrie, 2013, 4: 304 - 306). But this was not the most important issue. 

Descheemaeker et al. (2013) got lost in dimensions

Descheemaeker and colleagues aimed to differentiate an anaclitic and an introjective cluster of “psychological disorders” (PD) in DSM-IV PD-characteristics. In a cross-sectional study (n = 48) they investigated the relationship between subscales of the SCID-II Personality Questionnaire and the Depressive Experiences Questionnaire. Multiple linear-regression analyses were used to control for the severity of the depression - and controlling for the introjective and anaclitic dimension , “respectively” the authors add. But that is somewhat confusing - the model is: 

PQ subscale = B0 + B1(introjective) + B2(anaclitic) + B3(severity) + error

What is controlling what in this model is a matter of interpreting the regression coefficients “respectively”. However, assumptions for linear-regression analyses were not met and the conclusion are overstated. 

Van den Bosch et al. (2013) intended to treat in a pilot study

Van den Bosch and colleagues conducted a pilot study to prepare a randomised clinical trial comparing a short intensive course of dialectical behaviour therapy with the standard outpatient. For 39 female patients with borderline problems information was collected on (para)suicidal behaviour, drop-out, severity of borderline problems and quality of life. The English abstract suggests that the authors used participating observation as a data collection method: “We participated in 3-month-long inpatient dbt programme” (something was added in translation). Results showed that severity of borderline problems was significantly reduced, but there was no significant reduction in (para) suicidal behaviours. However, what happened to dropouts?