My p-value is smaller than your p-value The p-value is the probability of the test-statistic, not the probability that the null hypothesis is true.  Probability(D|H) reads: the probability of the test statistics, given the hypothesis; research does not predict the happening of a hypothesis: Probability(H|D). In other words: the probability of Death by Hanging is not to be confused with the probability of suicide by Hanging as cause of Death. ...See articles

Reality - an interesting hypothesis In general the nil hypothesis is the null hypothesis. But the "absolutely no effect whatsoever" hypothesis will almost always be rejected so that a Type I error (falsely rejecting the null) cannot occur, but the probability of a Type II error (falsely not accepting the alternative) often is about 50%. Correcting for multiple testing decreases the power of the test even more. What are we testing? ...See articles

Knowledge is power - Do statistics Using test statistics without checking assumptions can give trouble. For instance: percentages follow a binomial distribution and differences between groups can be tested with a simple t-test only when the percentages are not too close to 0% or 100%. T-tests and ANOVA's should not be used on counts and log-tranforming these counts to correct skewed data creates it's own bias. More troubles ...See articles

What's in a model Statistical tests can be viewed as models. A two-sided Student's t-test for comparing the means of two groups can be formulated as a simple (not easy but with one independent variable) regression model. The test evaluates if the estimated parameter in the model is different from 0. More complex models are more difficult to interpret and assumptions harder to test. So much can go wrong ...See articles

Science is fun The term "p-value fallacy" covers a range of misinterpretations of the meaning of p-values. One common mistakes is that if the result is not statistically significant, the null-hypothesis is accepted and we "have to" conclude that there is no difference between samples. But we should never interpret large p-values as establishing that the null hypothesis is true. Interpretations can go off-base ...See articles

Elementary table The classification table in logistic regression shows how many cases are correctly predicted (# cases are observed to be 0 and are correctly predicted to be 0; # cases observed to be 1 and correctly predicted to be 1), and how many cases are not correctly predicted (# cases observed to be 0 but are predicted to be 1; # cases observed to be 1 but are predicted to be 0). Or does it? ...See articles