{"id":903,"date":"2015-10-23T08:31:59","date_gmt":"2015-10-23T08:31:59","guid":{"rendered":"https:\/\/staffblogs.le.ac.uk\/bayeswithstata\/?p=903"},"modified":"2025-02-26T13:21:37","modified_gmt":"2025-02-26T13:21:37","slug":"banning-p-values","status":"publish","type":"post","link":"https:\/\/staffblogs.le.ac.uk\/bayeswithstata\/2015\/10\/23\/banning-p-values\/","title":{"rendered":"Banning p-values"},"content":{"rendered":"

If you were to look\u00a0back\u00a0at my\u00a0previous postings\u00a0on this blog, you would\u00a0find that I am often motivated by topics that arise from teaching on our Masters course in Medical Statistics. Well, a new cohort of students started recently and in the first couple of weeks we give them an overview of basic statistics to make sure that everyone is up to the same level. This year, as\u00a0part of that introduction,\u00a0one of my colleagues\u00a0asked the students to\u00a0discuss the fact that\u00a0the psychology journal, Basic and Applied Social Psychology<\/a><\/em>,\u00a0has banned p-values from\u00a0the articles that it publishes.<\/p>\n

This is a Bayesian blog so, as you might expect, I have a great deal of sympathy with anyone who has a problem with p-values, but is a ban the right reaction?<\/p>\n

Here is a question for you. What have the following in common?<\/p>\n

(a) the charge of Light Brigade<\/p>\n

(b) the election of Jeremy Corbyn to lead the UK Labour party<\/p>\n

(c) the banning of p-values by Basic and Applied Social Psychology<\/em><\/p>\n

My answer is that all three are admirable because they were inspired by sincerely held, well-founded beliefs, but\u00a0in each case it was clear from the outset that the venture was going to end in tears.<\/p>\n

Let me steer clear of history and politics and concentrate on the statistics.<\/p>\n

Anyone who knows anything about the fundamentals of statistics will know that the\u00a0logic of null hypotheses and p-values has very little of relevance to say about scientific investigation.\u00a0This is not the place to repeat\u00a0those arguments but\u00a0there are so many well-known problems with p-values that they definitely deserve\u00a0a place in the\u00a0rubbish bin of history.<\/p>\n

If that were not enough, we can add\u00a0to the list of problems\u00a0with\u00a0p-values\u00a0that so many scientists misunderstand and misuse them. In my experience, few people outside of the specialist statistics community have any real understanding of what a p-value actually is. So most people use p-values as a\u00a0blackbox method and even\u00a0then\u00a0there are\u00a0problems, because, due to a historical accident, conclusions are usually based on whether p<0.05.\u00a0If we were to reset\u00a0the\u00a0threshold today, we\u00a0would surely choose a stricter cut-off.<\/p>\n

So, good for Basic and Applied Social Psychology<\/em>, they have done the scientific world a great favour and I’m sure that people will look back\u00a0on the banning of p-values as an important step forward.<\/p>\n

The Journal’s ban extends to confidence intervals, as logically it must once p-values have been dropped, and they even express doubts about Bayesian methods, so they end up with something that is very close to an attack on statistics, masquerading as an attack on p-values.<\/p>\n

In fact, their doubts about Bayesian statistics are, in themselves, quite interesting. The editors\u00a0attack the Laplacian assumption that ignorance can be expressed by equal probabilities. Effectively an attack on the adoption of the flat priors that is so common in poor quality Bayesian analyses. So I am even in sympathy with those views.<\/p>\n

Where the ban turns into glorious farce is that the editors have so little that is positive to offer as an alternative.<\/p>\n

For instance, they say “we encourage the use of larger sample sizes than is typical in much psychological research”. I would certainly give my students a hard time if they said something like that. How large is large? The statement means nothing unless you have a method for defining the necessary sample size and how do you do that without a calculation that comes very close to depending on the standard error, which in turn is linked to the p-value.<\/p>\n

The editors’\u00a0other requirement is for “strong descriptive statistics, including effect sizes”. So I do a “large” psychological experiment and I find a 6% difference between the average scores of men and women. What do you conclude? Without some consideration of\u00a0sampling variation you can conclude nothing. So I tell you that men scored between 30 and 80 and women between 28 and 85. Descriptive but still not enough. Already you will probably be saying to yourself, a range of 50, so the standard deviation must have been\u00a0just over\u00a010, if I knew the sample size I could calculate the standard error of the mean.<\/p>\n

Drop p-values by all means but you cannot escape from a consideration of random variation.<\/p>\n

Banning p-values and then leaving the authors without firm guidance is gloriously daft. It is a pity that the editors\u00a0missed the opportunity to\u00a0advocate subjective Bayesian analysis or pure likelihood analysis, but I am really pleased that they have taken a step in the right direction. Their journal\u00a0will suffer, but science will gain.<\/p>\n","protected":false},"excerpt":{"rendered":"

If you were to look\u00a0back\u00a0at my\u00a0previous postings\u00a0on this blog, you would\u00a0find that I am often motivated by topics that arise from teaching on our Masters course in Medical Statistics. Well, a new cohort of students started recently and in the first couple of weeks we give them an overview of basic statistics to make sure […]<\/p>\n","protected":false},"author":134,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[87,86],"class_list":["post-903","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-ban","tag-p-values"],"_links":{"self":[{"href":"https:\/\/staffblogs.le.ac.uk\/bayeswithstata\/wp-json\/wp\/v2\/posts\/903","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/staffblogs.le.ac.uk\/bayeswithstata\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/staffblogs.le.ac.uk\/bayeswithstata\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/staffblogs.le.ac.uk\/bayeswithstata\/wp-json\/wp\/v2\/users\/134"}],"replies":[{"embeddable":true,"href":"https:\/\/staffblogs.le.ac.uk\/bayeswithstata\/wp-json\/wp\/v2\/comments?post=903"}],"version-history":[{"count":14,"href":"https:\/\/staffblogs.le.ac.uk\/bayeswithstata\/wp-json\/wp\/v2\/posts\/903\/revisions"}],"predecessor-version":[{"id":962,"href":"https:\/\/staffblogs.le.ac.uk\/bayeswithstata\/wp-json\/wp\/v2\/posts\/903\/revisions\/962"}],"wp:attachment":[{"href":"https:\/\/staffblogs.le.ac.uk\/bayeswithstata\/wp-json\/wp\/v2\/media?parent=903"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/staffblogs.le.ac.uk\/bayeswithstata\/wp-json\/wp\/v2\/categories?post=903"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/staffblogs.le.ac.uk\/bayeswithstata\/wp-json\/wp\/v2\/tags?post=903"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}