Statistics

To return to the main Resources menu, click here.

Scroll down for General Statistics, General YouTube Videos, Justifying Alpha, Power Analysis, Bayesian Statistics, Statistical Parametric Mapping, and Machine Learning.

For information relating to general use of R, please see the Data Visualisation section.

General Statistics

For ten common statistical mistakes, you can read my summary blog post or Twitter thread. You can also see my 5 minute video summary below:

Our editorial in Sports Biomechanics provides ‘Recommendations for statistical analysis involving null hypothesis significance testing’:

Editorial in @sportsbiomechj providing recommendations for conducting and reporting statistical analyses 👇🏻 https://t.co/Qy95u9rEe1
— Stuart McErlain-Naylor (@biomechstu) July 16, 2020

Kristin Sainani has a useful statistics column in PM&R.
This ‘Reference Collection to push back against “Common Statistical Myths”’ is very useful for educational purposes, for myth-busting, and also during the publication and peer-review process.

General YouTube Videos

Kristin Sainani gave an excellent lecture on Statistics in Sports Science as part of my Sports Biomechanics Lecture Series – Kristin is an excellent communicator and this should be useful for a wide range of disciplines:

I also strongly recommend Kristin’s webinar (below) on ‘How to Be a Statistical Detective’. Some of the tools recommended in this webinar can be seen in the Critical Evaluation section of this resources list.

Joshua Starmer has an excellent StatQuest YouTube channel which breaks down statistics and machine learning methods into short animated videos that are easy to understand.
Similarly, although slightly longer, Andy Field posted a series of Undergraduate statistics lectures on YouTube.

Justifying Alpha

This preprint gives practical examples (and related tools) to justify the alpha used in statistical tests, and explains why a default threshold p-value of 0.05 for statistical significance might not always be the best option:

How to justify an alpha (not necessarily 0.05)

A really well written preprint by @MaxMa1er and @lakens: https://t.co/MhPMD8cVWk

Did you know, with high sample sizes certain p values below alpha can be more likely under a true null hypothesis than under a true alternative hyp? pic.twitter.com/SAs6VFyLHe
— Stuart McErlain-Naylor (@biomechstu) June 14, 2021

Power Analysis

This editorial in Journal of Sports Sciences, titled ‘Power, precision, and sample size estimation in sport and exercise science research’ should be useful across all disciplines.
G*Power is a free and easy to use tool for power analysis. For a short (under 5 min) demonstration, see my tutorial video below:

For a more detailed overview, I recommend this introductory non-technical video by Dan Quintana on the ‘Why, how and when’ of power analysis:

Considerations such as resource constraints are also important when planning a study / sample size - as discussed in this pre-print by Daniel Lakens.
If you have to deal with small samples due to various constraints, then this paper provides recommendations for how best to do so.
The Twitter thread below walks you through the use of simulated data to determine power and other characteristics in R:

Here is a little intro thread on how to do simulations of randomized controlled trials.

This thread will take awhile to get all the way through & posted, so please be patient. Maybe wait a few minutes and then come back to it.
— Andrew Althouse (@ADAlthousePhD) January 25, 2021

This tweet gives a useful guide to writing up a power analysis:

Many power analyses I read in published manuscripts are not reported with sufficient transparency to understand what on earth they are doing! What is the effect estimate? Where did it come from? For what test does it apply? Etc. Here is a reporting guide I give students#phdchat pic.twitter.com/BHPThzaiZJ
— Guy Prochilo 🏳️ 🌈 (@GuyProchilo) August 10, 2020

jamovi is also very good and has the benefit of integrating with R. It has a great module for power analysis:

Interested in performing power analysis but not sure where to start? Check out the "jpower" module in @jamovistats, from @richarddmorey.

The way this module is structured is an excellent introduction to power analysis, here's how it works... pic.twitter.com/knBy78PrGx
— Dan Quintana (@dsquintana) May 19, 2019

Finally, Dan Quintana has a ’non-technical guide to performing power analysis in R’:

* Related to power analysis, here’s a fun way of remembering the difference between type 1 and type 2 errors (2 min video):

Bayesian Statistics

For an introduction to Bayesian inferential statistics, I recommend the paper Bayesian data analysis for newcomers.
Tony Myers gave a lecture on ‘Bayesian Statistics for Sport Science’ as part of my Sports Biomechanics Lecture Series:

JASP is an excellent free software for both Bayesian and frequentist statistical analyses, as demonstrated by Tony in his lecture (above). This paper provides a non-technical introduction to Bayesian inference with JASP and a comparison to frequentist null hypothesis significance testing (i.e. p-values). The JASP Guidelines for Conducting and Reporting a Bayesian Analysis may also be useful.

Statistical Parametric Mapping

For those dealing with continuous data, I recorded an ‘SPM in 6 minutes’ video for people with no coding/Matlab experience wanting to use statistical parametric mapping (SPM):

This was followed by a video on how to do other SPM tests such as ANOVAs and shorter 2 minute video tutorials on adding p-value labels and manually editing figures.
I presented ‘a practical open-source comparison of discrete and continuous biomechanical analysis techniques’ at the International Society of Biomechanics in Sports conference in 2020 (video below; paper here ):

For much more detail, this lecture by Todd Pataky may be useful on ‘Discrete vs Continuous Biomechanical Data Analysis’:

Machine Learning

For a free 80-video machine learning course, see this YouTube playlist.