Brain zapping, or non-invasive brain stimulation, in all its forms, has hit the mainstream. You can even watch a YouTube video about how to build your very own not-approved-for-human-use transcranial magnetic stimulation machine! Light-heartedness aside, the therapeutic benefits of brain stimulation are regularly highlighted in the news, and the number of medical clinics that specialize in transcranial magnetic stimulation is increasing.
Around since the early 1980’s, transcranial magnetic stimulation (TMS) has become a popular way to investigate brain function and even to treat some brain disorders. Unfortunately, research supporting this ever increasing popularity is beset by studies with small sample sizes and true effects that are often small or nonexistent.
Over the years, various research groups have had difficulty reproducing published results involving transcranial magnetic stimulation. We therefore decided to survey scientists who use TMS to alter the behaviour of the brain’s motor cortex about their own experiences and research practices.
WHAT DID WE FIND?
Our survey of TMS researchers who published papers using the technique revealed that they could reproduce the originally reported effects only about half the time. This does not match up with the published scientific literature, where virtually all publications in this field report positive results. One explanation for this would be a publication bias towards ‘positive’ studies.
We also discovered a worrying prevalence of questionable research practices. These include stopping a study when the results seem ‘significant’, selecting subjects who ‘respond’ to a protocol, or cherry-picking what data to report – surprisingly even just on a ‘gut feeling’. And despite unanimity among respondents that these practices should always be reported in publications, we found that they were not actually mentioned.
SIGNIFICANCE AND IMPLICATIONS:
These results have many worrying implications. They highlight a possible waste of research funds, personnel and resources in a field where insufficient attention is given to ensuring scientific discoveries are robust and reproducible. The increased use of TMS as a therapeutic tool is also worrying because physicians involved in its prescription may be unaware of its limitations, and ultimately patients may not receive appropriate information about a treatment.
While a remedy for these endemic problems has yet to be found, we believe that increasing the sample sizes in TMS studies and eliminating – or at the very least reporting – questionable research practices would be a simple first step towards reproducible TMS research.
Héroux ME, Taylor JL, Gandevia SC (2015). The Use and Abuse of Transcranial Magnetic Stimulation to Modulate Corticospinal Excitability in Humans. PLoS One 10: e0144151.