|
Rodgers J, Buchanan T, Heffernan T, Ling J, Scholey A.
“'Ecstasy use, by itself, does not result in residual neurotoxicity'- a powerful argument?”.
Addiction. 2011 Jul 03;106(7):1269-70.
|
|
Abstract
We read with interest the paper published in this journal by Halpern and colleagues [1], which reported findings from a study examining cognitive function in ecstasy/3,4-methylenedioxymethamphetamine (MDMA) users. The authors conclude that they had failed to demonstrate ‘marked residual cognitive effects in ecstasy users’ and they suggest that their data may indicate that ecstasy use, by itself, does not result in ‘residual neurotoxicity’. The publication of this paper has resulted in significant press coverage in the United Kingdom, including a report in the Guardian newspaper which concludes that ‘There is no evidence that ecstasy causes “brain damage” ’[2]. Given the prevalence of ecstasy use, a well-designed and credible study that demonstrated no long-term sequelae would indeed be very welcome; however, we have significant concerns about the study, the conclusions drawn and the subsequent over/misinterpretation of the findings by the popular media, which may mislead the public into assuming that ecstasy can be used without risk.
A key objection to Halpern et al.'s conclusion concerns the statistical power of the study and the effect sizes expected for the impact of ecstasy use on cognitive function. Halpern et al. present the maximum effect size that their design could exclude for each measure tested (i.e. they could say confidently that if an effect of that magnitude existed then their design would find it). They report that ‘we could exclude even a medium effect (Cohen's d ≥ 0.5) on many cognitive measures’ (p. 106). The problem here is that the typical effect sizes reported in previous research are lower than their threshold of likely detection. For example, Zakzanis et al. (2007) [3] conducted a meta-analysis of 35 empirical comparisons of the effects of ecstasy use on multiple domains of cognitive function. They present effect sizes (Cohen's d adjusted for sampling bias) for each of these domains. For those cognitive domains where the effect size was significantly different to zero, the majority were below the magnitude d ≥ 0.5 that Halpern et al. were confident of detecting (for learning and memory, d = −0.55; for verbal comprehension, d = −0.36; for processing speed, d = −0.33; for attention and concentration, d = −0.27; for executive function, d = −0.26; Zakzanis et al. 2007 [3], p. 431). Therefore, Halpern et al.'s design was not capable of detecting the effects found typically in similar research because their sample size was too small. The median value for the effect sizes Zakzanis et al. reported as significantly different from zero was −0.33. The Halpern et al. study had two groups (n = 59 and n = 52). Using power analysis package G*Power 3 to determine post-hoc achieved power for the Halpern study, based on an independent-samples t-test comparing the groups' achieved power was found to be only 0.405.
That is, if comparing the two groups directly with this (typical) magnitude of effect size and these numbers of participants, Halpern et al. had only a 40.5% chance of finding a significant difference between the groups should one actually exist. Their design simply does not have sufficient statistical power to detect effects of the magnitude found typically in comparisons of cognitive function between ecstasy users and non-users and is at significant risk of type II error.
It is also curious that having developed such an extensive test battery the authors downplay the significant differences that are detected between the groups on a variety of cognitive tasks, including memory, vocabulary and fine motor skills. The fact that the authors managed to detect significant differences on any measures despite very low statistical power indicates that these effects are real and important and should not be diminished.
It is also interesting to note that Halpern et al.'s paper makes no reference to the growing body of research into ecstasy-related memory deficits within a real-world context, such as prospective memory (PM) deficits. Given the expanding evidence base within this domain, including the observation that PM deficits persist after statistically controlling for other drug use [4], it would appear remiss of the authors to have excluded such research.
Research into the putative impact of ecstasy use is extensive, with many hundreds, if not thousands, of published studies in the field. A review of this literature appears to indicate that there is comprehensive and robust evidence suggesting that, at least for some users, ecstasy may confer cognitive risk. It is therefore disappointing that papers such as this indicate that perhaps the field has not moved on from simply asking ‘does ecstasy have negative consequences?’. We would hope that a more sophisticated approach which asks ‘if ecstasy use can be problematic what are the risk factors and who is most vulnerable?’ would be a timely and more appropriate use of expertise and resources.
In summary, it is our view that the conclusions drawn from the study published by Halpern et al. are misleading and do not fully acknowledge the significant limitations of their analyses in relation to the low statistical power and therefore potentially misleading the reader.
|
| # |
| Submit Comment |
|
