The following is an attempt to give an overview of some of the criticisms of the statistical assertions made in the Ricaurte 10/98 lancet article.¹

The comments are based on the paper published on the lancet's web site (www.thelancet.com) and no one I've spoken to has seen the raw underlying data so perhaps there are further data which should be considered.

This analysis is not intended to put forth an argument that there is NO relationship between MDMA use and changes in neurobiology. But work rather is intended to address the assertion (mostly in the popular press) that this study has 'proven' that MDMA 'causes brain damage'. ^{2, 3, 4}

---

Although the "Findings" and "Dicussion" sections of the paper suggests that MDMA use is causally related to decreased 5HT levels by asserting:

"MDMA users showed decreased global and regional brain 5-HT transporter binding compared with controls. Decreases in 5-HT transporter binding positively correlated with the extent of previous MDMA use.... (r=0.50, p=0.0005, figure 4)" and "Recreational MDMA use can lead to global, dose-related decreases in the brain 5-HT transporter".

There are four points that call the findings and summaries into question.

First, the MDMA users do not appear to have abnormally low 5-HT transporter levels. Second, the reported statistical correlation may not accurately reflect the pattern of the results. Third, it is unreliable to infer causation using correlational data from small samples. And Fourth, the paper appears to violate a rule for the MANOVA analysis it uses.

1. The most compelling argument is apparently based primarily on the measurements which are charted in figure 4. If we look at that chart, there is one individual who falls well outside the range for the rest of the test subjects (150 reported MDMA uses scoring a 1 on the chart's scale). To be conservative, outliers are often excluded from statistical analyses because of the exaggerated effect they can have on certain calculations. Excluding that individual, all of the rest of the MDMA user's scores are within the same range as the non-MDMA users. Since two of the 14 MDMA users are near the top of the non-MDMA user range (above the majority of controls), it seems hard to say with a great deal of confidence that the data supports the Findings.
   
   
2. The computed correlation (0.50) may be misleading because of the intentional inclusion of a large number of control subjects (zeroes on the horizontal axis). It is generally unwise to choose subjects for a correlational analysis on the basis of their scores on one of the variables in the correlation: in this case selecting people who have a score of 0 on the "number of MDMA uses". Generally speaking, it is difficult to interpret the meaning of correlations computed in these kinds of studies.
   
   You can see why this is so by examining the data in Figure 4. (Compare Graphs) When the zeroes are removed, the pattern appears to be largely due to one subject (who used 70 times). Analyzing the remaining data using statistical tests, reveals that the correlation is no longer significant, meaning that we cannot confirm that this pattern isn't just random.
   
   
3. The study paper not only suggests that there is a strong correlation between MDMA use and the tested 5-HT transporter, but that the relationship is causal: "Recreational MDMA use can lead to global, dose-related decreases in the brain 5-HT transporter, a structural element of brain 5-HT neurons." Even if the data supported correlation, this is far from indicating a causal relationship. There are many other ways to understand this association, including (but not limited to): a) subjects with pre-existing low levels of whatever may be predisposed to MDMA use; and (b) the MDMA users with low levels of whatever may differ from the other MDMA and non-MDMA using subjects in this study in some other way, for example, they may be polydrug abusers and it may be that polydrug abuse, and not MDMA use per se, that is related to low levels of whatever. 5,6
   
   
4. Another area which is troublesome is that MANOVA, the statistical procedure the authors use to make their point, has been described to me as 'notoriously unstable with small samples.' The n=14 of the study seems far too low and the tables used to do the analysis are not supplied in the paper. I have also been told that MANOVA is a method for experimental design in which subjects are assumed to have been randomly assigned to groups and it does not appear that this is true for this study, which would violate a basic assumption of this analysis method and invalidate the analysis.
   
   

   ---

   
   
   End Notes:
   
   1 - Ricaurte study
   
   2 - Press release from The Lancet
   
   3 - SJ Mercury News story
   
   4 -AP story
   
   5 - Lamont Granquist's response
   
   6 - H's response