By Eiko I. Fried and Michiel van Elk
Much optimism has been expressed about the potential of psychedelics to treat mental health problems such as suicidal ideation, depression, and post-traumatic stress disorder. This optimism goes hand in hand with a notable increase in research publications, investments from pharmaceutical companies, patent filings, media exposure, as well as shifts in political and legislative landscapes. In the U.S., hundreds of ketamine clinics have emerged in the last years, and Australia just recently acknowledged psychedelics as medicines.
In our new paper in print in the journal Therapeutic Advances in Psychopharmacology, titled “History repeating: Guidelines to address common problems in psychedelic science,” we critically discuss whether this optimism is warranted, given current empirical work. One of us (Dr. Eiko Fried) is an Associate Professor in Clinical Psychology who studies and teaches conducting and evaluating clinical trials for mental health problems such as depression, with a focus on the question what valid inferences can be drawn from empirical data. The other (Dr. Michiel van Elk) is an Associate Professor in Cognitive Psychology, with a long-standing interest in the topic of altered states of consciousness (including, but not limited to, states induced through psychedelics), who, together with the PRSM lab, uses a multi-method research approach, open science practices and replication studies to study these experiences in the lab and in naturalistic settings.
Our review identifies ten major challenges psychedelic science is facing—we discuss these challenges, classify them as easy / moderate / difficult, and provide a roadmap moving forward. We end the paper with a checklist for reviewers, journalists, funders, and policy makers to facilitate the accurate assessment of the quality and rigor of psychedelic studies.
In the remainder of the blog post, we briefly introduce 7 of these challenges. Many of them have in common that they are well studied and understood, and jeopardize the validity of other clinical interventions (e.g., pharmacological or psychotherapeutic interventions such as antidepressants or mindfulness-based therapy). It was therefore surprising to see that psychedelic science appears to ignore many of the lessons learned from decades of research in other fields. As we write in the paper: “Much of the recent work is history repeating itself.”
- Conclusions are dramatically overstated in many studies. This ranges from conclusions in the results sections, abstracts, and even titles of papers that are not consistent with the reported results. One example is a paper titled “Rapid and Sustained Reductions in Current Suicidal Ideation” in which only 14% of patients actually showed sustained improvement in a ketamine study.
- There is emerging evidence that adverse events resulting from psychedelic substances are both common and underreported. For example, 41.5% of serious adverse advents in ketamine trials for depression were not reported in the resulting studies, and 94% of these events occurred in the ketamine rather than the placebo groups. We also discuss the issue of safety in the therapy room, given reports of therapist abuse.
- Lack of control groups are a problem for interpretation, because treatments like psychedelics need to be compared against a placebo or other treatment to conclude that they work beyond the placebo effect or already existing, readily available treatments. Further, psychedelics may act as “super placebos” because of several factors, including the current media hype, resulting in increased expectations of study participants, and the fact that psychedelics can enhance suggestibility.
- Participants in psychedelic studies usually know if they are in the treatment or control group, which artificially increases the apparent efficacies of psychedelics in clinical studies (e.g., due to increased expectations in the active group, or disappointment and higher drop-out rates in the control condition).
- Small study samples threaten statistical power and generalizability. Sufficient statistical power is needed so that we can rely on results, for instance, that a psychedelic drug outperforms a control condition; this requires samples much larger than we see in most psychedelic studies. Small samples also mean that results are not representative. For example, participants with severe or comorbid mental health problems are commonly excluded from psychedelic studies, and therefore results may look better in these studies than in real-world psychiatric settings.
- Questionable research practices are commonplace, which are practices that increase the likelihood of positive findings. For example, outcome switching is a common practice, where researchers define a priori a specific outcome measure, such as a particular depression scale for a depression study that serves as the so-called “primary outcome”. However, they later do not report the results for this outcome, but instead analyze another outcome measure that was also collected, likely because the primary outcome measure did not show the desired results.
- Clinical studies usually only last for a few weeks, which is not enough to distinguish short-term symptom relief from successful treatment. Many psychedelic studies are performed with very ill populations, such as people with treatment-resistant depression who have often had depression for over a decade. Studying how these people are feeling a few days or weeks after they receive treatment is not sufficient to establish that they are indeed cured from depression, which requires much longer follow-ups of at least half a year, in much the same way we study treatments for other chronic diseases like cancer.
While many in psychedelic science work on better understanding these problems and addressing them, other prominent researchers have deflected with the argument that these problems are not unique to psychedelic science. We agree, but that means that contemporary psychedelic research has had the opportunity to learn from many decades of grappling with placebo effects, conflicts of interest, and transparent reporting of adverse events. Presently, psychedelic science is history repeating, and little concerted action has been taken to address these and other challenges.
As a result, when looking at the literature under the recently published guidelines by the National Institutes of Health, most work would be considered “low priority” for funding because it lacks “basic quality controls and methodological rigor.” Our paper concludes with a long list of specific recommendations for the field to move forward.
Eiko I. Fried is an Associate Professor at Leiden University’s Institute of Psychology.
Michiel van Elk is an Associate Professor at Leiden University’s Institute of Psychology.
