Over the past six years, the Experience Vaults have become a major cornerstone of the Erowid library. When we began Erowid, we never imagined how valuable these experience reports would become. Thanks to a robust triage and review system for submissions, both the quality and popularity of the Experience Vaults continues to rise.

Since we first wrote an introduction of the system in October 2002,1 we have received a number of questions and criticisms about the Experience Vaults. We hope this article will help provide some answers to the criticisms as well as insight into why we consider experience reports to be so important to our ongoing work.

The purpose of the Erowid Experience Vaults is to collect, review, and categorize a large compilation of first-person reports about the use of psychoactive plants, chemicals, and technologies. In 2000, when we initially launched the software used to manage these reports, we had in mind moving from several hundred reports to a goal of 100,000. We saw several benefits to publishing this many reports:
   • Reduce the likelihood of presenting an unbalanced picture due to having only a few unrepresentative reports;
   • Create a common format that can be used to describe hundreds or thousands of different substances;
   • Help document and communicate why people use psychoactive substances;
   • Allow users to describe, in their own words, the experiences they have and the impact these substances have on their lives;
   • Permit visitors who have little or no knowledge of a substance to learn about it from those who use it.

Erowid currently receives more than 25 experience reports every day; only a portion of these contain useful descriptions. Because of this, each submitted report is read and reviewed by knowledgeable peers--people well-read about a wide variety of psychoactives, their dosages, and their effects. The goal of this process is to check each report, before publishing, for interest, quality, accuracy, and general believability.

When a report is submitted, it enters a "triaging" system where it is read, graded from A to F, and commented on by at least two trained triagers. There are currently 36 volunteer triagers working on this process. Once a report has been triaged by two individuals, it is then edited by a reviewer who categorizes and gives a final rating to each report before publishing. It is the triager's job to wade through incoming submissions looking for the gems, so that the painstaking task of editing is not wasted on poor reports. Reviewers are selected from the most experienced triagers who have shown dedication and attention to detail during the triaging process. There are currently five active reviewers.

The triaging process was added to the review system in 2004 and provides several benefits. First, it allows new crew members to more easily get involved by creating an entry-level position that requires less training. It also creates a self-selecting process by which motivated volunteers can become reviewers. The triage system improves the overall quality of published reports by allowing reviewers to focus their attention on fewer, higher-quality reports. This shortens the time between the submission and publication of better reports. Finally, the triage system provides a feedback loop that allows administrators to keep an eye on whether the ratings of individual crew members are inconsistent with others, possibly indicating a need for more training.

The review process allows us to verify that submitted texts are believable first-person experiences, remove intentional or inadvertent product endorsements, and ensure that no one's privacy is compromised. All of this helps improve the quality of the collection.

With few exceptions, the Erowid Experience Vaults consist of first-person descriptions of the use of psychoactives. Our editorial policy is to reject most third-person descriptions of other people's experiences, impersonal descriptions of psychoactives and their use, instructional articles, or other writings that are not first-hand accounts.

Statements people make about their own experiences have an inherent validity. After all, answering the subjective question, "What is your experience like?" requires a first-person point of view.

Because people want to contribute to the growing field of knowledge about psychoactives, many are tempted to write "authoritative" articles that require a great deal of research and fact checking in order to be validated. However, most people do not have the required knowledge or resources to add to the literature at that level.

Experience reports provide an outlet for people to share their knowledge in a context where readers are primed to understand the article for what it is: a personal, first-person account. One may disagree with their statements or have conflicting experiences, but, at a fundamental level, a first-person description of an experience cannot be argued with.

I'm Glad they Write...

Feedback from an M.D. "I appreciate the Erowid site and the way it is run. I take no mind-altering substances. I am a doctor--specifically an emergency room physician in rural Illinois. I have to take care of people who take all kinds of substances: created chemicals, prescription and non-prescription medications, herbs and anything that will alter their experience of the universe. Unfortunately, of course, lots of them don't think, and don't read, and mix things that shouldn't be mixed. Most of the time they come in because they are too stoned to respond or are having a panic attack (or equivalent). Most of them can't even remember (or won't admit) what they took. But when they can, it's nice to be able to type it in to the search on Erowid and see if anyone else has had the same experience. Just today, someone took 200 mg Zoloft thinking it was Tylenol (after all, it was in the Tylenol bottle). Nothing else showed up in his urine. Poison control was worthless--they told me the amount he took didn't even qualify as an ingestion since it could have been prescribed in that dosage. Then they faxed me 13 pages of information that was worthless because it was too verbose to read. But all of the symptoms he had were described on your site. Succinctly. Having found your incredibly useful site, I have recommended it to other physicians and law-enforcement. They are all grateful. This sounds sarcastic, but I'm serious. Anyway, I'm sorry anybody takes mind-altering substances, because I consider it a preventable cause of a trip to the emergency room. But if they do, I'm glad they write about it." Sign me, Clean enough to sell pee

Experience reports offer a direct way to document the use and effects of psychoactives. Individual reports can be compared to medical "case reports" that are commonly published in peer-reviewed journals. The Experience Vaults are a hybrid between a survey and a peer-reviewed collection of articles.

Often dismissed as "anecdotal", self-reports highlight the fundamentally subjective nature of having one's thoughts and feelings influenced by taking a substance. As with medical case reports, a single experience report can not be assumed to be representative of the wider population; it is an individual data point about what happened to one person, who used a particular psychoactive, on a particular day, at a particular dose. In isolation, any single report is just one person's opinion, but en masse those opinions can be discussed objectively, in the same way that surveys can transform personal opinions into quantifiable data.

In some ways, first-person experience reports are weaker than medical case reports, which are generally written by an attending physician. Most of Erowid's experience reports are anonymous, for legal and privacy reasons, so there is no way to contact the author for follow-up questions. It is rare for experience reports to include toxicology information validating the identity of the substance(s) taken.

But experience reports are more valuable than medical case reports in important ways. Case reports are usually restricted to events that resulted in a medical emergency; this leaves a huge void in the collected data. Experience reports are not limited in this way. Experience reports provide greater insight into the thoughts of users, the way they make choices, and the meaning of their experiences. They provide the intimacy of the personal subjective narrative as well as the value of an objective collection. As the number of collected reports increases, one can make better generalizations about the range of doses used, how dose relates to subjective effects, the range of effects, and what doses or substance combinations are most likely to lead to health problems.

It is important to remind people that the Experience Vaults cannot, by themselves, be assumed to be representative of the general population. Not only are Erowid visitors likely to differ from the general population, but people willing to describe their (sometimes illegal) psychoactive use may not be characteristic of even the psychoactive-using public. Furthermore, our peer review process inevitably introduces biases that will affect the collection in currently unquantified ways.

Yet, despite the problems with making valid statistical extrapolations from the Experience Vaults, they can be compared to a long-term, ongoing, open-format survey. The individual submissions form a collection that can be analyzed to provide an outline of substances, dosages, contexts, and routes of administration, as well as the resulting range of effects. Sophisticated textual processing of the reports (see "Surfing the Matrix") may be able to quantify additional complex relationships between these factors.

The experience of using psychoactives is quintessentially subjective. This type of subjective data is extremely difficult to collect with a standardized set of multiple-choice questions.

Simple surveys collect aggregate data that tends to focus on majority and common effects. Numerically oriented surveys and research "instruments" (formalized survey-like forms filled out by research subjects), often lose or even intentionally remove unusual or unique reactions as "outliers". Most surveys result in distilled summaries that discount and flatten the value of unique experiences.

Small populations and niche groups often have their views better communicated through narrative experience reports. These reports spotlight individuals and emphasize the primacy of each person's experience, regardless of how uncommon that experience may be. A single eloquent description of an unusual reaction or experience can go a long way towards making that experience more "real" in the minds of those studying the subject.

In the late 1990s, we helped work on a research protocol involving the administration of a psychedelic to healthy adults. We read and discussed with researchers the existing "empirical instruments" that had been used to try to formally quantify psychedelic experiences. These include Strassman's Hallucinogen Rating Scale, the Piedmont Transcendence Scale, Hood's Mysticism Scale, and the Psychedelic Experience Questionnaire, among others. In 1998 and 1999, we also experimented with multiple-choice and short-answer surveys created and distributed at Burning Man.

It became clear that attempting to record and measure people's experiences with psychoactives is a very rough science; all of the available measurement instruments are flawed or limited, especially when trying to explore and study areas that have not already been well documented. Checkboxes, scales, and short-answer questions constrain responses, resulting in critical elements of an experience being missed.

While there are certainly great opportunities to collect interesting data through multiple-choice and numerically-oriented instruments, these do little to communicate the experiences themselves.

There is nothing quite like reading a large number of experience reports to get a sense of a given substance or the ways that people put it to use. The main reason many people write about their experiences is to share their hard-won knowledge with others. A huge portion of psychoactive users who visit Erowid want to be careful about their use, and reading reports can point out the difference between sloppy users and meticulous ones, or between careful and careless practices. Readers can quickly ascertain the potential dangers associated with a substance based on the bad choices that others have made and documented. Many people can learn from the mistakes of one individual. Reading these reports is also a stark reminder that it is impossible to capture the full spectrum of experiences in a single document or a single voice.

Over the last five years, we have been encouraged by an increase in the number of physicians, nurses, and teachers who tell us how Erowid's Experience Vaults have been useful in their work. Even experts who have read the available mainstream literature about a drug's effects can gain new and important insights through the reports.

A recent article published in Drug and Alcohol Dependence,2 by EJ Cone attempts to describe the various routes of administration that people use with psychoactive pharmaceuticals. His research was made much easier, in part, by the detailed categorization of Erowid experience reports by route of administration and substance. Unfortunately, while Cone cited sixteen individual Erowid reports that he used in his analysis, he did not acknowledge the painstaking work Erowid has done to make this type of research easier.

Beyond being valuable to others, writing an experience report can help the author more fully integrate and learn from their own experiences. By recording their thoughts, an author can solidify an experience in their memory and work through life issues that arise. Writing about one's experiences promotes a meta-awareness of the choices one makes about psychoactive use. Over time, this should increase the care with which people use psychoactive plants and chemicals.

Well-written experience descriptions can compellingly communicate to those outside drug-using subcultures. They can put a personal face on psychoactive use that is more detailed and nuanced than the caricature of the drug user portrayed by the mainstream media. A multiplicity of voices describing personal experiences can help provide insight for those who want to understand the broad diversity of people who choose to use psychoactives.

The primary complaint we receive about the Experience Vaults is that we publish too many poorly written or dull reports. Although quality of writing is one factor used in grading reports, we feel strongly that reports written by less educated or less skilled writers should not be excluded: accepting only erudite, articulate reports would introduce its own bias. This goes hand-in-hand with a decision not to completely correct and standardize grammar and spelling in reports. Writing skill and style are important elements that help convey a sense of the author. The style, tone, and word choices made by an author can help peer reviewers assess the validity of what the author describes.

We understand that some visitors would prefer not to read poorly written reports, and over time, we hope to improve filtering and sorting interfaces so that individuals can avoid reading reports that fall below their specified quality thresholds.

Some visitors express concern that experience reports dilute the value of other documents Erowid publishes because many reports seem trivial and uninteresting. These people complain that our What's New page is too dominated by the constant flow of reports being published. While a single report of LSD taken at a mall certainly is not as valuable, on a per-document basis, as a well-edited overview of the pharmacology of hallucinogens, it is significantly more accessible for the average person.

We continue to work to improve features such as the What's New page, search engine, and indexes to make it easier for visitors to avoid wading through documents that don't interest them. But, we do not feel, as some have suggested, that it is inappropriate to display experience reports side-by-side with scientific articles.

Some visitors have asked "Don't you have enough?" But in our view, the answer is clearly "No". Even with a large collection of reports, there are many substances, sub-sets of users, or types of use that are inadequately represented.

Of 10,299 published reports, only 61 describe absinthe use, less than 100 describe ayahuasca experiences, and only around 150 involve nitrous oxide. For truly obscure psychoactives, the numbers are tiny: only fourteen 2C-T-21 reports, two reports of TMA-6 use, and no reports about pure psilocybin. Although we have over 800 reports involving psilocybin-containing mushrooms, there is very little depth in certain areas: only 12 involve family themes and only one includes the combination of fluoxetine (Prozac) and mushrooms. If the goal is collecting enough reports for each substance to allow for the meaningful study of sub-categories such as gender, dose, setting, and drug combinations, even 1,000 per substance begins to seem like a small number.

We occasionally have members and visitors tell us that too much of Erowid's resources are focused on Experience Reports. Because of the review system we have developed, the triaging and reviewing of reports is done largely by volunteers. Although the process does require some management, we view it as an extremely effective use of the crew's time.

No description of experience reports would be complete without mentioning the value of visionary art created to express or depict these experiences. For those more visually inclined, art is a more accessible representation of experience. Art can give a strong sense of emotion and mental state at a single glance. We continue to be excited about the Visionary Art Vaults, both for their pure aesthetic value as well as for their ability to represent psychoactive experiences very differently from the text-based Experience Vaults.

We often think about how valuable it would be if we had large numbers of experience reports from times past. Imagine if there were 100,000 experience reports from the 1960s. And how fantastic it would be to have those same people write follow-up reports forty years later.

The Experience Vaults are contributing to this historical record. We hope that in the future, Erowid's collection will act as a powerful tool to communicate how psychoactives have affected hundreds of thousands of individuals throughout their lives. In the meantime, it serves as a unique way for people to share their experiences and add to the historical record.

We have many ideas for the Experience Vaults, from streamlining submission and review systems, to improving search and display features, to facilitating related research.

In particular, the list includes adding a structured experience report form to collect a larger standardized set of data about each author and experience, a more formalized system for reporting drug interactions, and a visitor/member rating and comment system to allow readers to provide quality commentary about published experiences.

In most cases, it is no longer necessary for people who intend to try a psychoactive material to do so without knowing what their peers and elders have to say about the experience; from errors to triumphs, warnings to suggestions, the people who take psychoactive substances have a lot to share with those who come looking for the information.

For more on Experience Reports from Erowid Extracts Issue 10, see "Surfing the Matrix" and "An Insanely Large Suppository of Knowledge".