How to Read Cannabis Research: A Critical Appraisal Guide

The Evidence Hierarchy

Not all research is equal. The evidence hierarchy ranks study designs by their ability to establish causation and minimize bias. From strongest to weakest: (1) Systematic reviews and meta-analyses of multiple RCTs; (2) Individual randomized controlled trials (RCTs); (3) Non-randomized controlled trials; (4) Cohort studies; (5) Case-control studies; (6) Cross-sectional surveys; (7) Case reports and case series; (8) Expert opinion and preclinical studies.

Most cannabis research sits in the lower half of this hierarchy. The majority of studies are observational (no randomization, no control group) or preclinical (animal or cell culture). This is not a criticism of researchers — it reflects the regulatory barriers to cannabis research, particularly in the US where Schedule I status limits access to research-grade cannabis and funding.

What Makes an RCT Strong?

A randomized controlled trial (RCT) is the gold standard for establishing therapeutic efficacy. Key quality indicators include: (1) Adequate randomization — participants are randomly assigned to treatment or control, not self-selected; (2) Blinding — participants and researchers don't know who received active treatment (double-blind); (3) Adequate sample size — enough participants to detect a meaningful effect; (4) Pre-registration — the study protocol was registered before data collection began (prevents selective reporting); (5) Intention-to-treat analysis — all randomized participants are included in the analysis, even dropouts.

Cannabis RCTs face a unique challenge: blinding. THC's psychoactive effects make it difficult for participants to not know whether they received active treatment. This "unblinding" can inflate apparent treatment effects — participants who know they received cannabis may report greater improvement due to expectation. Studies that assess blinding success and account for it are more credible.

The Preclinical-to-Clinical Gap

A large proportion of cannabis research is preclinical — conducted in cell cultures or animal models. Preclinical studies are essential for understanding mechanisms and generating hypotheses, but they frequently fail to translate to human clinical benefit. The history of medicine is littered with compounds that cured cancer in mice but failed in humans.

For cannabis specifically, the antitumor evidence is a cautionary tale. Hundreds of preclinical studies show cannabinoids kill cancer cells in vitro and reduce tumor growth in animal models. This has generated enormous public interest in cannabis as a cancer treatment. But no clinical trial has demonstrated antitumor efficacy in humans. When you see a headline about cannabis "killing cancer cells," check whether the study was in a petri dish, a mouse, or a human — the answer dramatically changes its clinical relevance.

Common Methodological Problems in Cannabis Research

Several methodological issues recur in cannabis research: (1) Heterogeneous products — studies use different cannabis strains, THC:CBD ratios, doses, and routes of administration, making comparison difficult; (2) Self-reported outcomes — many studies rely on patient-reported symptom scores, which are subjective and susceptible to placebo effects; (3) Short follow-up — most trials last weeks to months; long-term safety and efficacy data are scarce; (4) Publication bias — positive results are more likely to be published than null results, inflating the apparent evidence base; (5) Conflict of interest — industry-funded studies are more likely to report positive results; (6) Observational confounding — cannabis users differ from non-users in many ways that independently affect health outcomes.

How to Evaluate a Cannabis Study

When evaluating a cannabis study, ask these questions: Was it randomized and controlled? Was it blinded, and was blinding success assessed? How many participants were included? Was the study pre-registered? Who funded it? What was the primary outcome, and was it clinically meaningful? Were adverse effects reported? Was the cannabis product standardized and characterized? How long was the follow-up? Were the results statistically significant AND clinically meaningful (a statistically significant result can be too small to matter in practice)?

For systematic reviews: How many studies were included? What was the overall quality of included studies? Was there significant heterogeneity (I² statistic)? Was publication bias assessed (funnel plot)? What was the GRADE evidence rating?

Understanding Effect Sizes

Statistical significance (p < 0.05) tells you that an effect is unlikely to be due to chance — it does not tell you whether the effect is clinically meaningful. Effect sizes matter. In pain research, a clinically meaningful reduction is generally ≥30% from baseline (the IMMPACT threshold). In many cannabis pain trials, the mean pain reduction with cannabinoids is 0.5–1.0 points on a 10-point scale — statistically significant but below the 30% threshold for clinical meaningfulness in many patients.

Number needed to treat (NNT) is a more intuitive measure: the number of patients who need to be treated for one to benefit. For cannabinoids in neuropathic pain, NNT estimates range from 5–10, meaning 5–10 patients must be treated for one to achieve ≥30% pain reduction. This is comparable to other neuropathic pain treatments but not dramatically superior.