Selected projects
Assessing clinical trial interpretability
A research project funded by Heart & Stroke Canada to investigate possible interpretative biases in reporting clinical trials.
Overview
Past research has shown that medical doctors often exhibit innumeracy, statistical illiteracy, and poor probabilistic reasoning. These shortcomings have been repeatedly demonstrated in the diagnostic field, where test sensitivities and specificities are routinely confused with positive and negative predictive values and where uncertainty is often ignored, minimized, or misinterpreted.
Randomized controlled trials (RCTs) are often seen uncritically as above bias or misinterpretation. However, there are multiple reasons to believe that the same numerical and probabilistic issues also cloud trialists’ and clinicians’ interpretation of published RCT studies. First, the important frequentist statistical concepts of Type I and Type II errors are analogous to the sensitivity and specificity concepts in diagnostic testing that give clinicians problems. Second, the ubiquitous and widely misunderstood p values employed in standard frequentist analyses combined with null hypothesis significance testing undoubtedly compounds these cognitive difficulties. Thirdly, Bayesian reanalyzes of RCTs have sometimes reached contrary conclusions to the original conclusions and it is difficult to imagine how practicing clinicians could avoid misinterpretations that seem to, at least occasionally, plague even experienced trialists. Finally, pilot data from experienced academic clinicians that shows the immense variation in their probability assessments of RCTs results and their difficulty in updating their probabilities with the addition of new data.
This research has 3 aims.
Aim #1: a systematic review of all randomized controlled cardiovascular (CV) trials in the five leading general medical journals (New England Journal of Medicine, JAMA, BMJ, Lancet, Annals of Internal Medicine) which will be classified as “positive” or “negative” according to the authors’ own criteria and conclusions. This classification will then be compared with the interpretations provided by additional, but less frequently used, standard frequentist techniques of continuous p-valve graphs, binary Shannon information (surprisal or S values) and and (reverse) fragility indices (FI).
Aim #2: a Bayesian re-analysis of each identified trial from Aim #1 using vague priors (whereby the result is completely dominated by the current trial data) and informative priors (if the authors report previously completed RCTs in their original publication) to calculate: - the posterior probability of a negative or positive result (defined as < 5% or >95% probability, respectively- the probability range of a practically equivalent results (ROPE), determined by the context specific, user defined, minimally important clinical difference (MCID) and assess the concordance between inferences drawn between these analyses, the enhanced frequentist analyses from Aim # 1, and the original published interpretations.
Aim #3: Perform a Bayesian meta-regression to analyse author and study characteristics associated with the agreement/discordance between authors’ published conclusions and those from Aims 1 & 2.
Aortic stenosis
A CIHR funded project investigating the impact of patient, institutional, physician, and regional data on the trajectory of aortic stenosis (AS) treatment choices, outcomes, and cost effectiveness
Overview
Aortic stenosis (AS) is the most prevalent heart valve disorder in developed countries. Historically, surgical aortic valve replacement (SAVR) has been the standard treatment for symptomatic AS. Trans-catheter aortic valve replacement (TAVR) has now been demonstrated to significantly reduce mortality for inoperable AS patients compared to medical treatment, and to have comparable or improved outcomes compared across a spectrum of SAVR candidates. The most recent randomized trial has shown TAVR to be noninferior to surgery with respect to the composite end point of death or disabling stroke at 24 months follow-up in patients at low surgical risk. TAVR has, and will continue to, dramatically change the way patients with AS are managed and may be seen as “disruptive” technology.
It is important to have a clear understanding of the diagnostic, treatment and outcomes experienced by these patients. This is necessary to be sure the appropriate resources are available to meet these demands in a fair, transparent, and equitable manner.
Comparative effectiveness studies of dual antiplatelet therapy (DAPT)
Ticagrelor compared to clopidogrel in acute coronary syndromes (TC4) - the TC4 comparative effectiveness study
CIHR funded project (2019-2023) looking at the comparative effectiveness of ticagrelor versus clopidogrel in acute coronary syndromes.
Overview Cardiovascular diseases, especially acute coronary syndromes (ACS) are a leading cause of mortality and morbidity. Antiplatelet medications have a long history of improving ACS survival and for the prevention of a secondary cardiac event. Based on the consistency and strength of multiple randomized controlled trials (RCTs), the dual antiplatelet therapy (DAPT), clopidogrel and aspirin, following an ACS event, had long been considered the gold standard in the leading clinical guidelines. More recently, a new DAPT regimen, ticagrelor and aspirin, has become the recommended strategy. However this is based largely on one large RCT with a limited number of North American patients (who did not show any incremental benefit).
The Ticagrelor Compared to Clopidogrel in aCute coronary syndromes – the TC4 trial (NCT04057300) – is a pragmatic, open-label, active control, time clustered randomized clinical trial designed to assess the effectiveness and safety of ticagrelor versus clopidogrel as DAPT therapy in a ACS population undergoing percutaneous coronary intervention (PCI) in a single tertiary academic center in Montreal, Canada.
In addition to a standalone analysis, our main Bayesian analysis will incorporate the 1800 PLATO NA patients as our prior distribution, essentially providing a > 50% safety margin for our sample size calculations (see next section).
The complete funding protocol is found here.
This research program will also explore the comparative effectiveness of these different treatments by performing a
i) Bayesian network meta-analysis of all previous RCTs
ii) Pharmacoepidemiology study using Quebec administrative databases (manuscript in progress)
(Mostly) Clinical Epidemiology with R
This is some introductory teaching material related to clinical epidemiology using R.
Hopefully at some point this material will be updated, expanded and repackaged using Quarto