class: center, middle, inverse, title-slide .title[ # Decision Curve ] .subtitle[ ## Step by Step 👣 ] .author[ ### Uriah Finkel ] .date[ ### 2022-05-01 ] --- <style> .reg{ font-size: 20px } .opac{ opacity: 0.3; font-size: 20px } </style> # Acknowledgement: I'd like to thank Dr. Andrew Vickers for his help with this presentation. You can visit his page on **[mskcc.org/profile/andrew-vickers](https://www.mskcc.org/profile/andrew-vickers)**. --- # Agenda - This lecture follows the article **[A simple, step-by-step guide to interpreting decision curve analysis](https://diagnprognres.biomedcentral.com/articles/10.1186/s41512-019-0064-7)**. - Introduction: Motivation behind Decision Curve Analysis and how to draw a Decision Curve. - How to interpret a Decision Curve: Step by Step guide. # Code for Decision Curve Analysis 💻 - Standard software is available to run decision curves in R, SAS and Stata on **[decisioncurveanalysis.org](https://www.mskcc.org/departments/epidemiology-biostatistics/biostatistics/decision-curve-analysis)**. Python should be with us soon! - All interactive plots in this presentation were created with **[rtichoke](https://uriahf.github.io/rtichoke/)** R package (I am the author 👋), you are also invited to explore **.rtichoke_blog[[rtichoke blog](https://rtichoke-blog.netlify.app/)]** for reproducible examples and some theory. - For ggplot2 outputs **[dcurves](https://www.danieldsjoberg.com/dcurves)** R package is available on CRAN. --- class: inverse center middle # Introduction --- class: center middle #### Which Model is Better? 🤔 #### Select patients for biopsy amongst men with elevated PSA .left-column[ ## ROC ] .right-column[
] --- class: center middle #### Which Model is Better? 🤔 #### Select patients for biopsy amongst men with elevated PSA .left-column[ ## ROC ## Calibration Curve ] .right-column[
] --- class: center middle #### Which Model is Better? 🤔 #### Select patients for biopsy amongst men with elevated PSA .left-column[ ## ROC ## Calibration Curve ## Decision Curve ] .right-column[
] --- ### Traditional Statistical Metric: - Discrimination: How well a prediction model can discriminate those with the outcome from those without the outcome. Examples: ROC Curve, AUROC, Sensitivity, Specificity, NPV, PPV, Lift and more... - Calibration: Agreement between observed outcomes and predictions. Examples: Calibration Curve, Calibration in the large, Calibration in the small, Brier Score... **.red[Problem!]** These metrics are not directly informative to clinical value, nor to full decision analytic approaches. **.green[Solution! Decision Curve Analysis]**: - Decision Curve Analysis calculates a clinical "Net Benefit" for one or more prediction models or diagnostic tests in comparison to default strategies of treating all or no patients. --- ### How to draw a Decision Curve?: .pull-left[ - On the X axis: Probability Threshold - On the Y axis: `$$\begin{aligned} \scriptsize{ \text{Net Benefit} = \frac{\text{TP}}{\text{TP + FP + TN + FN}} - \frac{\text{FP}}{\text{TP + FP + TN + FN}} * {\frac{{p_{t}}}{{1 - p_{t}}}}} \end{aligned}$$` - Treat None strategy as a reference line: `$$\begin{aligned} \small{ \text{Net Benefit Treat None} = 0} \end{aligned}$$` - Treat All strategy as a reference line: `$$\begin{aligned} \scriptsize{ \text{Net Benefit Treat All} = {\text{Prevalence}} - {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}}} \end{aligned}$$` ] .pull-right[
] --- class: inverse center middle # Step 1 # 👣: Benefit is **.green[Good]** --- ### Step 1 👣: Benefit is **.green[Good]** .center[ #🙂 # **.green[Green]** = **.green[Good]** ] .center[ #🙁 # **.red[Red]** = **.red[Bad]** ] --- ### Step 1 👣: Benefit is **.green[Good]** .center[ <img src="Decision_Curve_Step_by_Step_files/Maccabi_Haifa_FC_logo.svg" width="100" height="100" /> # **.green[Green]** = **.green[Good]** ] .center[ <img src="Decision_Curve_Step_by_Step_files/Hapoel_Haifa_Football_Club_Logo.png" width="90" height="90" /> # **.red[Red]** = **.red[Bad]** ] <script type="text/x-mathjax-config"> MathJax.Hub.Config({ TeX: { Macros: { red: ["{\\color{red}{#1}}", 1], green: ["{\\color{green}{#1}}", 1], }, loader: {load: ['[tex]/color']}, tex: {packages: {'[+]': ['color']}} } }); </script> --- ### Step 1 👣: Benefit is **.green[Good]** .center[ ## Infected and Predicted as Infected - **.green[Good]** #💊<br>🤢 <br>
] --- ### Step 1 👣: Benefit is **.green[Good]** .center[ ## Not-Infected and Predicted as Infected - .red[BAD] #💊<br>🤨 <br>
] --- ### Step 1 👣: Benefit is **.green[Good]** .center[ ## Infected and Predicted as Not-Infected - .red[BAD] # <br>🤢 <br>
] --- ### Step 1 👣: Benefit is **.green[Good]** .center[ ## Not-Infected and Predicted as Not-Infected - **.green[Good]** # <br>🤨 <br>
] --- ### Step 1 👣: Benefit is **.green[Good]** .center[ ## Predicted as Infected - Not Good nor Bad #💊<br>😷 <br>
] --- ### Step 1 👣: Benefit is **.green[Good]** .center[ ## Predicted as Not-Infected - Not Good nor Bad # <br>😷 <br>
] --- ### Step 1 👣: Benefit is **.green[Good]** `$$\begin{aligned} \text{Net Benefit} = \frac{\text{TP}}{\text{TP + FP + TN + FN}} - \frac{\text{FP}}{\text{TP + FP + TN + FN}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}$$` --- ### Step 1 👣: Benefit is **.green[Good]** `$$\begin{aligned} \text{Net Benefit} = \frac{\text{TP}}{\text{N}} - \frac{\text{FP}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}$$` -- `$$\begin{aligned} \text{Sensitivity } = \frac{\text{TP}}{\text{TP + FN}} \end{aligned}$$` -- `$$\begin{aligned} \text{Specificity} = \frac{\text{TN}}{\text{TN + FP}} \end{aligned}$$` -- `$$\begin{aligned} {\text{Prevalence}} = \frac{\text{TP + FN}}{\text{TP + FP + TN + FN}} \end{aligned}$$` --- ### Step 1 👣: Benefit is **.green[Good]** `$$\begin{aligned} \text{Net Benefit} = \frac{\text{TP}}{\text{N}} - \frac{\text{FP}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}$$` `$$\begin{aligned} \text{Sensitivity } = \frac{\text{TP}}{\text{Real Positives}} \end{aligned}$$` `$$\begin{aligned} \text{Specificity} = \frac{\text{TN}}{\text{Real Negatives}} \end{aligned}$$` `$$\begin{aligned} {\text{Prevalence}} = \frac{\text{Real Positive}}{\text{N}} \end{aligned}$$` --- ### Step 1 👣: Benefit is **.green[Good]** `$$\begin{aligned} \text{Net Benefit} = \frac{\text{TP}}{\text{N}} - \frac{\text{FP}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}$$` `$$\begin{aligned} \text{Sensitivity } = \frac{\text{TP}}{\text{Real Positives}} \end{aligned}$$` `$$\begin{aligned} \text{1 - Specificity} = \frac{\text{FP}}{\text{Real Negatives}} \end{aligned}$$` `$$\begin{aligned} {\text{Prevalence}} = \frac{\text{Real Positives}}{\text{N}} \end{aligned}$$` -- `$$\begin{aligned} {\text{1 - Prevalence}} = \frac{\text{Real Negatives}}{\text{N}} \end{aligned}$$` -- `$$\begin{aligned} \text{Net Benefit} = {\text{Sensitivity}} * {\text{Prevalence}} - {\text{(1 - Specificity)}} * {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}$$` --- ### Step 1 👣: Benefit is **.green[Good]** `$$\begin{aligned} \green{\bf{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}$$` `$$\begin{aligned} \green{\bf{Sensitivity}} = \frac{\green{\bf{TP}}}{\text{Real Positives}} \end{aligned}$$` `$$\begin{aligned} \red{\bf{1 - Specificity}} = \frac{\bf{\red{FP}}}{\text{Real Negatives}} \end{aligned}$$` `$$\begin{aligned} {\text{Prevalence}} = \frac{\text{Real Positives}}{\text{N}} \end{aligned}$$` `$$\begin{aligned} {\text{1 - Prevalence}} = \frac{\text{Real Negatives}}{\text{N}} \end{aligned}$$` `$$\begin{aligned} \green{\bf{Net Benefit}} = {\green{\bf{Sensitivity}}} * {\text{Prevalence}} - {\red{\bf{(1 - Specificity)}}} * {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}$$` --- ### Step 1 👣: Benefit is **.green[Good]** `$$\begin{aligned} \green{\bf{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}$$` `$$\begin{aligned} \green{\bf{Sensitivity}} = \frac{\green{\bf{TP}}}{\text{Real Positives}} \end{aligned}$$` `$$\begin{aligned} \red{\bf{1 - Specificity}} = \frac{\bf{\red{FP}}}{\text{Real Negatives}} \end{aligned}$$` `$$\begin{aligned} {\text{Prevalence}} = \frac{\text{Real Positives}}{\text{N}} \end{aligned}$$` `$$\begin{aligned} {\text{1 - Prevalence}} = \frac{\text{Real Negatives}}{\text{N}} \end{aligned}$$` `$$\begin{aligned} \green{\bf{Net Benefit}} = {\green{\bf{Sensitivity}}} * {\text{Prevalence}} - {\red{\bf{(1 - Specificity)}}} * {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}$$` `$$\begin{aligned} \text{Net Benefit Treat All} = {\green{\bf{1}}} * {\text{Prevalence}} - {\red{\bf{(1 - 0)}}} * {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}$$` --- ### Step 1 👣: Benefit is **.green[Good]** `$$\begin{aligned} \green{\bf{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}$$` `$$\begin{aligned} \green{\bf{Sensitivity}} = \frac{\green{\bf{TP}}}{\text{Real Positives}} \end{aligned}$$` `$$\begin{aligned} \red{\bf{1 - Specificity}} = \frac{\bf{\red{FP}}}{\text{Real Negatives}} \end{aligned}$$` `$$\begin{aligned} {\text{Prevalence}} = \frac{\text{Real Positives}}{\text{N}} \end{aligned}$$` `$$\begin{aligned} {\text{1 - Prevalence}} = \frac{\text{Real Negatives}}{\text{N}} \end{aligned}$$` `$$\begin{aligned} \green{\bf{Net Benefit}} = {\green{\bf{Sensitivity}}} * {\text{Prevalence}} - {\red{\bf{(1 - Specificity)}}} * {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}$$` `$$\begin{aligned} \text{Net Benefit Treat All} = {\text{Prevalence}} - {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}$$` --- ### Step 1 👣: Benefit is **.green[Good]** `$$\begin{aligned} \green{\bf{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}$$` `$$\begin{aligned} \green{\bf{Sensitivity}} = \frac{\green{\bf{TP}}}{\text{Real Positives}} \end{aligned}$$` `$$\begin{aligned} \red{\bf{1 - Specificity}} = \frac{\bf{\red{FP}}}{\text{Real Negatives}} \end{aligned}$$` `$$\begin{aligned} {\text{Prevalence}} = \frac{\text{Real Positives}}{\text{N}} \end{aligned}$$` `$$\begin{aligned} {\text{1 - Prevalence}} = \frac{\text{Real Negatives}}{\text{N}} \end{aligned}$$` `$$\begin{aligned} \green{\bf{Net Benefit}} = {\green{\bf{Sensitivity}}} * {\text{Prevalence}} - {\red{\bf{(1 - Specificity)}}} * {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}$$` `$$\begin{aligned} \text{Net Benefit Treat None} = {\green{\bf{0}}} * {\text{Prevalence}} - {\red{\bf{(1 - 1)}}} * {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}$$` --- ### Step 1 👣: Benefit is **.green[Good]** `$$\begin{aligned} \green{\bf{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}$$` `$$\begin{aligned} \green{\bf{Sensitivity}} = \frac{\green{\bf{TP}}}{\text{Real Positives}} \end{aligned}$$` `$$\begin{aligned} \red{\bf{1 - Specificity}} = \frac{\bf{\red{FP}}}{\text{Real Negatives}} \end{aligned}$$` `$$\begin{aligned} {\text{Prevalence}} = \frac{\text{Real Positives}}{\text{N}} \end{aligned}$$` `$$\begin{aligned} {\text{1 - Prevalence}} = \frac{\text{Real Negatives}}{\text{N}} \end{aligned}$$` `$$\begin{aligned} \green{\bf{Net Benefit}} = {\green{\bf{Sensitivity}}} * {\text{Prevalence}} - {\red{\bf{(1 - Specificity)}}} * {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}$$` `$$\begin{aligned} \text{Net Benefit Treat None} = 0 \end{aligned}$$` --- class: inverse center middle # Step 2 👣: # Preference Differ --- ### Step 2 👣: Preference Differ .pull-left[ Low Probability Threshold means that I'm worried about the **outcome**: - I'm worried about **Prostate Cancer** 🦀 - I'm worried about **Heart Disease** 💔 - I'm worried about **Infection** 🤢 ] .pull-right[
] --- ### Step 2 👣: Preference Differ .pull-left[ High Probability Threshold means that I'm worried about the **Intervention**: - I'm worried about **Biopsy** 💉 - I'm worried about **Statins** 💊 - I'm worried about **Antibiotics** 💊 ] .pull-right[
] --- class: inverse center middle # Step 3 👣: # Unit of Preference = Threshold Probability --- ### Step 3 👣: Unit of Preference = Threshold Probability .pull-left[ .red[REMEMBER]: almost always (specially in Health Care) - Sensitivity does not have the same importance as Specificity and having 1 TP does not have the same clinical utility as having 1 FP. A good example for a bad practice in evaluating performance of prediction model is the Youden's J statistics: `$${\displaystyle J={\frac {\text{TP}}{{\text{TP}}+{\text{FN}}}}+{\frac {\text{TN}}{{\text{TN}}+{\text{FP}}}}-1}$$` Maximizing Youden's J statistic might lead to a Probability Threshold that will do more harm than good, even if the prediction model is accurate! ] .pull-right[
] --- ### Step 3 👣: Unit of Preference = Threshold Probability .pull-left[ I wouldn’t give more than 4 antibiotics in order to help 1 infected patient. If a patient’s risk was above 20% I will give him antibiotics, otherwise I won't. The risk of 20% is an odds of 1:4, so in using a threshold probability of 20%, the doctor is telling us “missing an infected patiant is 4 times worse than giving antibiotics to a healthy patient." `$$p_t = \frac{1}{1 + 4} = 0.2$$` `$$\frac{p_t}{1 - p_t} = \frac{0.2}{1 - 0.2} = \frac{1}{4}$$` ] .pull-right[
] --- ### Step 3 👣: Unit of Preference = Threshold Probability .pull-left[ I will be indifferent 😐 for having 1 **.green[TP]** for 4 **.red[FP]** `$$p_t = \frac{1}{1 + 4} = 0.2$$` `$$\frac{p_t}{1 - p_t} = \frac{0.2}{1 - 0.2} = \frac{1}{4}$$` In that case the Net Benefit of using the model will be: `$$\begin{aligned}[t] {\text{Net Benefit}} &= {\frac{\text{1}}{\text{5}} - \frac{\text{4}}{\text{5}} * {\frac{1}{4}} = 0} \end{aligned}$$` ] .pull-right[ ![](Decision_Curve_Step_by_Step_files/indifferent.svg) ] --- ### Step 3 👣: Unit of Preference = Threshold Probability .pull-left[ I will be sad 🙁 for having 1 **.green[TP]** for 5 **.red[FP]** `$$p_t = \frac{1}{1 + 4} = 0.2$$` `$$\frac{p_t}{1 - p_t} = \frac{0.2}{1 - 0.2} = \frac{1}{4}$$` In that case the Net Benefit of using the model will be: `$$\begin{aligned}[t] {\text{Net Benefit}} &= {\frac{\text{1}}{\text{6}} - \frac{\text{5}}{\text{6}} * {\frac{1}{4}} = -0.04166'} \end{aligned}$$` ] .pull-right[ ![](Decision_Curve_Step_by_Step_files/sad.svg) ] --- ### Step 3 👣: Unit of Preference = Threshold Probability .pull-left[ I will be happy 🙂 for having 1 **.green[TP]** for 3 **.red[FP]** `$$p_t = \frac{1}{1 + 4} = 0.2$$` `$$\frac{p_t}{1 - p_t} = \frac{0.2}{1 - 0.2} = \frac{1}{4}$$` In that case the Net Benefit of using the model will be: `$$\begin{aligned}[t] {\text{Net Benefit}} &= {\frac{\text{1}}{\text{4}} - \frac{\text{3}}{\text{4}} * {\frac{1}{4}} = 0.0625} \end{aligned}$$` ] .pull-right[ ![](Decision_Curve_Step_by_Step_files/happy.svg) ] --- class: inverse center middle # Step 4 👣: # Benefit is actually Net Benefit --- background-image: url("Decision_Curve_Step_by_Step_files/American_wines_can_into_chateau.png") background-size: contain <!-- ![](Decision_Curve_Step_by_Step_files/American_wines_can_into_chateau.png) --> --- ### Step 4 👣: Benefit is actually Net Benefit Let's think in terms of money 💸 A wine importer buys €1m of wine from France and sells it in the USA for $1.5m 🍷 -- Net Benefit = **.green[Income]** - **.red[Expenditure]** Net Benefit = **.green[1.5m$]** - **.red[1m€]** = ? 🤔 -- Let's say that 1€ is worth 1.25$ Exchange-Rate = 1.25 ($ / €) 1m€ = 1m * 1.25$ -- Net Benefit = **.green[1.5m$]** - **.red[1.25m$]** = **.green[250,000$]** Which is the equivalent of being given 250,000$: Net Benefit = **.green[250,000$]** - **.red[0$]** = **.green[250,000$]** --- ### Step 4 👣: Benefit is actually Net Benefit .pull-left[
.reg[ 💊💊 <br> 🤢🤢😷😷😷😷😷😷😷😷 ] ] .pull-right[
.reg[ 💊💊💊💊💊💊💊 <br> 🤢🤢🤢🤨🤨🤨🤨😷😷😷 ] ] <br> <br> <br> <br> `$$\begin{equation*} \begin{aligned}[t] \small{\text{Net Benefit}} &= \small{\frac{\text{TP}}{\text{N}} - \frac{\text{FP}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}}} \end{aligned} \qquad% adjust to suit \begin{aligned}[t] \small{\text{Net Benefit}} &= \small{\frac{\text{TP}}{\text{N}} - \frac{\text{FP}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}}} \end{aligned} \end{equation*}$$` --- ### Step 4 👣: Benefit is actually Net Benefit .pull-left[
.reg[ 💊💊 <br> 🤢🤢😷😷😷😷😷😷😷😷 ] ] .pull-right[
.reg[ 💊💊💊💊💊💊💊 <br> 🤢🤢🤢🤨🤨🤨🤨😷😷😷 ] ] <br> <br> <br> <br> `$$\begin{equation*} \begin{aligned}[t] \small{\text{Net Benefit}} &= \small{\frac{\text{TP}}{\text{10}} - \frac{\text{FP}}{\text{10}} * {\frac{1}{4}}} \end{aligned} \qquad% adjust to suit \begin{aligned}[t] \small{\text{Net Benefit}} &= \small{\frac{\text{TP}}{\text{10}} - \frac{\text{FP}}{\text{10}} * {\frac{1}{4}}} \end{aligned} \end{equation*}$$` --- ### Step 4 👣: Benefit is actually Net Benefit .pull-left[
.reg[ 💊💊 <br> 🤢🤢] .opac[😷😷😷😷😷😷😷😷 ] ] .pull-right[
.reg[ 💊💊💊 <br> 🤢🤢🤢 ].opac[ 💊💊💊💊 <br> 🤨🤨🤨🤨😷😷😷 ] ] `$$\begin{equation*} \begin{aligned}[t] \small{\text{Net Benefit}} &= \small{\frac{\green{\bf{2}}}{\text{10}} - \frac{\text{FP}}{\text{10}} * {\frac{1}{4}}} \end{aligned} \qquad% adjust to suit \begin{aligned}[t] \small{\text{Net Benefit}} &= \small{\frac{\green{\bf{3}}}{\text{10}} - \frac{\text{FP}}{\text{10}} * {\frac{1}{4}}} \end{aligned} \end{equation*}$$` --- ### Step 4 👣: Benefit is actually Net Benefit .pull-left[
.opac[ 💊💊 <br> 🤢🤢😷😷😷😷😷😷😷😷 ] ] .pull-right[
.opac[ 💊💊💊 <br> 🤢🤢🤢 ].reg[ 💊💊💊💊 <br> 🤨🤨🤨🤨 ].opac[ 💊💊💊 <br> 😷😷😷 ] ] `$$\begin{equation*} \begin{aligned}[t] \small{\text{Net Benefit}} &= \small{\frac{\text{2}}{\text{10}} - \frac{\red{\bf{0}}}{\text{10}} * {\frac{1}{4}}} \end{aligned} \qquad% adjust to suit \begin{aligned}[t] \small{\text{Net Benefit}} &= \small{\frac{\text{3}}{\text{10}} - \frac{\red{\bf{4}}}{\text{10}} * {\frac{1}{4}}} \end{aligned} \end{equation*}$$` --- ### Step 4 👣: Benefit is actually Net Benefit .pull-left[
.reg[ 💊💊 <br> 🤢🤢😷😷😷😷😷😷😷😷 ] ] .pull-right[
.reg[ 💊💊💊💊💊💊💊 <br> 🤢🤢🤢🤨🤨🤨🤨😷😷😷 ] ] `$$\begin{equation*} \begin{aligned}[t] \small{\text{Net Benefit}} &= \small{\frac{\text{2}}{\text{10}} - \frac{\text{0}}{\text{10}} * {\frac{1}{4}} = \frac{\text{2}}{\text{10}}} \end{aligned} \qquad% adjust to suit \begin{aligned}[t] \small{\text{Net Benefit}} &= \small{\frac{\text{3}}{\text{10}} - \frac{\text{4}}{\text{10}} * {\frac{1}{4}} = \frac{\text{2}}{\text{10}}} \end{aligned} \end{equation*}$$` --- class: inverse center middle # Step 5 👣: # Net benefit can also be expressed as Interventions Avoided --- ### Step 5 👣: Net benefit can also be expressed as Interventions Avoided .pull-left[
.reg[ 💊💊💊💊💊💊💊 <br> 🤢🤢🤢🤨🤨🤨🤨😷😷😷 ] ] .pull-right[
.reg[ 💊💊💊💊💊💊💊💊💊💊 <br> 🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨 ] ] <br> <br> <br> <br> `$$\begin{equation*} \begin{aligned}[t] \small{\text{Net Benefit}} &= \small{\frac{\text{TP}}{\text{N}} - \frac{\text{FP}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}}} \end{aligned} \qquad% adjust to suit \begin{aligned}[t] \small{\text{Net Benefit All}} &= \small{\frac{\text{TP}}{\text{N}} - \frac{\text{FP}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}}} \end{aligned} \end{equation*}$$` --- ### Step 5 👣: Net benefit can also be expressed as Interventions Avoided .pull-left[
.reg[ 💊💊💊💊💊💊💊 <br> 🤢🤢🤢🤨🤨🤨🤨😷😷😷 ] ] .pull-right[
.reg[ 💊💊💊💊💊💊💊💊💊💊 <br> 🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨 ] ] <br> <br> <br> <br> `$$\begin{equation*} \begin{aligned}[t] \small{\text{Net Benefit}} &= \small{\frac{\text{TP}}{\text{10}} - \frac{\text{FP}}{\text{10}} * {\frac{1}{4}}} \end{aligned} \qquad% adjust to suit \begin{aligned}[t] \small{\text{Net Benefit All}} &= \small{\frac{\text{TP}}{\text{10}} - \frac{\text{FP}}{\text{10}} * {\frac{1}{4}}} \end{aligned} \end{equation*}$$` --- ### Step 5 👣: Net benefit can also be expressed as Interventions Avoided .pull-left[
.reg[ 💊💊💊 <br> 🤢🤢🤢 ].opac[ 💊💊💊💊 <br> 🤨🤨🤨🤨😷😷😷 ] ] .pull-right[
.reg[ 💊💊💊 <br> 🤢🤢🤢 ].opac[ 💊💊💊💊💊💊💊 <br> 🤨🤨🤨🤨🤨🤨🤨 ] ] `$$\begin{equation*} \begin{aligned}[t] \small{\text{Net Benefit}} &= \small{\frac{\green{\bf{3}}}{\text{10}} - \frac{\text{FP}}{\text{10}} * {\frac{1}{4}}} \end{aligned} \qquad% adjust to suit \begin{aligned}[t] \small{\text{Net Benefit All}} &= \small{\frac{\green{\bf{3}}}{\text{10}} - \frac{\text{FP}}{\text{10}} * {\frac{1}{4}}} \end{aligned} \end{equation*}$$` --- ### Step 5 👣: Net benefit can also be expressed as Interventions Avoided .pull-left[
.opac[ 💊💊💊 <br> 🤢🤢🤢 ].reg[ 💊💊💊💊 <br> 🤨🤨🤨🤨 ].opac[ 💊💊💊 <br> 😷😷😷 ] ] .pull-right[
.opac[ 💊💊💊 <br> 🤢🤢🤢 ].reg[ 💊💊💊💊💊💊💊 <br> 🤨🤨🤨🤨🤨🤨🤨 ] ] `$$\begin{equation*} \begin{aligned}[t] \small{\text{Net Benefit}} &= \small{\frac{\text{3}}{\text{10}} - \frac{\red{\bf{4}}}{\text{10}} * {\frac{1}{4}}} \end{aligned} \qquad% adjust to suit \begin{aligned}[t] \small{\text{Net Benefit All}} &= \small{\frac{\text{3}}{\text{10}} - \frac{\red{\bf{7}}}{\text{10}} * {\frac{1}{4}}} \end{aligned} \end{equation*}$$` --- ### Step 5 👣: Net benefit can also be expressed as Interventions Avoided .pull-left[
.opac[ 💊💊💊💊💊💊💊 <br> 🤢🤢🤢🤨🤨🤨🤨 ].reg[ <br> 😷😷😷 ] ] .pull-right[
.opac[ 💊💊💊💊💊💊💊💊💊💊 <br> 🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨 ] ] `$$\begin{equation*} \begin{aligned}[t] \small{\text{Net Benefit}} &= \small{\frac{\text{3}}{\text{10}} - \frac{\text{4}}{\text{10}} * {\frac{1}{4}}} \end{aligned} \qquad% adjust to suit \begin{aligned}[t] \small{\text{Net Benefit All}} &= \small{\frac{\text{3}}{\text{10}} - \frac{\text{7}}{\text{10}} * {\frac{1}{4}}} \end{aligned} \end{equation*}$$` `$$\text{Interventions Avoided (per 100 cases)} = \frac{3}{10} * 100 = 30$$` --- ### Step 5 👣: Net benefit can also be expressed as Interventions Avoided .pull-left[
.opac[ 💊💊💊💊💊💊💊 <br> 🤢🤢🤢🤨🤨🤨🤨 ].reg[ <br> 😷😷😷 ] ] .pull-right[
.opac[ 💊💊💊💊💊💊💊💊💊💊 <br> 🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨 ] ] `$$\begin{equation*} \begin{aligned}[t] \small{\text{Net Benefit}} &= \small{\frac{\text{3}}{\text{10}} - \frac{\text{4}}{\text{10}} * {\frac{1}{4}}} \end{aligned} \qquad% adjust to suit \begin{aligned}[t] \small{\text{Net Benefit All}} &= \small{\frac{\text{3}}{\text{10}} - \frac{\text{7}}{\text{10}} * {\frac{1}{4}}} \end{aligned} \end{equation*}$$` `$$\text{(Net Benefit - Net Benefit All)} * \frac{1-p_t}{p_t} = \frac{3}{10} * \frac{1}{4} * 4 * 100 = 30$$` `$$\text{Interventions Avoided (per 100 cases)} = \frac{3}{10} * 100 = 30$$` --- ### Step 5 👣: Net benefit can also be expressed as Interventions Avoided .pull-left[
.opac[ 💊💊💊💊💊💊💊 <br> 🤢🤢🤢🤨🤨🤨🤨 ].reg[ <br> 🤨🤨🤨 ] ] .pull-right[
.opac[ 💊💊💊💊💊💊💊💊💊💊 <br> 🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨 ] ] `$$\text{Interventions Avoided (per 100 cases)} = \frac{3}{10} * 100 = 30$$` `$$\text{Interventions Avoided (per 100 cases)} = (\frac{\text{TN}}{\text{N}} -\frac{\text{FN}}{\text{N}} * \frac{1 - p_t}{p_t}) * 100 = (\frac{\text{3}}{\text{10}} -\frac{\text{0}}{\text{10}} * 4) * 100 = 30$$` --- ### Step 5 👣: Net benefit can also be expressed as Interventions Avoided .pull-left[ For the y axis instead of conventional Net Benefit you can use Interventions avoided. ] .pull-right[
] --- ### Step 5 👣: Net benefit can also be expressed as Interventions Avoided .pull-left[ For the y axis instead of conventional Net Benefit you can use Interventions avoided. Doing so does not change any conclusions as to which model or test has the highest net benefit. ] .pull-right[
] --- class: inverse center middle # Thank You # 👋