---
title: "Explaining a model with SHAP"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Explaining a model with SHAP}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.width = 6,
  fig.height = 4,
  dpi = 110
)
```

## Overview

The SHAP layer works with a fitted `uniml_result` or directly with trained
mlr3 learners. It returns both the standardised model input and optional raw
feature values, so the same explanation can be plotted on the scale the model
saw or on a more interpretable display scale.

Model-agnostic explanations are computed with `kernelshap`; XGBoost and
LightGBM models use their native TreeSHAP implementation when the learner
package is installed. The example below is deliberately small so the vignette
builds quickly.

## Fit a workflow

```{r fit-workflow}
library(uniml)

data <- read.csv(
  system.file("extdata", "uniml-cvd-mortality.csv", package = "uniml")
)
set.seed(123)
data <- data[1:400, ]

spec <- uniml_spec(
  task_type = "binary",
  outcome = "died",
  positive = "yes",
  id = "id"
)
spec <- set_preprocess(
  spec,
  impute = "median_mode",
  scale = "zscore",
  encode = "onehot"
)
spec <- set_feature_selection(spec, methods = "univariate")
spec <- set_learners(spec, "cart", tune = FALSE)
spec <- set_evaluation(spec, ci = "none", calibration = "none")

split <- make_split(
  as_uniml_data(data, spec),
  prop = c(0.8, 0.2),
  stratify = TRUE,
  seed = 123
)
result <- run_workflow(
  spec,
  train = split$train$data,
  validation = split$validation$data
)
length(result$selection$selected_features)
```

## Compute SHAP values

`explain_shap()` explains the training features stored in the result by
default. `n_explain` caps the number of explained rows (they are subsampled
with `seed`) and `bg_n` controls the background sample used by Kernel SHAP, so
the run time stays predictable.

```{r explain}
explained <- explain_shap(
  result,
  model = "best",
  n_explain = 30,
  bg_n = 18,
  seed = 123
)

explained$shap
head(explained$shap$importance, 6)
```

A direct learner can be explained the same way:

```{r direct-learner}
learner <- result$models[[1L]]$learner
explanation <- explain_shap(
  learner,
  data = result$preprocess$train[, setdiff(
    names(result$preprocess$train), "died"
  )],
  n_explain = 20,
  bg_n = 14,
  seed = 123,
  positive = "yes"
)
explanation$models
```

## Importance, beeswarm, dependence, and waterfall

```{r importance, fig.cap = "Mean absolute SHAP value per feature."}
plot_shap_importance(explained)
```

```{r beeswarm, fig.cap = "Distribution and direction of individual contributions."}
plot_shap_beeswarm(explained, max_display = 8)
```

```{r dependence, fig.cap = "SHAP dependence for one feature."}
plot_shap_dependence(explained, feature = "age")
```

```{r waterfall, fig.cap = "Single-observation waterfall for the first explained row."}
plot_shap_waterfall(explained, row_id = 1)
```

The importance plot ranks features by mean absolute SHAP value, the beeswarm
shows the distribution and direction of individual contributions, and the
dependence plot shows the relationship between a feature and its SHAP value.
The waterfall decomposes one prediction into its feature contributions. All of
them are also available through `plot(result, type = ...)`.

## Clinical thresholds

For a continuous feature with enough observations, `extract_thresholds()` fits
a smoothed SHAP curve and reports zero crossings together with a paper-ready
summary table. It needs a reasonable number of rows per feature, so the chunk
is not evaluated in this small example.

```{r thresholds, eval = FALSE}
thresholds <- extract_thresholds(
  explained$shap,
  target = "age",
  min_n = 20,
  min_unique = 8
)

thresholds$zero_crossings
thresholds$paper_ready
```

The threshold search is descriptive. It should be reviewed against the
biological or clinical context and validated in an independent sample before
being used as a decision rule.

## References

Lundberg SM, Lee SI (2017). A unified approach to interpreting model
predictions. *Advances in Neural Information Processing Systems*, 30.

Lundberg SM, Erion G, Chen H, et al. (2020). From local explanations to global
understanding with explainable AI for trees. *Nature Machine Intelligence*,
2(1), 56-67.

Covert I, Lee SI (2021). Improving KernelSHAP: practical Shapley value
estimation using linear regression. *Proceedings of AISTATS*, PMLR 130,
3457-3465.
