# Pooled inference and attribute-disclosure diagnostics

## Why analyse multiple synthetic datasets

`synth()` regenerates every column from a model fitted on the real data, using a freshly drawn skeleton — so its output is fully synthetic in the Reiter (2003) sense. Run the synthesiser twice and the analysis estimates can move. Analysing only one synthetic dataset does not quantify that between-synthesis variation; depending on the estimand, synthesis model, sample size, and analysis, its usual standard errors can therefore be misleading.

This is why `synth()` can emit `m > 1` datasets. Fit the analysis on each and combine the results using a published fully-synthetic rule. `flexsynth` provides two such rules; their large-sample calibration still relies on the assumptions of the synthesis and analysis models.

    n <- 800
    real <- data.frame(
      id  = seq_len(n),
      age = round(rnorm(n, 62, 11)),
      sex = sample(c("F", "M"), n, replace = TRUE, prob = c(0.45, 0.55))
    )
    real$sbp <- round(0.6 * real$age + ifelse(real$sex == "M", 5, 0) +
                        rnorm(n, 90, 10))

    res <- synth(real, structure = ~ id, m = 10, seed = 1)
    res

    #> <synth_result>
    #>   track        : A (high-utility; NOT differentially private)
    #>   datasets (m) : 10
    #>   rows each    : 800  (input: 800 )
    #>   synthesised  : age, sex, sbp
    #>   unit-level   : age, sex, sbp (once per unit)
    #>   method       : cart
    #>
    #> Get the data with as.data.frame(x) or x$syn[[i]]

We asked for `m = 10` synthetic datasets. Increasing `m` reduces Monte Carlo variation in the pooled result; `m >= 5` is a practical starting point, not a universal adequacy threshold.

## `synth_glm()` — the common case

For a linear or generalised-linear model, `synth_glm()` fits it on every synthetic dataset and pools the result in one call.

    pooled <- synth_glm(res, sbp ~ age + sex)
    pooled

    #> <flexsynth_pool> pooled inference from 10 synthetic dataset(s)
    #>   rule: synthpop   population inference   (n = 800, k = 800)
    #>         term estimate std.error statistic   p.value conf.low conf.high
    #>  (Intercept) 87.20280   2.31110    37.732 1.47e-311 82.67313  91.73247
    #>          age  0.64033   0.03574    17.915  9.03e-72  0.57027   0.71038
    #>         sexM  5.74845   0.83329     6.898  5.26e-12  4.11523   7.38168

Compare that with the model fitted on the real data — the estimand we are trying to recover:

    summary(lm(sbp ~ age + sex, data = real))$coefficients[, 1:2]

| Estimate | Std. Error
--- | --- | ---
(Intercept) | 86.199 | 2.128
age | 0.655 | 0.033
sexM | 5.694 | 0.748

In this simulation, the pooled estimates should be close to the generating coefficients (an `age` slope of 0.6 and a male offset of 5). The reported standard errors apply the selected combining rule to the within- and between-synthesis quantities; they are not a guarantee of calibrated coverage for every dataset or model.

By default the standard error targets the population the real data was drawn from. If instead you want the standard error an analyst of the original sample would have reported, set `population_inference = FALSE`.

## `pool_synth()` — any estimator

`synth_glm()` is a thin wrapper. For anything else — a quantile, a survival model, a hand-rolled statistic — pass `pool_synth()` a function that runs on one synthetic `data.frame` and returns either a fitted model (with `coef()` / `vcov()` methods) or an explicit `list(estimate =, variance =)`.

    # A proportion and its variance, supplied directly.
    pool_synth(res, function(d) {
      p <- mean(d$sex == "M")
      list(estimate = c(prop_male = p),
           variance = c(prop_male = p * (1 - p) / nrow(d)))
    })

    #> <flexsynth_pool> pooled inference from 10 synthetic dataset(s)
    #>   rule: synthpop   population inference   (n = 800, k = 800)
    #>       term estimate std.error statistic   p.value conf.low conf.high
    #>  prop_male  0.55912    0.0184    30.391 7.26e-203  0.52307   0.59518

Two combining rules are available via `rule =`: the default `"synthpop"` rule matching Raab, Nowok & Dibben (2016), whose implemented variance estimate is non-negative, and the `"reiter"` fully-synthetic rule from Reiter (2003), which needs `m >= 2` and can produce a negative variance estimate (reported as `NA`). Choose the rule to match the release and inferential target rather than by the size of its standard error.

## `compare_estimates()` — analysis-specific utility

`diagnose()` (see `vignette("getting-started")`) asks a general question: can a model tell real from synthetic records? `compare_estimates()` asks the question an analyst actually cares about: does my regression come out the same? It fits the analysis on both sides — pooling the synthetic side automatically — and reports, per term, the confidence-interval overlap (Karr et al. 2006; 1 means identical intervals, 0 means they just touch, negative means disjoint) and the standardised difference of the estimates.

    compare_estimates(real, res, function(d) lm(sbp ~ age + sex, data = d))

    #> <flexsynth_utility> real-vs-synthetic estimate comparison (synthetic pooled)
    #>   CI overlap (1 = identical, <0 = disjoint) at 95%; std_diff = |est_real - est_syn| / se_real
    #>         term est_real est_syn overlap std_diff
    #>  (Intercept)  86.1992 87.2028   0.886    0.472
    #>          age   0.6549  0.6403   0.893    0.443
    #>         sexM   5.6942  5.7485   0.949    0.073
    #>   mean overlap: 0.909

Larger interval overlap and smaller standardised differences indicate better agreement for this chosen analysis. They are utility diagnostics, not evidence that every analysis will agree or that the synthetic data is safe to release.

## Attribute disclosure — TCAP

Reproducing an analysis well is a utility win, but the same key-to-target structure an analyst exploits is what an attacker exploits. `disclosure_risk()` gains a `target =` argument for exactly this: the Target Correct Attribution Probability (TCAP; Taub, Elliot et al.). An attacker who knows a real record’s quasi-identifier keys looks up the synthetic records sharing those keys and reads off the conditional distribution of the sensitive `target`. TCAP averages the synthetic probability of the true value over real records whose keys occur in the synthetic data. The reported `baseline` evaluates a marginal-only attacker on those same covered records, so `lift` is the excess attribution the keys buy. Inspect `coverage` as well: TCAP can otherwise look favourable simply because difficult records have no synthetic key match. The `baseline_unconditional` value provides context across all scored real records.

TCAP is meant for categorical keys and target, so we use a small categorical dataset where a sensitive status depends on the quasi-identifiers.

    m <- 800
    clinic <- data.frame(
      region = sample(c("N", "S", "E", "W"), m, replace = TRUE),
      agecat = sample(c("40s", "50s", "60s", "70s"), m, replace = TRUE)
    )
    # hypertension risk rises steeply with age, with a small regional bump.
    prob_htn <- c("40s" = 0.12, "50s" = 0.40, "60s" = 0.75, "70s" = 0.93)
    reg_bump <- c("N" = 0, "S" = 0, "E" = 0.03, "W" = 0.05)
    p <- pmin(0.98, prob_htn[clinic$agecat] + reg_bump[clinic$region])
    clinic$hypertension <- ifelse(runif(m) < p, "yes", "no")
    clinic$id <- seq_len(m)

    cres <- synth(clinic, ~ id, m = 1, seed = 1)

Name the sensitive column as `target`; the remaining quasi-identifiers become the attacker’s keys.

    disclosure_risk(clinic, cres,
                    quasi  = c("region", "agecat", "hypertension"),
                    target = "hypertension",
                    seed   = 1)

    #> <flexsynth_disclosure>
    #>   rows            : real 800  synthetic 800
    #>   quasi-identifiers: region, agecat, hypertension
    #>
    #> Replicated uniques (identity risk):
    #>   real sample-uniques : 3
    #>   reproduced in syn   : 3  (100.00% of uniques, 0.38% of real rows)
    #>   syn rows copying a real row: 800  (100.00% of syn)
    #>
    #> Distance to closest record (Gower, 0 = exact copy):
    #>   syn->real  : median 0.0000   5th pct 0.0000   exact copies 100.00%
    #>   real->real : median 0.0000   (baseline)
    #>   median syn distance is not smaller than the real-neighbour baseline
    #>   descriptive only: inspect lower-tail distances and exact copies; this is not a safety guarantee
    #>
    #> Membership inference: not run (supply `holdout` of non-training records).
    #>
    #> Attribute disclosure (TCAP, target = hypertension):
    #>   TCAP 0.707   baseline 0.506   lift +0.200   (coverage 100.0%, keys: region, agecat)
    #>   key-conditioning attributes the target well above the covered-record margin -> inspect

Read the `Attribute disclosure` block: a `lift` well above zero means the synthetic data lets an attacker attribute the true `hypertension` status meaningfully better than the marginal rate alone — here because `agecat` carries real signal about it. That is a governance signal, not a verdict: some attribute association is the whole point of useful data. Weigh the lift against how sensitive the target is and who receives the release. A `lift` near zero means the keys add almost nothing beyond the margin.

## Where this fits

Track A output must never be described as differentially private. The tools here provide evidence for a documented release review, but the release decision must also consider data sensitivity, plausible attackers, access controls, and independent governance review. For a formal privacy guarantee instead, see `vignette("differential-privacy")` and verify all of its stated domain and budget-accounting conditions. Pooled inference (`pool_synth()`, `synth_glm()`) is a Track A tool — inference from a differentially private release must also account for DP noise and is out of scope here.