---
title: "A real-data example: riboflavin production in B. subtilis"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{A real-data example: riboflavin production in B. subtilis}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  eval = requireNamespace("lme4", quietly = TRUE)
)
```

```{r setup}
library(EMLMLasso)
```

## The data

`riboflavinV100` is a longitudinal version of the well-known riboflavin
production dataset for *Bacillus subtilis* (Bühlmann, Kalisch and Meier,
2014), reconstructed from the
metadata in the original microarray file names to reproduce the
longitudinal structure of the real-data application in Oliveira,
Schumacher and Lachos (2026, Section 5): 28 strains, each measured 2-4
times, 71 observations in total. See `?riboflavinV100` for more information.

```{r data}
data(riboflavinV100)
dim(riboflavinV100)
table(table(riboflavinV100$strain))
```

`strain` is the grouping variable, `time` is measured in hours after induction,
`y` is log riboflavin production, and the remaining 100
columns are gene expression levels -- the same 100-gene reduction used by the paper itself.

## Fitting

Two data-prep choices are made before fitting, both following Section 5 of
Oliveira et al. (2026). First, the response and all covariates (the 100
genes and `time`) are standardized to mean 0 and variance 1. Second, `time`
is included as a candidate *fixed*-effect covariate, in addition to being a
random slope via `formRandom`, matching the model structure of Eq. 14 of
the paper. Coefficients are therefore reported on the standardized scale.

We also set `refit = TRUE`. Lasso coefficients are shrunk toward zero by
construction, which biases them as point estimates even once the right
covariates are selected; `refit = TRUE` follows the BIC-optimal lasso fit
with an ordinary, unpenalized `lme4::lmer()` fit on just the covariates it
selected, and reports those as the coefficients below. This is a
**point-estimate correction only** -- the refit's standard errors,
p-values and confidence intervals are not valid inference, because the
covariate set was chosen using the same data, and ignoring that selection
step would understate their uncertainty. `EMLMLasso()` does not print
or summarize refit-based inferential statistics for this reason; the
underlying `lme4` fit is available as `refit_fit`, but its inferential
output should not be used.

```{r bic-search}
genes <- setdiff(names(riboflavinV100), c("strain", "time", "y"))
fixedForm <- as.formula(paste("y ~ time +", paste(genes, collapse = " + ")))

riboflavinV100_std <- riboflavinV100
std_vars <- c("y", "time", genes)
riboflavinV100_std[std_vars] <- lapply(riboflavinV100_std[std_vars],
                                       function(v) as.numeric(scale(v)))

set.seed(2026)
search <- EMLMLasso(riboflavinV100_std, formFixed = fixedForm, groupVar = "strain",
                    formRandom = ~ time, MaxIter = 500, refit = TRUE)
```

## Results

```{r results}
search
```

```{r bic-plot, fig.width=6, fig.height=4}
plot(search)
```

The BIC-optimal model keeps 22 genes plus `time`. The coefficients below
are the post-selection refit's point estimates, not the raw (shrunk)
lasso ones:

```{r coef}
b <- coef(search, refit = TRUE)
sort(abs(b[!is.na(b) & b != 0]), decreasing = TRUE)
```

For comparison, the raw lasso estimates the BIC search actually selected
on:

```{r coef-lasso}
b_lasso <- coef(search)
sort(abs(b_lasso[b_lasso != 0]), decreasing = TRUE)
```

## Interpretation

BIC is flat over the largest lambda values, where no covariate is
selected, and changes little until lambda falls to about 0.006. It then
drops sharply to its minimum at `lambda = 0.005` (23 active covariates: 22
genes and `time`) and rises again over the next few, smaller values (24 to
27 active covariates). `time` is selected as a fixed effect, with a negative
coefficient, consistent with riboflavin production decreasing over time.

These results differ from Table 5 of Oliveira et al. (2026), which also
reports 22 selected genes but retains `time` only as a random effect; 11 of
the 22 genes selected here are among the paper's. The
difference is expected: the paper selects $\lambda$ by cross-validation at every EM
iteration, while this package selects it by BIC over a grid (see
`?EMLMLasso`).

## References

Oliveira DCR, Schumacher FL, Lachos VH (2026). The use of the EM algorithm
for regularization problems in high-dimensional linear mixed-effects
models. *Statistical Methods in Medical Research*, 35(2), 240-254.
[doi:10.1177/09622802251399913](https://doi.org/10.1177/09622802251399913)

Bühlmann P, Kalisch M, Meier L (2014). High-Dimensional Statistics with a
View Toward Applications in Biology. *Annual Review of Statistics and Its
Application*, 1, 255-278.
