EMLMLasso fits linear mixed models with a lasso penalty on the
fixed-effects coefficients, for clustered or longitudinal data, via an
Expectation-Maximization (EM) algorithm. At each E-step, the random
effects are treated as missing data and replaced by their conditional
means and covariances; the fixed effects are updated at each M-step with
a coordinate-descent lasso fit (glmnet::glmnet()) on the
random-effects- adjusted response.
One function, EMLMLasso(), does the fitting, and what
lambda is decides the mode:
lambda as NULL (the default) or
passing a vector of candidate values fits the model over a grid and
selects the one that minimizes BIC. This is the recommended entry point
for most uses, since choosing lambda by hand rarely gives
the best model.lambda, no search.simulate_lasso_lmm() generates longitudinal data from a
sparse linear mixed model: p fixed-effect covariates, only
pstar of which have a non-zero true coefficient, plus a
random intercept and random slope on time.
set.seed(2026)
p <- 10
pstar <- 3
dat <- simulate_lasso_lmm(n = 40, nj = 5, p = p, pstar = pstar, sigmae = 1, seed = 2026)
head(dat)
#> id time y x1 x2 x3 x4 x5
#> 1 1 1 1.9120118 0.52058907 -0.4320264 1.3368926 0.962138142 -1.6052617
#> 2 1 2 -0.2994622 -1.07969076 0.4585911 0.5527104 -0.178134779 -0.5024982
#> 3 1 3 -0.9837634 0.13923812 -0.2293520 -1.0565264 0.003289601 -1.1958575
#> 4 1 4 -0.5575649 -0.08474878 -1.8569622 -0.1901830 0.511615166 -1.0710676
#> 5 1 5 -1.0219079 -0.66663962 -0.2896742 -0.9092242 -0.456060688 0.7461867
#> 6 2 1 -4.5323892 -2.51608903 1.7671412 -2.0009155 -0.210904119 -0.4368864
#> x6 x7 x8 x9 x10
#> 1 1.7705647 0.292106057 1.5912746 -0.7983005 -1.56410465
#> 2 0.9003771 0.735734309 -1.9549243 1.1286730 -0.57399073
#> 3 1.3304134 -0.004291783 -0.9887574 -0.5711334 -0.09445245
#> 4 -0.3492971 -0.446130570 0.9901177 1.1552387 -0.73474588
#> 5 1.0621850 1.014299812 0.9884280 -0.6623258 1.35623937
#> 6 -0.6199544 -1.290849154 -0.5208329 1.2274016 0.15436091The response is y; the grouping variable is
id; the fixed-effect covariates are
x1, ..., x10, of which only x1, x2, x3 have a
non-zero true coefficient (equal to 1 in this simulation).
formFixed and formRandom are ordinary R
formulas. Here formFixed includes all ten covariates, and
formRandom = ~ time requests a random intercept and a
random slope on time (the true data-generating
structure).
fit <- EMLMLasso(dat, formFixed = y ~ ., groupVar = "id", formRandom = ~time,
lambda = 0.15, verbose = FALSE)
fit
#> EM algorithm for lasso-penalized linear mixed models
#> Call:
#> EMLMLasso(data = dat, formFixed = y ~ ., groupVar = "id", formRandom = ~time,
#> lambda = 0.15, verbose = FALSE)
#>
#> lambda: 0.15 | active covariates: 3 of 11
#>
#> Fixed effects (non-zero only):
#> x1 x2 x3
#> 0.9209 0.7119 0.7202
#>
#> Random effects covariance (D):
#> (Intercept) time
#> (Intercept) 0.7495 0.6451
#> time 0.6451 0.6889
#>
#> Residual variance: 1.0296
#> logLik: -370.1433 AIC: 754.2866 BIC: 777.3748
#> EM iterations: 200 (did not converge)coef(), fitted() and
residuals() methods are available:
In practice, lambda is rarely known in advance. Leaving
it as NULL fits the model across a data-driven grid of
lambda values (sized by n_lambdas) and reports the one with
the lowest BIC.
search <- EMLMLasso(dat, formFixed = y ~ ., groupVar = "id", formRandom = ~time,
n_lambdas = 15)
search
#> EM algorithm for lasso-penalized linear mixed models
#> Call:
#> EMLMLasso(data = dat, formFixed = y ~ ., groupVar = "id", formRandom = ~time,
#> n_lambdas = 15)
#>
#> BIC search over 15 lambda values
#>
#> lambda: 0.1083 | active covariates: 3 of 11
#>
#> Fixed effects (non-zero only):
#> x1 x2 x3
#> 0.9717 0.7661 0.7790
#>
#> Random effects covariance (D):
#> (Intercept) time
#> (Intercept) 0.7751 0.6344
#> time 0.6344 0.6967
#>
#> Residual variance: 0.9675
#> logLik: -365.8906 AIC: 745.7812 BIC: 768.8694
#> EM iterations: 200 (did not converge)The selected model correctly identifies x1,
x2 and x3 as the active covariates, shrinking
the other seven to exactly zero.
coef(search)
#> time x1 x2 x3 x4 x5 x6 x7
#> 0.0000000 0.9717481 0.7661102 0.7790091 0.0000000 0.0000000 0.0000000 0.0000000
#> x8 x9 x10
#> 0.0000000 0.0000000 0.0000000plot() shows BIC across the lambda grid, with the
selected value marked:
summary() gives a more detailed readout of the selected
model, including which covariates were shrunk to zero:
summary(search)
#> EM algorithm for lasso-penalized linear mixed models
#> Call:
#> EMLMLasso(data = dat, formFixed = y ~ ., groupVar = "id", formRandom = ~time,
#> n_lambdas = 15)
#>
#> Number of groups: 40 | total observations: 200
#> BIC search over 15 lambda values
#>
#> Selected lambda (by BIC): 0.1083
#>
#> Selected fixed effects ( 3 of 11 non-zero):
#> x1 x2 x3
#> 0.9717 0.7661 0.7790
#>
#> Shrunk-to-zero covariates: time, x4, x5, x6, x7, x8, x9, x10
#>
#> Random effects covariance (D):
#> (Intercept) time
#> (Intercept) 0.7751 0.6344
#> time 0.6344 0.6967
#> Residual variance: 0.9675
#>
#> logLik: -365.8906 AIC: 745.7812 BIC: 768.8694 ( 7 effective parameters )The covariance for cluster/subject \(i\) is \(\Sigma_i
= Z_i D Z_i' + \sigma^2 I\), with \(b_i \sim N(0, D)\) independent of the
residual errors. EMLMLasso() assumes Gaussian random
effects and errors, and conditional independence within a cluster given
the random effects.
riboflavinV100 (bundled with this package; see
?riboflavinV100) is a longitudinal version of the
well-known riboflavin production dataset for Bacillus subtilis,
with 100 gene-expression covariates as fixed-effect candidates – a
high-dimensional (p >> n) setting. It is worked
through separately in vignette("EMLMLasso-riboflavin").
This package implements the EMLMLasso algorithm of Oliveira,
Schumacher and Lachos (2026) with modifications. Most notably, the
penalty λ is selected by BIC over a grid of candidate values, with λ
held fixed within each EM fit, instead of being re-selected by
cross-validation at every EM iteration as in Algorithm 1 of the paper.
Convergence is also assessed on the change in parameter estimates rather
than on the relative change in the penalized log-likelihood. Results may
therefore differ from those reported in the paper. See
?EMLMLasso for details (full citation below; run
citation("EMLMLasso") for a BibTeX-ready entry).
Oliveira DCR, Schumacher FL, Lachos VH (2026). The use of the EM algorithm for regularization problems in high-dimensional linear mixed-effects models. Statistical Methods in Medical Research, 35(2), 240-254. doi:10.1177/09622802251399913