Package {clustEff}

Type:	Package
Title:	Clusters of Effects Curves in Quantile Regression Models
Version:	1.0.0
Description:	Clustering of curves based on a dissimilarity that combines the concordance of their shapes and their closeness, as proposed by Sottile and Adelfio (2019) <doi:10.1007/s00180-018-0817-8>. The method can be applied to the effects curves of quantile regression coefficients models fitted with 'qrcm', with univariate or multivariate responses, and to curves observed in time, as in functional data analysis. The package also implements the clustering of waveforms by trimmed k-means of functional principal component scores (Adelfio, Chiodi, D'Alessandro and Luzio, 2011).
License:	GPL-2
URL:	https://github.com/gianluca-sottile/clustEff
BugReports:	https://github.com/gianluca-sottile/clustEff/issues
Depends:	R (≥ 4.1.0)
Imports:	cluster, graphics, grDevices, splines, stats
Suggests:	fda, knitr, qrcm (≥ 3.0), rmarkdown, testthat (≥ 3.0.0)
VignetteBuilder:	knitr
Config/testthat/edition:	3
Encoding:	UTF-8
Language:	en-US
Config/roxygen2/version:	8.1.1
NeedsCompilation:	no
Packaged:	2026-10-30 11:12:55 UTC; gianlucasottile
Author:	Gianluca Sottile [aut, cre], Giada Adelfio [aut]
Maintainer:	Gianluca Sottile <gianluca.sottile@unipa.it>
Repository:	CRAN
Date/Publication:	2026-11-03 23:20:02 UTC

clustEff: Clusters of Effects Curves in Quantile Regression Models

Description

Hierarchical clustering of curves based on the shape-distance dissimilarity of Sottile and Adelfio (2019): effects curves of quantile regression coefficients models (univariate or multivariate responses, see extract_curves() and clustEff()) and curves observed in time. The package also implements the functional principal components clustering (FPCAC) of Adelfio et al. (2011) for waveforms (fpcac(), opt.fpcac()).

Author(s)

Maintainer: Gianluca Sottile gianluca.sottile@unipa.it (ORCID)

Authors:

• Gianluca Sottile gianluca.sottile@unipa.it (ORCID)

• Giada Adelfio giada.adelfio@unipa.it

References

Sottile, G. and Adelfio, G. (2019). Clusters of effects curves in quantile regression models. Computational Statistics, 34, 551-569. doi:10.1007/s00180-018-0817-8

Adelfio, G., Chiodi, M., D'Alessandro, A. and Luzio, D. (2011). FPCA algorithm for waveform clustering. Journal of Communication and Computer, 8, 494-502.

See Also

Useful links:

• https://github.com/gianluca-sottile/clustEff

• Report bugs at https://github.com/gianluca-sottile/clustEff/issues

Clusters of Effects Curves

Description

Hierarchical clustering of curves, such as the effects curves \beta_j(p) of quantile regression coefficients models or curves observed in time, based on the shape-distance dissimilarity of Sottile and Adelfio (2019).

Usage

clustEff(
  curves,
  lower = NULL,
  upper = NULL,
  p = NULL,
  k = NULL,
  alpha = NULL,
  step = c("both", "shape", "distance"),
  deriv = 2,
  combine = c("product", "mean"),
  shape.tol = 1e-06,
  criterion = c("height", "pi.out", "dist.rel", "silhouette", "conf.int"),
  method = "ward.D2",
  diss = NULL,
  smooth = FALSE,
  df = NULL,
  conf.level = 0.9,
  stand = FALSE,
  legacy = FALSE,
  verbose = FALSE,
  ...
)

## S3 method for class 'clustEff'
print(x, ...)

Arguments

Details

The algorithm (Sottile and Adelfio, 2019, Section 3.2) computes the dissimilarity matrix with distshape(), builds a dendrogram with stats::hclust(), selects the number of clusters l^* and computes the mean curve (and mean bands) of each cluster.

Criteria for the number of clusters l:

• "height" (default): l^* corresponds to the largest gap between consecutive heights of the dendrogram.

• "pi.out" (effects curves with bands): with \bar{LB}_j(p) and \bar{UB}_j(p) the mean bands of cluster j,

  \pi_{out}(l) = l^{-1}\sum_{j=1}^l q_j^{-1}\sum_{i \in j}\int [1 - I\{\bar{LB}_j(p) \le \beta_i(p) \le \bar{UB}_j(p)\}]\,dp

  is the average proportion of the curves lying outside the mean bands of their cluster (Sottile and Adelfio, 2019, Section 3.1). It decreases with l; l^* is the elbow, i.e. the value at which the decrease slows down most.

• "dist.rel" (waveforms): dist_{rel}(l) = l\,\max_j q_j^{-1}\sum_{i\in j} \int|\bar\beta_j(p) - \beta_i(p)|\,dp (Section 3.1), and l^* is the first number of clusters at which dist_{rel} stops decreasing.

• "silhouette": l^* maximizes the average silhouette width (default criterion of 'clustEff' 0.3.x).

• "conf.int": criterion "conf.int" of 'clustEff' 0.3.x (coverage of the mean bands weighted by the silhouette widths).

The paper selects l^* minimizing \pi_{out}(l) - \pi_{out}(l+1) or dist_{rel}(l) - dist_{rel}(l+1), and writes \pi_{out} with the indicator of the curves inside the bands and a factor l. In simulations with 2 to 4 groups of effects curves these literal rules rarely found the true number of clusters, while the rules above did better and "height" was the most accurate; see the vignette.

Value

An object of class "clustEff", a list with items:

References

Sottile, G. and Adelfio, G. (2019). Clusters of effects curves in quantile regression models. Computational Statistics, 34, 551-569. doi:10.1007/s00180-018-0817-8

See Also

distshape(), extract_curves(), summary.clustEff(), plot.clustEff(), fpcac().

Examples

## Waveforms: three groups of noisy curves
set.seed(1234)
tt <- seq(0, 1, length.out = 100)
W <- cbind(sapply(1:5, function(i) sin(3 * pi * tt) + rnorm(100, 0, 0.1)),
           sapply(1:5, function(i) cos(3 * pi * tt) + rnorm(100, 0, 0.1)),
           sapply(1:5, function(i) sin(3 * pi * tt) * cos(pi * tt) + rnorm(100, 0, 0.1)))
fit <- clustEff(W, p = tt, k = c(2, 5), smooth = TRUE)
fit
summary(fit)
plot(fit)

## Effects curves with confidence bands: three groups of four curves
p <- seq(0.01, 0.99, 0.01)
B <- cbind(sapply(1:4, function(i) 1 + p + rnorm(1, 0, 0.05)),
           sapply(1:4, function(i) -1 + 2 * p^2 + rnorm(1, 0, 0.05)),
           sapply(1:4, function(i) 2 - 3 * p^3 + rnorm(1, 0, 0.05)))
fit2 <- clustEff(B, lower = B - 0.2, upper = B + 0.2, p = p, k = c(2, 5))
fit2$clusters
plot(fit2)

Defunct Functions and Arguments in clustEff

Description

The following were removed in 'clustEff' 1.0.0:

• extract.object(): use extract_curves() on models fitted with qrcm::iqr() (e.g., extract_curves(lapply(responses, function(y) iqr(y ~ x))));

• the arguments Beta, Beta.lower, Beta.upper, diss.mat, cut.method, approx.spline, nbasis, plot, trace and ask of clustEff(): use curves, lower, upper, diss, criterion, smooth, df, plot(), verbose, and a fixed k after inspecting plot(fit, "dendrogram");

• the arguments X, fd, niter, Ksteps, seed and disp of fpcac() and opt.fpcac(): use curves, a matrix of curves, nstart, iter.max, set.seed() and verbose; method of opt.fpcac() is now criterion.

Usage

extract.object(...)

Arguments

Value

No return value: an error is raised.

Shape-Distance Dissimilarity between Curves

Description

Computes the dissimilarity between curves of Sottile and Adelfio (2019, eq. 1), which combines the concordance of their shapes (the signs of their second derivatives) and their closeness (their distance relative to the distances between all pairs of curves), pointwise on a grid.

Usage

distshape(
  curves,
  p = NULL,
  alpha = 0.5,
  step = c("both", "shape", "distance"),
  deriv = 2,
  combine = c("product", "mean"),
  d = NULL,
  shape.tol = 1e-06,
  legacy = FALSE,
  ...
)

Arguments

Details

For two curves \beta_i and \beta_{i'},

d_{shape}(p) = I\{sign(\beta_i''(p)) = sign(\beta_{i'}''(p))\},\qquad d_{distance}(p) = I\{|\beta_i(p) - \beta_{i'}(p)| \le f_\alpha(p)\},

where f_\alpha(p) is the \alpha-quantile of the distances |\beta_j(p) - \beta_{j'}(p)| between all pairs of curves at p. The dissimilarity is

d(i, i') = 1 - \int d_{shape}(p)\, d_{distance}(p)\,dp,

where the integral (normalized by the length of the interval) is computed with the trapezoidal rule on the grid p. The computations are vectorized over the pairs of curves and need O(q^2) memory for q curves.

The paper defines the shape concordance as sign(\beta_i'') \times sign(\beta_{i'}'') = 1; here derivatives below shape.tol are set to zero and two zero derivatives are concordant, which avoids that, e.g., two linear curves are always discordant. The shape component alone is a Hamming distance between the sign patterns; the distance component, based on a threshold, does not satisfy the triangle inequality in general. With only two curves the distance component is always 1, and step = "both" reduces to "shape".

Value

An object of class "dist" (see stats::dist()) with attributes alpha, step, deriv, combine and legacy.

References

Sottile, G. and Adelfio, G. (2019). Clusters of effects curves in quantile regression models. Computational Statistics, 34, 551-569. doi:10.1007/s00180-018-0817-8

See Also

clustEff()

Examples

tt <- seq(0, 1, length.out = 100)
curves <- cbind(a1 = sin(3 * pi * tt), a2 = sin(3 * pi * tt) + 0.1,
                b1 = cos(3 * pi * tt), b2 = cos(3 * pi * tt) - 0.1)
distshape(curves, p = tt)
distshape(curves, p = tt, step = "shape")

Effects Curves of Quantile Regression Coefficients Models

Description

Extracts the estimated quantile regression coefficient functions \hat\beta_j(p) (the effects curves), their pointwise confidence bands and their second derivatives from one or more models fitted with qrcm::iqr(), in the format used by clustEff().

Usage

extract_curves(object, p = seq(0.01, 0.99, 0.01), which, level = 0.95)

## S3 method for class 'effects_curves'
print(x, ...)

Arguments

Details

The bands are \hat\beta_j(p) \pm z_{(1+level)/2}\,se\{\hat\beta_j(p)\}, with standard errors from predict(object, type = "beta", se = TRUE). The second derivatives \hat\beta_j''(p), used by the shape component of the dissimilarity of distshape(), are computed by central differences of the smooth fitted functions (step 10^{-3}; exact up to rounding errors when \beta(p) is a polynomial of degree up to 3 in p, and very accurate otherwise, unlike derivatives of interpolating splines of the estimates).

With a list of models (e.g., the same covariates for different responses, as in the multivariate case of Sottile and Adelfio, 2019), the curves are named "model:coefficient", where model is the name of the list element (or its position).

Value

An object of class "effects_curves", a list with items curves, lower, upper and d2 (matrices with one row for each value of p and one column for each curve), p, level and info (a data frame with the model and the coefficient of each curve).

References

Sottile, G. and Adelfio, G. (2019). Clusters of effects curves in quantile regression models. Computational Statistics, 34, 551-569. doi:10.1007/s00180-018-0817-8

See Also

clustEff(), qrcm::iqr()

Examples

if (requireNamespace("qrcm", quietly = TRUE)) {
  set.seed(1)
  n <- 300
  x <- runif(n, 0, 5)
  # five responses with three different effects of x
  slopes <- list(function(u) 0.5 + u, function(u) 0.5 + u, function(u) -1 + 0.5 * u,
                 function(u) -1 + 0.5 * u, function(u) 2 * u^2)
  fits <- lapply(slopes, function(b) {
    u <- runif(n)
    y <- qnorm(u) + b(u) * x
    qrcm::iqr(y ~ x, formula.p = ~ I(p) + I(p^2))
  })
  names(fits) <- paste0("y", 1:5)
  ec <- extract_curves(fits, which = "x")
  ec
  fit <- clustEff(ec, k = 3)
  fit$clusters
}

Functional Principal Components Analysis of Curves on a B-spline Basis

Description

Represents each curve on a B-spline basis by least squares and computes the functional principal components (harmonics) and the scores, as in Ramsay and Silverman (2005, Chapter 8). It is used by fpcac().

Usage

fpca_bspline(
  curves,
  t = NULL,
  nbasis = 5,
  norder = 3,
  nharmonics = 3,
  lambda = 1e-08
)

Arguments

Value

A list with items scores (curves x harmonics), values (the eigenvalues), varprop (proportion of variance explained), coefs (basis coefficients of the curves), harmonics (basis coefficients of the harmonics), mean (basis coefficients of the mean curve), fitted (the smoothed curves on t), basis (the basis matrix on t) and gram (the Gram matrix of the basis).

References

Ramsay, J. O. and Silverman, B. W. (2005). Functional Data Analysis, 2nd ed. Springer.

Examples

tt <- seq(0, 1, length.out = 100)
X <- sapply(1:20, function(i) sin(2 * pi * tt + runif(1)) + rnorm(100, 0, 0.1))
f <- fpca_bspline(X, tt, nbasis = 10, nharmonics = 2)
f$varprop

Functional PCA-based Clustering of Curves (FPCAC)

Description

Clusters curves (e.g., seismic waveforms) by a trimmed k-means applied to their functional principal component scores (Adelfio et al., 2011).

Usage

fpcac(
  curves,
  t = NULL,
  K = 2,
  nbasis = 5,
  norder = 3,
  nharmonics = 3,
  alpha = 0,
  nstart = 30,
  iter.max = 25,
  init = c("kmeans++", "random"),
  conf.level = 0.9,
  verbose = FALSE,
  ...
)

## S3 method for class 'fpcac'
print(x, ...)

Arguments

Details

Each curve is represented on a B-spline basis and summarized by the scores of its first nharmonics functional principal components (see fpca_bspline()). The trimmed k-means of García-Escudero and Gordaliza finds K centers minimizing the mean squared distance of the closest \lfloor N(1-\alpha)\rfloor curves to their nearest center. Use set.seed() for reproducible results.

Value

An object of class "fpcac", a list with items clusters (0 = trimmed), K, centers, obj.function, radius, scores, fpca (the output of fpca_bspline()), curves, t, mean, mean.lower, mean.upper (mean curve and pointwise bands of each cluster), mean.dist (correlation distance of each curve from the mean of its cluster) and call.

References

Adelfio, G., Chiodi, M., D'Alessandro, A. and Luzio, D. (2011). FPCA algorithm for waveform clustering. Journal of Communication and Computer, 8, 494-502.

García-Escudero, L. A. and Gordaliza, A. (2005). A proposal for robust curve clustering. Journal of Classification, 22, 185-201. doi:10.1007/s00357-005-0013-8

See Also

opt.fpcac(), clustEff()

Examples

set.seed(1)
tt <- seq(0, 1, length.out = 100)
X <- cbind(sapply(1:10, function(i) sin(3 * pi * tt) + rnorm(100, 0, 0.2)),
           sapply(1:10, function(i) cos(3 * pi * tt) + rnorm(100, 0, 0.2)),
           sapply(1:2, function(i) rnorm(100, 0, 0.2)))          # two outliers
fit <- fpcac(X, t = tt, K = 2, nbasis = 10, alpha = 0.1)
fit
plot(fit)

Number of Clusters for FPCAC

Description

Fits fpcac() for K = 1, \dots, k.max and computes the average silhouette width or the total within-cluster sum of squares.

Usage

opt.fpcac(
  curves,
  t = NULL,
  k.max = 5,
  criterion = c("silhouette", "wss"),
  nbasis = 5,
  norder = 3,
  nharmonics = 3,
  alpha = 0,
  nstart = 30,
  iter.max = 25,
  init = c("kmeans++", "random"),
  diss = NULL,
  ...
)

## S3 method for class 'opt.fpcac'
print(x, ...)

## S3 method for class 'opt.fpcac'
plot(x, ...)

Arguments

Value

An object of class "opt.fpcac", a list with items criterion (the values for each K), K.opt, clusters (matrix of the clusters for each K), fits (the fpcac() fits) and criterion.name. Trimmed curves are excluded from the computation of the criteria.

See Also

fpcac()

Examples

set.seed(1)
tt <- seq(0, 1, length.out = 100)
X <- cbind(sapply(1:8, function(i) sin(3 * pi * tt) + rnorm(100, 0, 0.2)),
           sapply(1:8, function(i) cos(3 * pi * tt) + rnorm(100, 0, 0.2)),
           sapply(1:8, function(i) sin(3 * pi * tt) * cos(pi * tt) + rnorm(100, 0, 0.2)))
o <- opt.fpcac(X, t = tt, k.max = 5, nbasis = 10)
o$K.opt
plot(o)

Plot Clusters of Curves

Description

Plot Clusters of Curves

Usage

## S3 method for class 'clustEff'
plot(
  x,
  type = c("clusters", "dendrogram", "silhouette", "dissimilarity", "criterion"),
  which,
  polygon = TRUE,
  col,
  ...
)

Arguments

Value

No return value, called for its side effects.

See Also

clustEff()

Plot an FPCAC Fit

Description

Plot an FPCAC Fit

Usage

## S3 method for class 'fpcac'
plot(x, which, polygon = TRUE, col, ...)

Arguments

Value

No return value, called for its side effects.

See Also

fpcac()

Summarize Clusters of Curves

Description

Summarize Clusters of Curves

Usage

## S3 method for class 'clustEff'
summary(object, ...)

## S3 method for class 'summary.clustEff'
print(x, digits = max(3L, getOption("digits") - 3L), ...)

Arguments

Value

An object of class "summary.clustEff": a list with the number of clusters, the number of curves and grid points, the options used, the clustering table, the within-cluster measures (with their weighted average) and the criterion used to select the number of clusters.

See Also

clustEff()

Summarize an FPCAC Fit

Description

Summarize an FPCAC Fit

Usage

## S3 method for class 'fpcac'
summary(object, ...)

## S3 method for class 'summary.fpcac'
print(x, digits = max(3L, getOption("digits") - 3L), ...)

Arguments

Value

An object of class "summary.fpcac" with the clustering table, the number of trimmed curves, the proportion of variance of each harmonic and, for each cluster, the average correlation distance of its curves from the mean curve.

See Also

fpcac()