Skip to contents

Builds the same lag-1 design used by fit_graphical_var() and fit_var(), optionally detrends or differences each series, and returns tidy diagnostics for missingness, day-boundary drops, simple linear trends, AR(1) persistence, split-half mean/variance drift, an ADF-style unit-root screen, and zero-variance variables. It makes the modelling input explicit before estimating VAR, graphical VAR, uSEM, GIMME, or mlVAR models; with detrend it also cleans a non-stationary series in place so the flags can be rechecked on the transformed data.

Usage

preprocess(
  data,
  vars,
  id = NULL,
  day = NULL,
  beep = NULL,
  scale = TRUE,
  center_within = TRUE,
  detrend = "none",
  checks = c("trend", "high_ar", "unit_root", "mean_shift", "sd_shift", "zero_variance"),
  delete_missings = TRUE,
  min_obs = NULL,
  subject = NULL,
  trend_alpha = 0.05,
  ar_threshold = 0.95,
  mean_shift_threshold = 0.8,
  sd_ratio_threshold = 2,
  unit_root_t_cutoff = -2.86
)

Arguments

data

A data.frame or matrix with columns for variables and optional id/day/beep columns.

vars

Character vector of variable names.

id

Character. Name of the person-ID column, or NULL for a single series.

day

Character. Name of the day/session column, or NULL.

beep

Character. Name of the measurement-occasion column, or NULL.

scale

Logical. Whether to standardize variables before lagging. Default TRUE.

center_within

Logical. Whether to centre within person when more than one id is present. Default TRUE.

detrend

How to remove non-stationarity from each series before lagging. Either a single string applied to every variable, or a named character vector giving a per-variable method (unlisted variables are left untouched, e.g. c(planning = "difference", value = "linear")). The available methods are:

"none"

Default; diagnose only, transform nothing.

"auto"

Detrend only the subject-series that are flagged, leaving the stationary ones untouched: differencing a stochastic trend (unit root or near-unit-root persistence) and linearly detrending a deterministic trend. The "clean whoever needs it" option – no subsetting, one call over everyone. Can be set per variable too.

"linear"

Replace the series with the residuals of a within-person regression on a linear time index.

"difference"

First-difference the series within id/day blocks.

The diagnostics and the returned design reflect the detrended series, so the trend and unit-root flags can be rechecked after cleaning.

checks

Character vector selecting which stationarity checkups to run: any of "trend", "high_ar", "unit_root", "mean_shift", "sd_shift", "zero_variance". Defaults to all of them. Deselecting a check turns its flag off in the report, in the flag_stationarity_risk roll-up, and in the "auto" detrend decision, so you can screen for only what you care about.

delete_missings

Logical. If TRUE, $pairs contains only complete current/lagged rows; if FALSE, first rows of blocks and incomplete rows are retained with NA lags, matching .gvar_tsdata(). Default TRUE.

min_obs

Integer or NULL. Keep only subjects with at least this many observations.

subject

Optional vector naming the subject(s) to preprocess.

trend_alpha

Numeric p-value cutoff for the trend flag. Default 0.05.

ar_threshold

Numeric absolute AR(1) cutoff for the high-persistence flag. Default 0.95.

mean_shift_threshold

Numeric absolute standardized split-half mean shift cutoff. Default 0.8.

sd_ratio_threshold

Numeric split-half SD ratio cutoff. Default 2.

unit_root_t_cutoff

Numeric cutoff for the ADF-style lag-level t-statistic. Values greater than this cutoff are flagged as unit-root risk. Default -2.86, a common large-sample intercept-only screening cutoff.

Value

A preprocess_result object. as.data.frame() returns the per-subject, per-variable diagnostics; summary() rolls them up to one row per variable; matrices() returns the design matrices. It holds:

pairs

The ordered current/lagged design table, including intercept and L1_* columns.

counts

Per-subject/per-day row and lag-pair counts.

diagnostics

Per-subject/per-variable missingness, trend, AR(1), split-half drift, unit-root screen, and stationarity risk indicators.

matrices

The exact data_c and data_l matrices returned by the VAR/GVAR preprocessing path.

Examples

set.seed(1)
d <- data.frame(id = 1, day = 1, beep = 1:40,
                A = cumsum(rnorm(40)), B = rnorm(40))
pp <- preprocess(d, vars = c("A", "B"), id = "id", day = "day", beep = "beep")
#> 1 of 2 subject-series show a trend or unit-root that can bias the temporal network. preprocess() only diagnosed this; to clean just the series that need it, re-run with:
#>   preprocess(data = d, vars = c("A", "B"), id = "id", day = "day", beep = "beep", detrend = "auto")
pp
#> Idiographic Preprocessing
#>   Variables:      2 (A, B)
#>   Ordered rows:   40
#>   Retained pairs: 39
#>   Trend flags:    1
#>   High AR flags:  0
#>   Drift flags:    1
#>   Unit-root risk: 1
#>   Zero variance:  0
#>   Tables:         as.data.frame(x) | summary(x) | matrices(x)
#> 
#> 1 of 2 subject-series show a trend or unit-root that can bias the temporal network. preprocess() only diagnosed this; to clean just the series that need it, re-run with:
#>   preprocess(data = d, vars = c("A", "B"), id = "id", day = "day", beep = "beep", detrend = "auto")
summary(pp)
#>   variable n_series mean_sd n_trend n_high_ar n_unit_root n_flagged
#> 1        A        1       1       1         0           1         1
#> 2        B        1       1       0         0           0         0
as.data.frame(pp)
#>   subject variable  n n_observed missing_prop         mean sd  trend_slope
#> 1       1        A 40         40            0 2.637593e-17  1  0.054326088
#> 2       1        B 40         40            0 1.507583e-17  1 -0.008082785
#>      trend_t      trend_p        ar1      ar1_t        ar1_p mean_first_half
#> 1  5.0683995 1.072035e-05  0.8271452 10.1409016 3.128890e-12     -0.65545859
#> 2 -0.5851021 5.619376e-01 -0.1054387 -0.6407008 5.256656e-01      0.02001441
#>   mean_second_half  mean_shift mean_shift_std mean_shift_p sd_first_half
#> 1       0.65545859  1.31091717     1.31091717 3.485167e-06     0.8382505
#> 2      -0.02001441 -0.04002882     0.04002882 9.012450e-01     0.8745922
#>   sd_second_half sd_ratio unit_root_coef unit_root_t flag_zero_variance
#> 1      0.6674518 1.255897     -0.1728548   -2.119221              FALSE
#> 2      1.1344059 1.297068     -1.1054387   -6.717223              FALSE
#>   flag_trend flag_high_ar flag_mean_shift flag_sd_shift flag_unit_root
#> 1       TRUE        FALSE            TRUE         FALSE           TRUE
#> 2      FALSE        FALSE           FALSE         FALSE          FALSE
#>   flag_stationarity_risk
#> 1                   TRUE
#> 2                  FALSE
# Difference the trending series and recheck the flags:
preprocess(d, vars = c("A", "B"), id = "id", day = "day", beep = "beep",
           detrend = "difference") |> summary()
#>   variable n_series mean_sd n_trend n_high_ar n_unit_root n_flagged
#> 1        A        1   0.528       0         0           0         0
#> 2        B        1   1.499       0         0           0         0