polars_online.sim

A seeded simulator for correlation regimes (docs/ENHANCEMENTS.md E64).

The detectors in this library — deco, hmm, corrchange, bocpd — are claims about streams whose correlation structure changes. A claim like that can only be measured against data whose truth is known, and the truth has to include the awkward parts: series that report at their own times, levels observed with additive noise, autocorrelated increments, a scale that moves with the regime, a periodic pattern, and an activity clock.

regimes() produces all of it from one seed, and hands back the truth beside the data:

rows

what a consumer sees: levels x_1 .. x_m (so polars_online.prep.refresh_time() and then .diff() apply), a clock, a cycle index and an optional activity count.

truth_rows

per row: the block, the state, the scale multiplier and the interpolation fraction.

truth_blocks

per block: the state, the row count, and the block’s true correlation matrix as vech.

Everything is drawn from numpy.random.default_rng(seed) in one order, so two calls with the same seed give byte-identical frames. numpy only; no scipy.

polars_online.sim.regimes(m: int, *, states: Sequence[Any], transition: Any, n_blocks: int, rows_per_block: int, durations: Sequence[int] | None = None, design: str = 'step', smooth_rows: int = 0, phi: float | Sequence[float] = 0.0, scale_state: float | Sequence[float] = 0.0, async_rates: Sequence[float] | None = None, noise: float = 0.0, cycle_profile: Sequence[float] | None = None, cycle_rows: int | None = None, activity: tuple[float, float] | None = None, seed: int = 0) → dict[str, DataFrame]

A simulated stream of m series whose correlation changes by regime.

states is a list of K correlation matrices (m x m, unit diagonal, PSD) or K floats, each an equicorrelation. transition is the K x K row-stochastic matrix that drives one state per block, and there are n_blocks blocks of rows_per_block rows. durations (one positive block count per state) makes the sojourn deterministic instead and draws the next state from transition with its diagonal removed — the recurring-state design, where each state lasts exactly as long as it says.

design="step" switches at the block boundary. design="smooth" interpolates the correlation matrix linearly over smooth_rows rows around it; a convex combination of two correlation matrices is one, so every matrix along the way is valid.

The latent returns are eps_t ~ N(0, R_t), filtered to y_it = phi_i y_i,t-1 + eps_it and scaled by exp(scale_state * s_t). The documented truth is the innovation correlation: an AR filter moves the return correlation of a pair with unequal phi, which is exactly what polars_online.corr.fisher_se()’s inflation is about. scale_state as a scalar makes the noise scale rise with the state index; as a list it gives one multiplier exponent per state.

cycle_profile is cycle_rows multipliers in (0, 1] applied to the off-diagonal of R_t at row t mod cycle_rows — a mix towards the identity, so the matrix stays PSD. cycle_rows defaults to rows_per_block, and session = t // cycle_rows.

activity is (mean, shape) for a Gamma count per row, with the mean scaled by the same scale multiplier; clock is then the cumulative activity, and the row index otherwise.

x_1 .. x_m are levels: the cumulative sum of the latent returns plus noise * N(0, 1) per observed row. Noise on the level is what the literature models, and it makes the observed return an MA(1) with a negative first autocorrelation — the microstructure effect rcov exists to undo. With async_rates (expected observations per row, per series) a row where series i reported nothing carries null for x_i: last-observation sampling is one forward_fill away, and unpivot over the non-null rows is polars_online.prep.refresh_time()’s long input.

Returns {"rows", "truth_rows", "truth_blocks"}. Two calls with the same seed give byte-identical frames.