macroforecast.window#
macroforecast.window defines the estimation/val/test time frame. It is the object
passed between data, feature engineering, model selection, models, and evaluation to
answer five questions:
how the pre-test estimation sample expands or rolls
how validation splits are created inside the estimation sample for model selection
where the final test origins start and end
how far each test target horizon runs
when the model is retrained versus reused
where each runner stage may fit stateful operations
The public unit is one WindowSpec. Internally it is composed in this order:
EstimationWindow, ValWindow, TestWindow, then AlignmentWindow.
Configuration Axes#
All major macro-forecasting time-frame choices are explicit:
Question |
Setting |
Where |
|---|---|---|
Should the pre-test estimation sample expand, roll, or stay fixed? |
|
|
How long is a rolling estimation sample? |
|
|
What is the minimum estimation sample before a test origin is allowed? |
|
|
How often does the final test origin move? |
|
|
Is test-origin movement row-count based or calendar based? |
positive integer or pandas offset |
|
How far does each final test target run? |
|
|
Which inner validation design is used for model selection? |
|
|
How many time blocks are used for tail-block validation? |
|
|
How large is each time block? |
|
|
How many chronological CV folds are used? |
|
|
How many randomly assigned CV folds are used? |
|
|
How often is the model refit? |
positive integer or pandas offset |
|
How often are hyperparameters reselected? |
positive integer or pandas offset |
|
Should retuning happen only when the model is refit? |
|
|
Can skipped retune origins reuse the previous selected parameters? |
|
|
Where may preprocessing, feature engineering, or model selection fit state? |
|
|
Validation choices have explicit time-order semantics:
Function |
Design |
Typical use |
|---|---|---|
|
One final holdout block inside the estimation sample. |
Simple holdout model selection. |
|
Pseudo-out-of-sample one-step tail splits. |
Recursive historical model selection with many tail origins. |
|
Expanding inner training sample and forward validation blocks. |
Walk-forward validation inside each estimation window. |
|
Several consecutive tail time blocks. |
Time-block model selection over recent history. |
|
Chronological blocked folds with only past data used for training. |
Time-aware CV. This is not random iid k-fold. |
|
Randomly assigned iid-style folds over the estimation sample. |
Paper replication when the original study explicitly used random folds. This is not time-safe validation. |
window = mf.window.spec(
estimation=mf.window.estimation_rolling(
size=120,
embargo=1,
retrain_every="12ME",
),
val=mf.window.val_expanding(
min_train_size=80,
horizon=12,
step=1,
retune_every="12ME",
retune_on_retrain=True,
reuse_params=True,
),
test=mf.window.test_origins(
first_origin="2000-01-31",
last_origin="2023-12-31",
horizon=12,
step="1ME",
),
alignment=mf.window.alignment_drop_incomplete(),
)
Rolling estimation with time-block validation:
window = mf.window.spec(
estimation=mf.window.estimation_rolling(
size=120,
min_size=80,
embargo=1,
retrain_every="12ME",
),
val=mf.window.val_rolling_blocks(
n_blocks=4,
block_size=12,
embargo=1,
retune_every="12ME",
retune_on_retrain=True,
reuse_params=True,
),
test=mf.window.test_origins(
first_origin="2000-01-31",
last_origin="2023-12-31",
horizon=12,
step="1ME",
),
)
Expanding estimation with chronological blocked CV:
window = mf.window.spec(
estimation=mf.window.estimation_expanding(
min_size=120,
embargo=1,
retrain_every=1,
),
val=mf.window.val_blocked_kfold(
n_splits=5,
embargo=1,
retune_every=1,
),
test=mf.window.test_origins(
first_origin="2000-01-31",
last_origin="2023-12-31",
horizon=4,
step="1QE",
),
)
Public Functions#
Task |
Functions |
|---|---|
Compose full window |
|
Build from common cutoffs |
|
Configure estimation |
|
Configure val |
|
Configure test |
|
Configure alignment |
|
Configure runner stage timing |
|
Shortcut windows |
|
Inspect windows |
|
Runner handoff |
|
Low-level split generators |
|
StagePolicy#
macroforecast.window.stage_policy(
scope="fit_window",
update="every_origin",
reference_start=None,
reference_end=None,
apply_to=("fit", "test"),
metadata=None,
)
StagePolicy is the runner-facing timing rule for stateful stages. It is used
as preprocessing_policy, feature_policy, and selection_policy in
macroforecast.forecasting.run(...).
Scope |
Meaning |
|---|---|
|
Fit the stage on the complete panel once. |
|
Fit the stage on rows available by each origin. |
|
Fit the stage only on the model fit window. |
|
Fit the stage on a fixed reference period and reuse that state. |
|
Use a user selector callable to choose the allowed labels. Build this with |
update controls how often a runner refits stateful preprocessing or feature
engineering stages. Current accepted values are:
Update |
Meaning |
|---|---|
|
Refit the stage at every test origin. |
|
Refit when the window row has |
|
Fit once at the first origin and reuse the fitted state. |
Positive integer |
Refit every N origins. |
Pandas date offset string, such as |
Refit when the current origin is at least the offset after the last update. |
Selection retuning follows the validation window’s retune_every setting. The
stage update field is mainly for fitted preprocessing state, fixed PCA
loadings, and other feature-engineering states.
stage_index(index, item, policy) and stage_panel(panel, item, policy) are
the low-level handoff helpers used by runners. They resolve the exact rows
allowed by a policy for an origin item returned by WindowSpec.iter_origins().
This keeps policy-to-index logic in macroforecast.window, not in
preprocessing, feature engineering, model, or model selection code.
feature_policy = mf.window.stage_policy(
"fixed_reference",
reference_start="2000-01-31",
reference_end="2019-12-31",
update="never",
)
custom_stage_policy#
Build a StagePolicy whose rows are selected by user code.
macroforecast.window.custom_stage_policy(
selector,
*,
update="every_origin",
apply_to=("fit", "test"),
metadata=None,
) -> StagePolicy
The selector receives the full index and the current origin item:
selector(index: pandas.Index, *, item: dict, policy: StagePolicy)
It may return:
Return type |
Meaning |
|---|---|
Boolean |
Mask over the full index. |
|
Positional slice into the full index. |
Integer positions |
Positional labels from the full index. |
Index labels |
Labels to keep. Every requested label must exist in the supplied index. Duplicate requested labels raise. |
The selected labels must not be empty. Missing labels and duplicate label
requests raise instead of being silently dropped. The runner stores the policy under
ForecastResult.metadata["stage_policies"]; the callable itself is recorded by
name in metadata.
def last_half_of_fit(index, *, item, policy):
fit_idx = item["fit_idx"]
return fit_idx[len(fit_idx) // 2 :]
result = mf.forecasting.run(
panel,
"ridge",
window=window,
features=features,
model_selection_policy=mf.window.custom_stage_policy(last_half_of_fit),
)
WindowSpec#
macroforecast.window.WindowSpec(
method="expanding",
estimation=EstimationWindow(...),
val=ValWindow(...),
test=TestWindow(...),
alignment=AlignmentWindow(...),
validation_size=None,
validation_ratio=0.2,
min_train_size=None,
n_splits=5,
step=1,
horizon=1,
embargo=0,
metadata=None,
)
The explicit component fields are the preferred API. The scalar validation fields remain to preserve the older model-selection split behavior.
Output methods:
Method |
Output |
Meaning |
|---|---|---|
|
list of inner train/val integer positions |
Validation splits for model-parameter selection. |
|
DataFrame |
Inspectable inner train/val split ranges. |
|
DataFrame |
Combined estimation/val/test execution plan. |
|
DataFrame |
Test-origin rows with estimation, fit, and test ranges. |
|
list of train/val positions |
Absolute-position inner validation splits for one test origin. |
|
iterator of dicts |
Origin metadata plus absolute |
|
iterator of dicts |
Same origin metadata plus sliced |
|
dict |
User-facing validation report with |
|
Series[bool] |
Dates included in the final test region. |
|
DataFrame or |
Feature/target index alignment. |
|
dict |
JSON-ready metadata. |
origins(index) returns:
Column |
Meaning |
|---|---|
|
Test origin label and position. |
|
Full pre-test sample available at that origin. |
|
Sample actually used by the current fitted model. This can lag |
|
Test label range produced at the origin. |
|
Integer positions. |
|
Window sizes and test-origin movement metadata. |
|
Whether this origin refits the model and the refit group id. |
|
Refit cadence metadata and estimation-window mode. |
plan(index) adds:
Column |
Meaning |
|---|---|
|
Inner validation splitter used when retuning. |
|
Whether hyperparameters are retuned at this origin and the retune group id. |
|
Hyperparameter retuning cadence metadata. |
|
Whether scheduled retunes are allowed only at retrain origins. |
|
Whether non-retune origins may reuse the last selected parameters. |
|
Label range of the estimation sample used for the active model-parameter selection run. Non-retune origins reuse the previous range. |
|
Integer positions and length for the active model-selection sample. |
|
Number of inner train/val splits evaluated at this origin. Zero when |
|
Label range covered by the validation folds at retune origins. |
|
Integer positions for the validation-fold label range. |
All methods that consume an index require unique, monotonic increasing labels. This keeps time order explicit and avoids silent reordering.
retrain_every and retune_every are separate cadences:
retrain_everycontrols when model coefficients are refit.retune_everycontrols when hyperparameters are reselected.Positive integers count emitted test origins.
retrain_every=12means every twelfth emitted origin, regardless of the calendar distance between labels.Pandas offsets use calendar time.
retrain_every="12ME"means the first emitted origin retrains, then the next origin on or after last retrain date plus 12 month ends retrains.Calendar
retrain_everyandretune_everyrequire aDatetimeIndex.With
retune_on_retrain=True, retuning is allowed only at origins that also retrain.With
reuse_params=True, skipped retune origins reuse the last selected parameters.With
reuse_params=False,validate(index)requires every emitted origin to retune.
TestWindow.step can be either row-count based or calendar based:
step=1means every emitted observation in the supplied index.step=3means every third emitted observation.step="1ME"means one month-end calendar move between test origins.step="1QE"means one quarter-end calendar move.step=pd.DateOffset(months=3)or a pandas offset object such aspd.offsets.MonthEnd(3)is also accepted.
Calendar/date-offset steps require a DatetimeIndex. If the offset lands
between two available labels, the next available label is used. For example, on
an irregular monthly panel, a target date of May 29 moves to the first available
index label on or after May 29. This calendar option applies to final test
origins only; ValWindow.step and the low-level validation splitters remain
row-count based because they operate inside an already selected estimation
sample.
Current row-count-only settings:
Setting |
Meaning |
|---|---|
|
Number of rows in the final test horizon. |
|
Number of rows between estimation end and test origin. |
|
Row-count movement between inner validation splits. |
|
Row-count length of each inner validation target block. |
Low-level splitters |
All low-level splitters operate on integer positions. |
Irregular calendar example:
idx = pd.date_range("2000-01-31", periods=18, freq="ME").delete([7, 10, 14])
X = pd.DataFrame({"x": range(len(idx))}, index=idx)
window = mf.window.spec(
estimation=mf.window.estimation_expanding(
min_size=3,
retrain_every="6ME",
),
val=mf.window.val_last_block(
size=2,
retune_every="3ME",
retune_on_retrain=False,
),
test=mf.window.test_origins(
first_origin=idx[4],
last_origin=idx[-1],
horizon=1,
step="3ME",
),
)
plan = window.plan(X.index)
plan[[
"origin",
"test_step",
"retrain",
"retrain_cadence",
"retune",
"retune_cadence",
]]
If idx[4] + 3ME lands on a missing label, the next available index label is
used. The cadence metadata columns make the effective runner plan auditable
without reopening the original WindowSpec.
plan = window.plan(X.index)
report = window.validate(X.index)
for origin in window.iter_slices(X, y):
X_fit = origin["X_fit"]
y_fit = origin["y_fit"]
X_test = origin["X_test"]
if origin["row"]["retune"]:
inner_splits = origin["val_splits"]
Minimal runner loop:
selected_params = None
fit = None
for origin in window.iter_slices(X, y):
row = origin["row"]
if row["retune"]:
selected_params = your_selection_function(
origin["X_fit"],
origin["y_fit"],
val_splits=origin["val_splits"],
)
if row["retrain"]:
fit = your_model_function(origin["X_fit"], origin["y_fit"], **selected_params)
prediction = fit.predict(origin["X_test"])
from_cutoffs#
macroforecast.window.from_cutoffs(
test_start,
test_end=None,
estimation_start=None,
mode="expanding",
estimation_size=None,
estimation_min_size=None,
embargo=0,
retrain_every=1,
val_method="last_block",
val_size=None,
val_ratio=0.2,
val_min_train_size=None,
val_n_splits=5,
val_random_state=None,
val_horizon=None,
val_step=1,
val_embargo=None,
retune_every=1,
retune_on_retrain=True,
reuse_params=True,
horizon=1,
step=1,
)
This helper builds the same WindowSpec from common cutoff choices.
window = mf.window.from_cutoffs(
estimation_start="1960-01-31",
test_start="2000-01-31",
test_end="2023-12-31",
mode="rolling",
estimation_size=120,
val_method="last_block",
val_size=24,
horizon=12,
retrain_every="12ME",
retune_every="12ME",
retune_on_retrain=True,
reuse_params=True,
step="1ME",
)
Components#
EstimationWindow#
macroforecast.window.EstimationWindow(
mode="expanding",
start=None,
end=None,
min_size=None,
size=None,
embargo=0,
retrain_every=1,
)
Builders:
mf.window.estimation_expanding(
start=None,
min_size=None,
embargo=0,
retrain_every=1,
)
mf.window.estimation_rolling(
start=None,
size=120,
min_size=None,
embargo=0,
retrain_every=1,
)
mf.window.estimation_fixed(
start=None,
end=None,
min_size=None,
embargo=0,
retrain_every=1,
)
Input:
Argument |
Meaning |
|---|---|
|
|
|
First allowed estimation label or integer position. |
|
Last allowed estimation label or integer position for fixed bounds. |
|
Minimum estimation observations required before an origin is emitted. |
|
Rolling estimation length. Required for |
|
Gap between the last estimation observation and the test origin. |
|
Refit cadence. Positive integers count emitted test origins; pandas offset strings or |
ValWindow#
macroforecast.window.ValWindow(
method="expanding",
size=None,
ratio=0.2,
min_train_size=None,
n_splits=5,
random_state=None,
horizon=1,
step=1,
embargo=None,
retune_every=1,
retune_on_retrain=True,
reuse_params=True,
)
Builders:
mf.window.val_last_block(
size=None,
ratio=0.2,
embargo=None,
retune_every=1,
retune_on_retrain=True,
reuse_params=True,
)
mf.window.val_poos(
size=None,
ratio=0.25,
embargo=None,
retune_every=1,
retune_on_retrain=True,
reuse_params=True,
)
mf.window.val_expanding(
min_train_size=None,
step=1,
horizon=1,
embargo=None,
retune_every=1,
retune_on_retrain=True,
reuse_params=True,
)
mf.window.val_rolling_blocks(
n_blocks=3,
block_size=None,
embargo=None,
retune_every=1,
retune_on_retrain=True,
reuse_params=True,
)
mf.window.val_blocked_kfold(
n_splits=5,
embargo=None,
retune_every=1,
retune_on_retrain=True,
reuse_params=True,
)
mf.window.val_random_kfold(
n_splits=5,
random_state=0,
retune_every=1,
retune_on_retrain=True,
reuse_params=True,
)
Input:
Argument |
Meaning |
|---|---|
|
Validation splitter: |
|
Explicit validation size for holdout-style splitters. |
|
Validation ratio when |
|
Minimum inner training size for expanding validation inside the estimation window. |
|
Number of validation folds or blocks. |
|
Seed for |
|
Validation target length. Defaults to one-step validation. |
|
Validation split movement. |
|
Validation-specific embargo. If absent, estimation embargo is used. |
|
Hyperparameter retuning cadence. Positive integers count emitted test origins; pandas offset strings or |
|
If |
|
If |
TestWindow#
macroforecast.window.TestWindow(
first_origin=None,
last_origin=None,
horizon=1,
step=1,
drop_incomplete=True,
exclude=(),
)
Builder:
mf.window.test_origins(
first_origin=None,
last_origin=None,
horizon=1,
step=1,
drop_incomplete=True,
exclude=(),
)
Input:
Argument |
Meaning |
|---|---|
|
First final-test origin label or integer position. |
|
Last final-test origin label or integer position. |
|
Test horizon length in rows. |
|
Origin movement. Positive integers move by row count. Pandas offset strings or |
|
Drop origins whose full horizon exceeds the available index. |
|
Sequence of |
first_origin and last_origin are forecast-origin labels, not realized
target-date labels. With a monthly index and step=1, the test origin moves one
row at a time, so h-step forecasts overlap in the usual macro-forecasting
sense. The horizon still controls target availability: an origin t is
evaluable only when t + horizon is in the supplied index. If the panel ends at
2017-12 and horizon=24, the last evaluable origin is 2015-12; origins in
2016 and 2017 have no realized 24-month-ahead actual.
With drop_incomplete=True, those tail origins are removed before the runner
fits/evaluates them. A calendar block can therefore be empty for long horizons
even when step=1; this is normal for evaluation runs and prevents RMSE/MAE
from using forecasts without realized actuals. For future-only forecasting,
build a forecast-only/scenario panel separately instead of scoring those tail
origins.
AlignmentWindow#
macroforecast.window.AlignmentWindow(
join="inner",
drop_missing=True,
require_full_horizon=True,
)
Builders:
mf.window.alignment_drop_incomplete(join="inner", require_full_horizon=True)
mf.window.alignment_keep_missing(join="inner", require_full_horizon=True)
alignment_drop_incomplete() removes rows with missing features or missing
targets. alignment_keep_missing() keeps missing feature rows after index
alignment, but with require_full_horizon=True it still drops rows whose target
horizon is incomplete.
Shortcuts#
Shortcuts create a full WindowSpec and still support model-selection-style use.
last_block#
macroforecast.window.last_block(validation_size=None, validation_ratio=0.2, embargo=0)
One final validation block.
poos#
macroforecast.window.poos(validation_size=None, validation_ratio=0.25, embargo=0)
Pseudo-out-of-sample one-step tail validation splits.
expanding#
macroforecast.window.expanding(min_train_size=None, step=1, horizon=1, embargo=0)
Expanding inner-train window with forward validation blocks.
rolling_blocks#
macroforecast.window.rolling_blocks(n_blocks=3, block_size=None, embargo=0)
Consecutive validation blocks over the sample tail.
blocked_kfold#
macroforecast.window.blocked_kfold(n_splits=5, embargo=0)
Chronological blocked folds with past-only training.
random_kfold#
macroforecast.window.random_kfold(n_splits=5, random_state=0)
Randomly assigned iid-style folds. Each fold trains on all non-validation rows,
so the training set can contain rows later than validation rows. Use this only
to reproduce studies whose appendix or code explicitly used random folds. For
ordinary macro forecasting, prefer blocked_kfold, poos, expanding, or
other time-aware validation designs.
Low-Level Splitters#
Low-level splitters return iterators of (train_idx, val_idx).
macroforecast.window.last_block_split(n_samples, validation_size=None, validation_ratio=0.2, embargo=0)
macroforecast.window.poos_split(n_samples, validation_size=None, validation_ratio=0.25, embargo=0)
macroforecast.window.expanding_split(n_samples, min_train_size=None, step=1, horizon=1, embargo=0)
macroforecast.window.rolling_blocks_split(n_samples, n_blocks=3, block_size=None, embargo=0)
macroforecast.window.blocked_kfold_split(n_samples, n_splits=5, embargo=0)
macroforecast.window.random_kfold_split(n_samples, n_splits=5, random_state=0)
split_table#
macroforecast.window.split_table(
window,
n_samples,
*,
index=None,
validation_size=None,
validation_ratio=0.2,
min_train_size=None,
n_splits=5,
step=1,
horizon=1,
random_state=None,
embargo=0,
)
Output columns:
Column |
Meaning |
|---|---|
|
Split id. |
|
Split sizes. |
|
Labels for the training range. |
|
Labels for the validation range. |
|
Integer positions for the same ranges. |
Aliases#
normalize_window_name() and resolve_window() accept these aliases.
resolve_stage_policy() applies the same normalization idea to stage policies,
accepting a StagePolicy, string scope, mapping, or None.
Alias |
Canonical |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|