Mitchell Foutz All writing

Look-ahead bias in time-series ML: an audit checklist

Two repositories implement six look-ahead checks for an intraday ML pipeline, from resample alignment to a purge of walk-forward folds. On synthetic bars the correlation screen flagged one of the two planted leaks, and the truncation test flagged both. A daily-merge fix as first imported into claude-skills shifted the date key with the values and left the look-ahead in place.

Published
2026-10-28 (draft)
Topic
Quant trading research
Source
darblinko/llm-trading-lab
Revision
7e4ec74

Scope: two repositories, synthetic commands

llm-trading-lab holds the audit code in src/leakage_audit/ and session-aligned walk-forward folds in src/walkforward/. Its project page covers the rest of the repository. claude-skills holds ts-ml-lookahead-audit, a Claude Code skill with a correlation audit and leak-free versions of a bucket resample and a daily merge. Claude loads the skill while it builds or reviews a time-series dataset, and user-invocable: false keeps it out of the slash-command menu.

Every result in this article is the output of a command run on synthetic random-walk bars. llm-trading-lab publishes no result tables, so the article reports no measurement on market data.

The checklist

#CheckWhat it catchesHow to run itSource
1Resample alignmentA bucketed feature that gives every bar the bucket's last value, so a bar reads closes recorded after it.Use the previous completed bucket's last value per session. uv run python examples/lookahead_demo.py in claude-skills counts bars that read a later close.src/leakage_audit/patterns.py; skill SKILL.md, Resample bucket fix
2Correlation screenA numeric feature whose absolute Pearson r with a short-horizon forward return exceeds 0.15 on a 5,000-row sample. A leak with a weak linear signal passes it.python -m leakage_audit --data bars.csv, section Correlation screen; lookahead_audit(df, "fwd_return_5m") in the skill.src/leakage_audit/correlation.py; skill SKILL.md, Detection
3Truncation testA feature whose value at row t changes when the rows after t are removed.Same command, section Truncation test. Features are recomputed on the first rows at 5 cut points and compared with the full-history values.src/leakage_audit/truncation.py
4Triple-barrier label auditA label window that includes the labeled bar or is shorter than max_bars at the end of the data. The window length is the forward reach that check 6 has to purge.triple_barrier_labels(bars, upper_pct, lower_pct, max_bars); pytest tests/leakage_audit -k triple_barrier.src/leakage_audit/barrier.py lines 20 to 42
5Barrier balanceA label configuration whose upper, lower or timeout share leaves its band in some year. It does not detect look-ahead.balance_by_year(labels, timestamps, HARD_GATE) or DIAGNOSTIC. The audit command reports the diagnostic and a 40% timeout limit.src/leakage_audit/barrier.py lines 46 to 87
6Fold overlap and purgeTraining rows whose forward target reads a price in the test fold.Same command, section Fold overlap. --purge-bars 10 on leakage_audit and walkforward drops those rows.src/leakage_audit/overlap.py; src/walkforward/folds.py

With --fail-on-leak the audit command exits with status 1 when check 2, 3 or 6 has a finding, so it can stop a CI job. Barrier balance does not change the exit status. The correlation screen skips the target and any column that starts with fwd_, mae_, mfe_ or label_; the skill states that excursion columns (MFE, MAE) belong in targets only.

Where each check sits in the pipeline

flowchart TD
  bars["1-minute OHLCV bars"] --> feat
  subgraph feat["Feature build"]
    direction LR
    c1["1 Resample alignment
previous completed bucket"] c2["2 Correlation screen
abs(r) above 0.15"] c3["3 Truncation test
5 cut points"] end feat --> lab subgraph lab["Targets and labels"] direction LR c4["4 Label audit
window t+1 to t+max_bars"] c5["5 Barrier balance
shares by year"] end lab --> folds subgraph folds["Walk-forward folds"] c6["6 Fold overlap
purge the target horizon"] end folds --> fit["Fit and score each fold"] fit --> gates["Out-of-sample gates"]

Checks 1 to 3 inspect features, checks 4 and 5 inspect the labels, and check 6 inspects how labels and folds meet. In llm-trading-lab a research candidate goes through the leakage audit before any gate. The Stage A gate measures expectancy only, and on synthetic data an entry family that peeks five bars ahead passes it.

Two planted leaks: correlation screen and truncation test

src/leakage_audit/patterns.py reproduces two look-ahead patterns found in an intraday dataset, each with its fix. A five-minute resample with transform("last") gives every bar in a bucket the bucket's closing value, so the 10:01 bar sees 10:04. A daily close joined on the bar's date lets the 09:30 bar see the 16:00 close. The skill gives two red flags for a model that reads features like these: above 65% accuracy on binary direction prediction for short-horizon returns, and accuracy that does not fall between in-sample and out-of-sample periods.

On the 17,550-bar synthetic sample (45 sessions, seed 0), python -m leakage_audit --data bars_1min.csv --plant-leaks reports:

Planted featureCorrelation screenTruncation test
ret_to_5m_bucket_close|r| 0.6370, flaggedChanges at 5 of 5 cut points, 10 rows
ret_to_same_day_close|r| 0.1424, not flaggedChanges at 5 of 5 cut points, 620 rows

The same-day merge feature misses the 0.15 threshold by 0.0076. The 11 clean features score between 0.0003 and 0.0219. Inference: the leaked value is the return to the session close, which spans up to 389 bars, so the 5-bar target is a small part of it and the linear correlation stays low. The truncation test compares values directly and does not depend on correlation.

Resample alignment: which fix still reads a later close

The skill in claude-skills gives two fixes for the bucket pattern. bucket_last_safe shifts the leaky bucket value by one bucket's worth of rows within the trading day. bucket_last_safe_with_gaps takes the previous bucket's last value per day. examples/lookahead_demo.py runs the functions as written in SKILL.md and counts the bars whose feature is a close recorded after the bar.

On gapless 1-minute bars the leaky pattern reads a later close on 4 of every 5 bars. The row-count shift reads none on 1-minute bars with or without gaps. On 30-second bars a 5-row shift covers 2.5 minutes, lands inside the current bucket, and reads a later close on 40% of bars; SKILL.md states this limit beside the function. The previous-bucket variant reads none in all three conditions. llm-trading-lab's previous_bucket_last uses the same previous-bucket rule.

The skill's correlation audit, run on 10 synthetic sessions with two planted leaks, flags peek_3m (|r| 0.773) and bucket_gap_leaky (0.647). It passes the row-shift bucket feature (0.016) and the 1-minute return (0.006).

The daily-merge fix that left the look-ahead in place

The skill as first imported into claude-skills gave this fix for the same-day merge:

# RIGHT: shift daily data by one row before merging
daily_shifted = daily_data.copy()
for col in daily_shifted.columns:
    daily_shifted[col] = daily_shifted[col].shift(1)
df.merge(daily_shifted, left_on="date", right_on="date")

The loop shifts every column, the date key included. Each date moves down one row together with its own value, and the merge then pairs every date with its own close. That is the leak the fix was meant to remove. The staging commit replaced it with a function that shifts the value columns only:

def merge_daily_safe(intraday, daily, on="date"):
    daily = daily.sort_values(on)
    value_cols = [c for c in daily.columns if c != on]
    shifted = daily.assign(**{c: daily[c].shift(1) for c in value_cols})
    return intraday.merge(shifted, on=on, how="left")

Two tests cover it. One asserts that the fixed merge gives the first date no value and each later date the previous date's close. The other keeps the original pattern as a negative case: it merges a frame shifted in every column and asserts that the result equals the leaky merge on the first three days. The function assumes one daily row per trading day. llm-trading-lab's merge_daily(prior_day=True) shifts the value columns only.

Inference: a review that looks for a shift before the merge passes the original code. A test that compares the merged value with the previous day's value fails it, and so does the truncation test when the daily series is built from the bars being cut.

Fold overlap and purge

By default add_forward_returns computes targets across session boundaries, as the original run did, so the last bars of a session take their target from the next session's prices. In an expanding walk-forward split the last training session is followed by the first test session. With a 10-bar target the last 10 training rows of each fold read test-fold prices. The synthetic sample shows this count in every fold, and --purge-bars 10 removes it:

FoldTrain rows, no purgeRows reading the test foldTrain rows, purge 10Rows reading the test fold
011,7001011,6900
113,6501013,6400
215,6001015,5900

The purge has to cover the longest forward reach among the targets. The audit command measures overlap at 10 bars, the walk-forward default horizon. It also builds triple-barrier labels with a 30-bar window for the balance check. Inference: a model trained on those labels would need a 30-bar purge; the repository trains no model on them.

Triple-barrier labeler and barrier balance

The labeler starts each window at the next bar, gives the lower label when one bar touches both barriers, and leaves the last max_bars bars unlabeled. A unit test checks the window and the unlabeled tail on a 6-bar frame. Two rule sets then apply to the label shares. The hard gate checks the upper and lower shares against 5% to 80% in every year and fails the audit when one falls outside. The diagnostic checks all three shares against 10% to 70% and only warns. A separate chop rule warns when the timeout share over the whole sample exceeds 40%.

On the synthetic sample the audit command labels with 0.3% barriers and a 30-bar window and reports one year: upper 37.0%, lower 36.4%, timeout 26.5%, PASS, and 26.5% against the 40% chop limit.

Limits of these results