Look-ahead bias in time-series ML: an audit checklist
Two repositories implement six look-ahead checks for an intraday ML pipeline, from resample alignment to a purge of walk-forward folds. On synthetic bars the correlation screen flagged one of the two planted leaks, and the truncation test flagged both. A daily-merge fix as first imported into claude-skills shifted the date key with the values and left the look-ahead in place.
- Published
- 2026-10-28 (draft)
- Topic
- Quant trading research
- Revision
7e4ec74
Scope: two repositories, synthetic commands
llm-trading-lab holds the audit code in src/leakage_audit/ and session-aligned walk-forward folds in src/walkforward/. Its project page covers the rest of the repository. claude-skills holds ts-ml-lookahead-audit, a Claude Code skill with a correlation audit and leak-free versions of a bucket resample and a daily merge. Claude loads the skill while it builds or reviews a time-series dataset, and user-invocable: false keeps it out of the slash-command menu.
Every result in this article is the output of a command run on synthetic random-walk bars. llm-trading-lab publishes no result tables, so the article reports no measurement on market data.
The checklist
| # | Check | What it catches | How to run it | Source |
|---|---|---|---|---|
| 1 | Resample alignment | A bucketed feature that gives every bar the bucket's last value, so a bar reads closes recorded after it. | Use the previous completed bucket's last value per session. uv run python examples/lookahead_demo.py in claude-skills counts bars that read a later close. | src/leakage_audit/patterns.py; skill SKILL.md, Resample bucket fix |
| 2 | Correlation screen | A numeric feature whose absolute Pearson r with a short-horizon forward return exceeds 0.15 on a 5,000-row sample. A leak with a weak linear signal passes it. | python -m leakage_audit --data bars.csv, section Correlation screen; lookahead_audit(df, "fwd_return_5m") in the skill. | src/leakage_audit/correlation.py; skill SKILL.md, Detection |
| 3 | Truncation test | A feature whose value at row t changes when the rows after t are removed. | Same command, section Truncation test. Features are recomputed on the first rows at 5 cut points and compared with the full-history values. | src/leakage_audit/truncation.py |
| 4 | Triple-barrier label audit | A label window that includes the labeled bar or is shorter than max_bars at the end of the data. The window length is the forward reach that check 6 has to purge. | triple_barrier_labels(bars, upper_pct, lower_pct, max_bars); pytest tests/leakage_audit -k triple_barrier. | src/leakage_audit/barrier.py lines 20 to 42 |
| 5 | Barrier balance | A label configuration whose upper, lower or timeout share leaves its band in some year. It does not detect look-ahead. | balance_by_year(labels, timestamps, HARD_GATE) or DIAGNOSTIC. The audit command reports the diagnostic and a 40% timeout limit. | src/leakage_audit/barrier.py lines 46 to 87 |
| 6 | Fold overlap and purge | Training rows whose forward target reads a price in the test fold. | Same command, section Fold overlap. --purge-bars 10 on leakage_audit and walkforward drops those rows. | src/leakage_audit/overlap.py; src/walkforward/folds.py |
With --fail-on-leak the audit command exits with status 1 when check 2, 3 or 6 has a finding, so it can stop a CI job. Barrier balance does not change the exit status. The correlation screen skips the target and any column that starts with fwd_, mae_, mfe_ or label_; the skill states that excursion columns (MFE, MAE) belong in targets only.
Where each check sits in the pipeline
flowchart TD
bars["1-minute OHLCV bars"] --> feat
subgraph feat["Feature build"]
direction LR
c1["1 Resample alignment
previous completed bucket"]
c2["2 Correlation screen
abs(r) above 0.15"]
c3["3 Truncation test
5 cut points"]
end
feat --> lab
subgraph lab["Targets and labels"]
direction LR
c4["4 Label audit
window t+1 to t+max_bars"]
c5["5 Barrier balance
shares by year"]
end
lab --> folds
subgraph folds["Walk-forward folds"]
c6["6 Fold overlap
purge the target horizon"]
end
folds --> fit["Fit and score each fold"]
fit --> gates["Out-of-sample gates"]
Checks 1 to 3 inspect features, checks 4 and 5 inspect the labels, and check 6 inspects how labels and folds meet. In llm-trading-lab a research candidate goes through the leakage audit before any gate. The Stage A gate measures expectancy only, and on synthetic data an entry family that peeks five bars ahead passes it.
Two planted leaks: correlation screen and truncation test
src/leakage_audit/patterns.py reproduces two look-ahead patterns found in an intraday dataset, each with its fix. A five-minute resample with transform("last") gives every bar in a bucket the bucket's closing value, so the 10:01 bar sees 10:04. A daily close joined on the bar's date lets the 09:30 bar see the 16:00 close. The skill gives two red flags for a model that reads features like these: above 65% accuracy on binary direction prediction for short-horizon returns, and accuracy that does not fall between in-sample and out-of-sample periods.
On the 17,550-bar synthetic sample (45 sessions, seed 0), python -m leakage_audit --data bars_1min.csv --plant-leaks reports:
| Planted feature | Correlation screen | Truncation test |
|---|---|---|
ret_to_5m_bucket_close | |r| 0.6370, flagged | Changes at 5 of 5 cut points, 10 rows |
ret_to_same_day_close | |r| 0.1424, not flagged | Changes at 5 of 5 cut points, 620 rows |
The same-day merge feature misses the 0.15 threshold by 0.0076. The 11 clean features score between 0.0003 and 0.0219. Inference: the leaked value is the return to the session close, which spans up to 389 bars, so the 5-bar target is a small part of it and the linear correlation stays low. The truncation test compares values directly and does not depend on correlation.
Resample alignment: which fix still reads a later close
The skill in claude-skills gives two fixes for the bucket pattern. bucket_last_safe shifts the leaky bucket value by one bucket's worth of rows within the trading day. bucket_last_safe_with_gaps takes the previous bucket's last value per day. examples/lookahead_demo.py runs the functions as written in SKILL.md and counts the bars whose feature is a close recorded after the bar.
On gapless 1-minute bars the leaky pattern reads a later close on 4 of every 5 bars. The row-count shift reads none on 1-minute bars with or without gaps. On 30-second bars a 5-row shift covers 2.5 minutes, lands inside the current bucket, and reads a later close on 40% of bars; SKILL.md states this limit beside the function. The previous-bucket variant reads none in all three conditions. llm-trading-lab's previous_bucket_last uses the same previous-bucket rule.
The skill's correlation audit, run on 10 synthetic sessions with two planted leaks, flags peek_3m (|r| 0.773) and bucket_gap_leaky (0.647). It passes the row-shift bucket feature (0.016) and the 1-minute return (0.006).
The daily-merge fix that left the look-ahead in place
The skill as first imported into claude-skills gave this fix for the same-day merge:
# RIGHT: shift daily data by one row before merging
daily_shifted = daily_data.copy()
for col in daily_shifted.columns:
daily_shifted[col] = daily_shifted[col].shift(1)
df.merge(daily_shifted, left_on="date", right_on="date")
The loop shifts every column, the date key included. Each date moves down one row together with its own value, and the merge then pairs every date with its own close. That is the leak the fix was meant to remove. The staging commit replaced it with a function that shifts the value columns only:
def merge_daily_safe(intraday, daily, on="date"):
daily = daily.sort_values(on)
value_cols = [c for c in daily.columns if c != on]
shifted = daily.assign(**{c: daily[c].shift(1) for c in value_cols})
return intraday.merge(shifted, on=on, how="left")
Two tests cover it. One asserts that the fixed merge gives the first date no value and each later date the previous date's close. The other keeps the original pattern as a negative case: it merges a frame shifted in every column and asserts that the result equals the leaky merge on the first three days. The function assumes one daily row per trading day. llm-trading-lab's merge_daily(prior_day=True) shifts the value columns only.
Inference: a review that looks for a shift before the merge passes the original code. A test that compares the merged value with the previous day's value fails it, and so does the truncation test when the daily series is built from the bars being cut.
Fold overlap and purge
By default add_forward_returns computes targets across session boundaries, as the original run did, so the last bars of a session take their target from the next session's prices. In an expanding walk-forward split the last training session is followed by the first test session. With a 10-bar target the last 10 training rows of each fold read test-fold prices. The synthetic sample shows this count in every fold, and --purge-bars 10 removes it:
| Fold | Train rows, no purge | Rows reading the test fold | Train rows, purge 10 | Rows reading the test fold |
|---|---|---|---|---|
| 0 | 11,700 | 10 | 11,690 | 0 |
| 1 | 13,650 | 10 | 13,640 | 0 |
| 2 | 15,600 | 10 | 15,590 | 0 |
The purge has to cover the longest forward reach among the targets. The audit command measures overlap at 10 bars, the walk-forward default horizon. It also builds triple-barrier labels with a 30-bar window for the balance check. Inference: a model trained on those labels would need a 30-bar purge; the repository trains no model on them.
Triple-barrier labeler and barrier balance
The labeler starts each window at the next bar, gives the lower label when one bar touches both barriers, and leaves the last max_bars bars unlabeled. A unit test checks the window and the unlabeled tail on a 6-bar frame. Two rule sets then apply to the label shares. The hard gate checks the upper and lower shares against 5% to 80% in every year and fails the audit when one falls outside. The diagnostic checks all three shares against 10% to 70% and only warns. A separate chop rule warns when the timeout share over the whole sample exceeds 40%.
On the synthetic sample the audit command labels with 0.3% barriers and a 30-bar window and reports one year: upper 37.0%, lower 36.4%, timeout 26.5%, PASS, and 26.5% against the 40% chop limit.
Limits of these results
- Synthetic data. Every command output in this article, both planted-leak runs, the fold overlap counts and the resample measurements in claude-skills come from random-walk bars. They show which patterns read the future. They do not measure the size of the effect on a real model.
- Not measured. The effect of either leak, or of a missing purge, on a model trained on market data. llm-trading-lab publishes no result tables and computes features from OHLCV only.
- Correlation thresholds. The skill calls 0.15 and 0.05 rules of thumb. A leak whose effect is nonlinear can pass a Pearson screen, and the screen inspects one feature at a time against one target. The planted same-day merge leak passed it at 0.1424.
- Truncation scope. The test needs a feature function that can be rerun on a prefix of the data. In the demonstration the daily closes are derived from the bars being cut. Inference from the code: a daily file loaded from outside the bars is not cut with them, so its rows published after the cut have to be dropped for the test to see a same-day merge.
- Label checks. Barrier balance and the label audit test label quality and the label window. They do not test features. The repository labeler uses fixed-percent barriers only.