# Full 2003–2022 reforecast retrieval plan (Phase 4)

Scheme validated (L16): EC model version **2023**; each of the **104** MM-DD dates
in `hdate_2023.txt` is one request bundling **hyear 2003–2022** (20 hindcast years).
Control forecast only, native **1.5°**, from `~/.cdsapirc_1` (ECDS). Downloader
`scripts/05_download_reforecasts.py` is resumable (skip-existing) with retry+backoff.

## Variable groups (by native step frequency + branch domain)

| group | branch / domain | vars | levels | leads | per-date size | 104-date total |
|---|---|---|---|---|---|---|
| `sfc_tx` | local (95–120E,5–30N) | `mx2t6` | sfc | **6-hourly** (184) | ~2.9 MB | ~0.30 GB |
| `sfc_dyn` | local | `2t, msl, 10u, 10v` | sfc | daily (47) | ~3 MB | ~0.31 GB |
| `pl_hi` | **large (40–180E,20S–50N)** | `t, gh, u, v, q` | 850/700/500/200 hPa | daily (47) | **~162 MB** | **~16.8 GB** |
| **total** | | | | | | **≈ 17.4 GB** |

Storage: 17.4 GB into `/work` (35 TB free) — fine. Est. wall time (one request at a
time): `sfc_tx` ~3.8 h, `sfc_dyn` ~3.8 h, `pl_hi` ~9.5 h → **~17 h**, resumable.

**Why the split:** the surface fields have different native frequencies — `mx2t6` is
6-hourly (needed for daily max), `msl`/`2t`/winds are archived daily; a single
6-hourly request silently dropped `2t` (only 1 step returned). Grouping by frequency
fixes this (L18).

## Commands

```bash
python3 scripts/05_download_reforecasts.py --group sfc_tx  --dates all
python3 scripts/05_download_reforecasts.py --group sfc_dyn --dates all
python3 scripts/05_download_reforecasts.py --group pl_hi   --dates all
```
Files: `data/raw/ecmwf_s2s_1p5deg/s2s_cf_<group>_<loc|ls>_mv2023-MM-DD_hy2003-2022.grib`.

## Assembly (scripts/08) — 104 files → one logical cube per variable

Standardize every field to **daily lead days LT0–LT45** on its native grid:
- `mx2t6` → daily-max Tx on the **local UTC+7 day** (via `daily_max_from_6h`);
- daily fields → the 24-h value per lead day.

Output per variable: dims `(model_version_date[104], hindcast_year[20], lead[46],
lat, lon)` — i.e. 104×20×46 ≈ **95 680** forecast time-slices per variable, matching
the expected full count. Predictors kept on their native coarse grids (local vs
large-scale); never regridded to 0.25° before the encoder.

## Not yet included (add on request)
SST + soil + flux/radiation surface fields (large-scale SST for ENSO); pressure
levels 1000/925/300; `vertical_velocity`. High-priority set above is the
prototype/base; expand once it trains.

## Gate
Bulk = 104 requests × 3 groups on the account over ~17 h. Launch order: `sfc_tx` →
`sfc_dyn` (small, target-side) → `pl_hi` (16.8 GB, large-scale). Do the full run
only with approval; the downloader is safe to stop/resume at any point.