# Handoff / resume note — S2S-FM-VN

**Last updated:** 2026-09-23. **Stop reason:** waiting on the user to configure &
test the ECDS credential. Everything runnable without ECMWF data is done and green.

## Where we are in the phase plan

- **Phase 0 (audit + target prototype): COMPLETE and gated-approved.**
- **Gate decisions D1–D7 received (`My_swers.md`) and applied.**
- **Phase 1 (minimal ECDS retrieval → GRIB inspection → schema freeze → preprocessing):
  BLOCKED at step 1** on the ECDS credential. Metadata-independent scaffolding for
  Phase 1 is built and tested.

## Single blocker to resume

`~/.cdsapirc` still points at the **old Copernicus CDS** (`cds.climate.copernicus.eu`),
not **ECDS** (`ecds.ecmwf.int`). The minimal retrieval script refuses to run until
this is fixed (verified: `outcome: blocked_wrong_endpoint`).

### What the user must do (local; no secrets in chat)

1. Configure the ECDS credential. **Replace, don't add** — url+key are a matched
   pair and the CDS key won't authenticate against ECDS. Two options:
   - **A (overwrite):** put in `~/.cdsapirc`:
     ```
     url: https://ecds.ecmwf.int/api
     key: <ECDS token>
     ```
   - **B (keep CDS too, recommended):** create `~/.ecdsapirc` with the ECDS
     url+key, then `export CDSAPI_RC=~/.ecdsapirc`. The retrieval script already
     honors `CDSAPI_RC`/`CDSAPI_URL`.
   Copy exact url+key from the ECDS **"Show API request code" / API-token** page.
2. Put a real minimal request into `config/ecds_minimal_request.json` (copy from
   `config/ecds_minimal_request.json.template`, but use ECDS-generated values so
   `model_version_date`/`hindcast_year` are valid). Keep it minimal: cf, 1.5°, one
   init, 1–2 lead hours, one surface var (2t).
3. Test auth with the smallest possible request first.

## Resume command

```bash
cd /home/tanpv/Progs/claude/FM
# (Option B only) export CDSAPI_RC=~/.ecdsapirc
python3 scripts/01_minimal_ecmwf_retrieval.py     # -> metadata/ecmwf_minimal_retrieval_status.json
```
Then tell the agent **`run`**. Next agent steps (agreed sequence, do NOT skip):
1. inspect real GRIB (`grib_ls`/`grib_dump`/`cdo sinfon` via `data.interfaces.inspect_grib`)
2. document actual archive structure → fill `TBC` rows in
   `metadata/ecmwf_reforecast_inventory.csv` + `docs/ecmwf_s2s_inventory.md`
3. **freeze** `docs/canonical_schema.md`
4. implement Phase 1 preprocessing against the frozen schema
5. run it on the minimal sample; verify temporal alignment vs VnGAD (`utils.timealign`)
6. **STOP before the 2003–2022 bulk download** (explicit user instruction)

## Done & verified this project (files that show it)

- Fixed P90 + EHD threshold, 2003–2022, cyclic ±15 d, 29-Feb pooled with 28-Feb →
  `data/processed/{P90_2003_2022.nc,EHD_THRESHOLD_2003_2022.nc}`. Sanity: mean daily
  EHD freq 0.101 → `outputs/metrics/target_event_frequency.csv`.
- Target/event code (P90, EHD, weekly aggregates, episodes, cross-week onset/active/
  days split, censoring) → `src/targets/`. Tests: `tests/test_targets.py` **11/11**.
- Temporal alignment (valid=init+lead by date; UTC+7 local-day; align-by-date;
  partial-LT0) → `src/utils/timealign.py`. Tests: `tests/test_timealign.py` **6/6**.
- Phase 1 scaffolding (metadata-independent): `src/utils/{device,logdir,config}.py`,
  `src/data/{interfaces,observations}.py`. Concrete GRIB preprocessing =
  `NotImplementedError` until schema freeze.
- D3 grid/mask alignment **verified PASS** → `metadata/mask_alignment_report.json`.
- D7 unassigned cells → `region=0` field `data/processed/region_codes_vngad.nc`,
  `metadata/unassigned_region_cells.csv`, `outputs/figures/unassigned_region_cells.png`.
- Gate report: `docs/PHASE0_AUDIT.md`, `outputs/PHASE0_AUDIT.html`.

## Environment facts (so next session doesn't re-probe)

- Python 3.9.5 (`/home/hapt/miniconda3`); numpy/xarray/netCDF4/cdsapi/matplotlib/pyyaml
  present; **cfgrib NOT installed** (use eccodes CLI + cdo per D6). `ecmwf-api-client`
  present but **legacy token EXPIRED** and NOT used.
- **Cold imports over the NFS home are slow** (xarray/pandas can take 1–2 min): run
  data/xarray work as ONE background job, not many short calls. `sleep` in the
  foreground is blocked — use `run_in_background`.
- Slurm `workq`, 11×20 CPU, **no GPU** (GRES=null). D4: CPU-sized, device-agnostic
  prototype approved; GPU only for full pretraining/CV.
- `/work` 35 TB free (bulk target); `/home` ~400 GB.
- Data originals are read-only symlinks in `data/` + `/work/projects/G09/QG.25.196/OBS/`.
  Never modify/move them. Target grid = VnGAD (61×31, 102–109.5°E/8.5–23.5°N);
  regions = `mask7reg_VnGAD` (codes 1–7).

## Open decision items still pending (non-blocking except D1/D2)

- **D1/D2:** ECDS credential + confirmed request syntax (the current blocker).
- **D6:** optionally install cfgrib later (not required).
- After minimal retrieval: confirm initializations/yr, model-cycle consistency across
  2003–2022, daily-max 2t aggregation window & UTC-vs-local mapping — all from the
  real GRIB, before proposing the bulk download.

Full rationale for every decision is in `docs/decision_log.md` (newest first).