================================================================================ PHAN TICH LUONG LAM VIEC TU CRONTAB CUA NAM (user: nampq) Lien quan den viec chay hai mo hinh RegCM va clWRF (Seasonal_FC / CFSv2) Nguoi phan tich: CLAUDE - Ngay: 2026-09-06 ================================================================================ -------------------------------------------------------------------------------- 0. CAC DONG CRONTAB -------------------------------------------------------------------------------- (1) 10 10 * * * cd /work/users/nampq/working/Proj_T.Tan/Operational/Seasonal_FC/CLWRF/CFS ; ./ope_download.sh >& cfs_log (2) 10 10 * * * cd /work/users/nampq/working/Proj_T.Tan/NCCB_562/forecast ; ./get_data_monthly.sh >& log (3) 0 9 * * * cd /work/users/nampq/working/Proj_T.Tan/QG.25.196/S2S_FC/data/CFS ; ./ope_download.sh >& cfs_log - Dong (1) la dong DUY NHAT khoi dong toan bo he thong chay 2 mo hinh RCM (clWRF + RegCM) cho bai toan Du bao mua nghiep vu (Operational Seasonal_FC). - Dong (2) NCCB_562/forecast/get_data_monthly.sh: chi TAI DU LIEU du bao thang (khong tich phan RegCM/clWRF) -> khong thuoc pham vi khoi phuc mo hinh. (Luu y: file hgt_domain_20km.nc dung cho hau xu ly clWRF nam trong project nay: .../NCCB_562/plot_domain/hgt_domain_20km.nc) - Dong (3) QG.25.196/S2S_FC/data/CFS/ope_download.sh: chi TAI du lieu CFS cho bai toan S2S (download-only, dung thredds_dl.py) -> khong tich phan mo hinh. => Chi tiet duoi day tap trung vao DONG (1): chuoi script chay clWRF va RegCM. -------------------------------------------------------------------------------- 1. THU MUC & CAU HINH CHUNG -------------------------------------------------------------------------------- ROOT (WORK_DIR) : /work/users/nampq/working/Proj_T.Tan/Operational configure_wrf : $ROOT/configure_wrf (module env + duong dan WRF/WPS/ARWpost) configure_regcm : $ROOT/configure_regcm (module env + duong dan RegCM bin) CLWRF dir : $ROOT/Seasonal_FC/CLWRF/CFS RegCM dir : $ROOT/Seasonal_FC/RegCM/CFS Ket qua cuoi (current): $ROOT/Seasonal_FC/data/current/{CLWRF,RegCM}/ (link 'done' cua moi mo hinh tro toi day) Cac duong dan trong configure_wrf: WRF_DIR = $ROOT/sorc/CLWRF_v4.3/WRF (real.exe, wrf.exe, run/*.TBL, RRTMG*, ozone*) WPS_DIR = $ROOT/sorc/CLWRF_v4.3/WPS (geogrid/metgrid/ungrib, *.TBL, Vtable.CFSR) ARWpost = $ROOT/sorc/CLWRF_v4.3/ARWpost/ARWpost.exe DATA_DIR = /home/nampq/data (CFS_Ope, WPS_GEOG) TOPO_DIR = /home/nampq/data/WPS_GEOG Cac duong dan trong configure_regcm: PATH += $ROOT/sorc/RegCM-4.9.5/bin (terrain/sst/icbc/regcmMPI/sigma2p ... CLM45) DATA_DIR = /home/nampq/data Cau hinh mo hinh (GIU NGUYEN, khong tao moi): - clWRF WPS : namelist.wps_org (link namelist.wps -> namelist.wps_org) - clWRF WRF : namelist.input_org - clWRF post: convert_nc/namelist.ARWpost_org - RegCM : namelist.in_org (mien 160(jx) x 150(iy), kz=18, idynamic=2 MM5 non-hydro, do phan giai ~20km, CLM4.5) - Cac file tinh RegCM: d01_DOMAIN000.nc, d01_CLM45_surface.nc, d01_LANDUSE, d01_TEXTURE - clWRF geo : geo_em.d01.nc -------------------------------------------------------------------------------- 2. DU LIEU DAU VAO: CFSv2 GRIB2 -------------------------------------------------------------------------------- Nguon : https://nomads.ncep.noaa.gov/pub/data/nccf/com/cfs/prod cfs.//6hrly_grib_01/ Luu : /home/nampq/data/CFS_Ope/00// File : flxf.01..grb2 (thong luong be mat -> SST/TMP:surface) pgbf.01..grb2 (muc ap suat -> TMP,RH,UGRD,VGRD,HGT) VALID cach nhau 6 gio, tu S_DATE den E_DATE (~7 thang; vd 769 cap). S_DATE = thoi diem khoi tao (INIT) = ten file dau (flxf dau tien) E_DATE = VALID cua file cuoi cung. Chu ky chay: cac ngay co dd thuoc {01,06,11,16,21,26} luc HH=00. -------------------------------------------------------------------------------- 3. SCRIPT DOWNLOAD (diem vao cron dong 1) -------------------------------------------------------------------------------- CLWRF/CFS/ope_download.sh: - Xoa *.log cu, dat url + odir=/home/nampq/data/CFS_Ope. - Lap 3 ngay gan nhat; neu dd thuoc {01,06,11,16,21,26} thi: curl liet ke thu muc 6hrly_grib_01, luu cfs..log wget -c tung file flxf*/pgbf*.grb2 ve $odir/00// - Cuoi cung goi ./autorun.sh (bat dau tich phan clWRF, roi RegCM). -------------------------------------------------------------------------------- 4. CHUOI CHAY clWRF (CLWRF/CFS/) -------------------------------------------------------------------------------- autorun.sh: - source ../../../configure_wrf ; idir=$DATA_DIR/CFS_Ope/00 - Voi moi thu muc trong idir: S_DATE = INIT tu file flxf dau ; E_DATE = INIT... (thuc te lay tu ten file) Neu CHUA co wdir/$S_DATE, wdir2/$S_DATE, done/$S_DATE, output/$S_DATE: ./run_wps.sh $S_DATE $E_DATE (chay WPS truc tiep) sbatch run_wrf.sh $S_DATE $E_DATE (nop job SLURM ten 'clWRF_run') - Sau cung: cd sang RegCM/CFS ; ./grb2nc_CFS.sh (khoi dong nhanh RegCM). run_wps.sh S_DATE E_DATE [GD tien xu ly clWRF]: - Tao wdir/$S_DATE/input ; link GEOGRID.TBL, METGRID.TBL. - Link toan bo cap flxf*/pgbf*.grb2 (buoc 6h tu S_DATE den E_DATE) vao input/ (thieu file -> bao 'Missing' va exit 1). - Chay 2 lan ungrib (FLXF roi PGBF) voi namelist.wps sinh tu namelist.wps_org (thay S_/E_ YYYY/MM/DD/HH, TOPO_DIR, FFFF). Vtable = Vtable.CFSR. Link geo_em.d0?.nc. Cuoi cung chay metgrid.exe -> met_em.d01.*. run_wrf.sh S_DATE E_DATE [GD1 tich phan clWRF - SLURM 2 node x 20]: - cd wdir/$S_DATE ; doc num_metgrid_levels tu met_em (ncdump). - Sinh namelist.input tu namelist.input_org (thay S_/E_, NUM_MET_LEVEL). - Link *.TBL, ozone*, RRTMG*, CAMtr_volume_mixing_ratio (RCP4.5). - mpirun real.exe (>& real.log) ; mpirun -np $SLURM_NTASKS wrf.exe (>& wrf.log). - mkdir output ; mv wrfout*/wrfxtrm* -> output/. - Goi ./run_post-process.sh (chuyen sang GD2 ngay trong job). clean [dung boi run_post-process.sh cua clWRF]: - Voi moi wdir/$d co rsl.error.0000 chua "SUCCESS COMPLETE WRF": chuyen wrfout/wrfxtrm -> wdir/$d/output, xoa tap tin tam, MV wdir/$d -> wdir2/. - (Danh dau GD1 hoan tat = ton tai wdir2/$S_DATE/output/wrfout_*). run_post-process.sh [GD2 hau xu ly clWRF]: - source configure_wrf ; export domain=.../NCCB_562/plot_domain/hgt_domain_20km.nc - ./clean (dua cac run xong ve wdir2/). - Voi moi wdir2/*/output: idir=wdir2/$d ; tao netcdf/ ; copy convert_nc/* vao. S_DATE/E_DATE lay tu ten wrfout dau/cuoi. Neu chua co .../data/current/CLWRF/$S_DATE/u.nc thi: 0_run_ARWpost.sh -> wrfout_d01_.nc (ARWpost.exe + cdo import_binary) 1_selvar_packed.sh -> tach tung bien (u,v,w,tk,rh,theta,qvapor,geopt,...), nen nc4 zip_9, dat _FillValue. 2_selvar_xtrm.sh -> gop wrfxtrm (T2MAX/MIN/MEAN,U10MAX,V10MAX,RAIN...) -> xtrm.nc Don *.sh/wrfout/namelist ; mkdir done/$d ; mv *.nc -> done/$d/ (dong 'rm -rfv wdir2/$d' DA BI COMMENT -> khong xoa). -> Ket qua clWRF cuoi cung nam o done/ (= data/current/CLWRF/). run_check.sh: liet ke cac chu ky con thieu rainc.nc (kiem tra hoan thanh GD2). Cac script phu: manual.sh / run_wrf_man.sh / run_real.sh / re-run_wrf.sh (chay tay 1 ngay, chi real, chay lai). -------------------------------------------------------------------------------- 5. CHUOI CHAY RegCM (RegCM/CFS/) - khoi dong tu CLWRF/autorun.sh -------------------------------------------------------------------------------- grb2nc_CFS.sh [GD chuyen doi GRIB2 -> NetCDF - SLURM 'do_tasks']: - source configure_regcm ; idir=$DATA_DIR/CFS_Ope/00. - Voi moi : S_DATE/E_DATE tu ten flxf. Neu chua co Dataset/CFS/$S_DATE, wdir, wdir2, done: wgrib2 flxf (TMP:surface) -> Dataset/CFS/$S_DATE/01/SST/sst..nc wgrib2 pgbf (TMP,RH,UGRD,VGRD,HGT tren cac 'mb') -> Dataset/CFS/$S_DATE/01/PLEV/{air,rhum,uwnd,vwnd,hgt}..nc (dung bang prc.table, xuat nc4). Sau do goi ./autorun.sh. autorun.sh: - source configure_regcm ; module add cdo. - Voi moi Dataset/CFS/$d: dung cdo doc ntime va S_DATE/E_DATE tu SST. Neu chua co wdir/wdir2/done cho $S_DATE VA kich thuoc Dataset du lon: ./run_icbc.sh $S_DATE $E_DATE sbatch run_regcm.sh $S_DATE $E_DATE (job SLURM ten 'RegCM_run', 1 node x 20) run_icbc.sh (can S_DATE,E_DATE tu env) [GD tien xu ly RegCM]: - Sinh namelist.in_ tu namelist.in_org (thay S_DATE,E_DATE,restart flags). - mkdir wdir/$S_DATE/{input,output} ; link d01_* tinh vao input/. - sstCLM45 ; icbcCLM45 -> IC/BC. run_regcm.sh S_DATE E_DATE [GD1 tich phan RegCM - SLURM]: - Sinh lai namelist.in_ ; mkdir wdir/$S_DATE/{input,output}. - ./run_icbc.sh $S_DATE $E_DATE. - srun -n $SLURM_NTASKS regcmMPICLM45 >& logs_run_. - ./run_post-process.sh (chuyen sang GD2 ngay trong job). clean [dung boi run_post-process.sh cua RegCM]: - Voi moi logs_run_ chua "RegCM V4 simulation successfully reached end": mv wdir/$S_DATE -> wdir2/ ; RM -rfv ** Dataset/CFS/. => Day la BUOC XOA can BO QUA khi khoi phuc (theo yeu cau). - (Danh dau GD1 hoan tat = ton tai wdir2/$S_DATE). run_post-process.sh [GD2 hau xu ly RegCM]: - source configure_regcm ; ./clean. - Voi moi wdir2/*/output: tao netcdf/ ; copy convert_nc/*.sh ; chay lan luot: 1_extract_ATM.sh (sigma2pCLM45 -> ua,va,wa,ta,hus,rh,hgt,mslp,ts; 6hourly/) 2_extract_SRF.sh (pr,prc,uas,vas,ts,huss,hurs,psl,evspsbl,zmla,hfss,hfls; 3hourly/) 3_extract_STS.sh (nhom STS) 4_extract_RAD.sh (nhom buc xa) mkdir done/$d ; mv ?hourly, daily -> done/$d/ ; 'rm -rfv wdir2/$d' => BUOC XOA can BO QUA khi khoi phuc. -> Ket qua RegCM cuoi cung nam o done/ (= data/current/RegCM/). run_check.sh: liet ke chu ky thieu daily/pr.nc. -------------------------------------------------------------------------------- 6. SO DO TONG QUAT (dong cron 1) -------------------------------------------------------------------------------- ope_download.sh (tai CFSv2 GRIB2 -> /home/nampq/data/CFS_Ope/00/) | +-> autorun.sh (clWRF) | WPS (run_wps.sh) -> sbatch run_wrf.sh | real.exe -> wrf.exe -> [clean] -> run_post-process.sh | ARWpost + cdo/nco -> done/ (data/current/CLWRF) | +-> grb2nc_CFS.sh (RegCM): wgrib2 GRIB2 -> NetCDF (SST/PLEV) autorun.sh -> run_icbc.sh (sst/icbc CLM45) -> sbatch run_regcm.sh regcmMPICLM45 -> [clean] -> run_post-process.sh sigma2p + cdo/nco (ATM/SRF/STS/RAD) -> done/ (data/current/RegCM) -------------------------------------------------------------------------------- 7. DIEU KIEN NHAN BIET TRANG THAI (dung cho logic idempotent khi khoi phuc) -------------------------------------------------------------------------------- clWRF: GD2 xong : data/current/CLWRF//rainc.nc (hoac u.nc) ton tai hoac done/ co *.nc. GD1 xong : wdir2//output/wrfout_* ton tai hoac wdir//rsl.error.0000 co "SUCCESS COMPLETE WRF". Dang chay : wdir/ ton tai (job dang trong hang doi / dang chay). RegCM: GD2 xong : data/current/RegCM//daily/pr.nc ton tai hoac done/ co *.nc. GD1 xong : wdir2/ ton tai hoac logs_run_ co "RegCM V4 simulation successfully reached end". Dang chay : wdir/ ton tai. Quy tac khoi phuc mot ngay: - Neu GD2 da co -> BO QUA. - Neu GD1 xong, GD2 chua-> chi chay hau xu ly (run_post-process, KHONG xoa ket qua). - Neu chua chay -> chay ca GD1 (tich phan) + GD2 (hau xu ly). - Neu wdir/ dang co (job song) -> KHONG nop lai. ================================================================================ HET ================================================================================