Reimplementing the CREST hydrologic model (after EF5) in an autodiff framework, then learning parameters by backpropagation against Earth-observation truth. Target: a global 0.1° parameter field — built CONUS-first, in three stages: snow → water balance → routing.
CREST is a distributed, grid-based water-balance model used for large-scale flash-flood forecasting. EF5 calibrates it with basin-uniform parameter multipliers via MCMC/random search. We want something different: spatially-distributed parameters that generalize globally, learned efficiently with gradients.
EF5 uses DREAM (MCMC) and ARS. We reimplement the per-grid physics so it is fully differentiable and optimize parameters with Adam — orders of magnitude more sample-efficient in high dimensions.
Snow and water balance are column processes with no lateral coupling. Each 0.1° cell is solved independently — vmap over the grid, embarrassingly parallel. Only routing couples cells.
Force with reanalysis (ERA5-Land / MSWEP), but calibrate against independent observations (SNODAS, SNOTEL, fSCA, streamflow, ET, GRACE) — never model-to-model, to avoid circularity.
Build the differentiable model bottom-up, validating physics before scale. Calibrate stage-by-stage so each component is well-posed, freezing upstream outputs as inputs to the next stage — then optionally fine-tune everything end-to-end.
For snow, SWE truth is near-global, so we calibrate parameters θ directly as trainable tensors (no neural net yet), with light spatial regularization for smooth maps.
The calibrated θ field is kept structured to warm-start differentiable parameter learning (dPL) later — a network θ = gφ(attributes) (incl. AlphaEarth embeddings) that extrapolates to ungauged cells for the runoff stage and the global rollout.
Each stage lists its forcing inputs, the observational truth/constraints, the learnable output parameters, the modeling route, and the calibration scale. Parameter names follow the EF5 source.
A column snow-accumulation/ablation model. Parameters θ are trainable leaf tensors optimized by autodiff + Adam — replacing SCE-UA/DDS with backprop. No neural net at this stage.
* added as learnable beyond EF5, which
hardcodes PXTEMP=1°C and implements no ADC / ground-melt.
L = Σcell wcell · [ KGESWE + λ₁·meltout + λ₂·fSCACE ] + λsp·Rspatial
Debugging rig (e.g. East River nr Gunnison, CO). All gates must pass before scaling up.
θ as [N_cells, N_params], vmap forward. Acceptance: SWE KGE median > 0.7 (temporal holdout) · melt-out bias within ±7 days median · SNOTEL leave-out NSE/KGE reported · spatially-coherent θ maps · gradient health logged.
CREST's variable-infiltration curve turns rain + snowmelt into fast (surface) and slow (interflow) runoff. Grid-independent → vmap over cells. This is where dPL becomes primary for global generalization.
The only network-coupled stage: cells drain downstream along the river network. Routing approach is still being decided — see tech routes.
Parameters follow EF5's actual routing schemes (the table's coeM/expM/coeR/coeS map onto these):
Autodiff core, parallel over the grid; reproducible, config-driven runs on HPC.
JAX (lax.scan for time, vmap over grid) preferred; PyTorch acceptable (loop + gradient checkpointing). Optimizer: Adam.
OSCER SLURM cluster · data stored as zarr · access via Google Earth Engine where possible · YAML config, seeded & reproducible.
pyCREST: modular differentiable Snow17 / CREST / routing + dPL, data loaders, training, objectives. Built in a later round.
| Role | Datasets | Used in |
|---|---|---|
| Forcing (P, T) | ERA5-Land · MSWEP | All stages |
| Snow truth | SNODAS · SNOTEL · MODIS/VIIRS fSCA · UA-SWE · SWEML | Stage 1 |
| Runoff / flux truth | Caravan / GRDC-Caravan (Q) · MOD16 / PML / FLUXCOM (ET) · GRACE (TWS) | Stage 2 |
| Hydrograph truth | Gauge daily discharge | Stage 3 / Joint |
| River network | MERIT-Hydro | Stage 3 / Joint |
| Attributes (for dPL) | Soil · climate · topography · AlphaEarth embeddings | Stage 2+ · global rollout |
Open design decisions, with the leading candidate marked. These will be resolved as the work proceeds — the routing approach in particular is still undecided.
The exact experimental setup for the full-CONUS snow calibration (WY2010–2024).
| Role | Data |
|---|---|
| Temperature | ERA5-Land daily T2m (GEE) + per-cell warming (West from SNOTEL bias; East uncorrected) |
| Precipitation | ERA5-Land daily total precip (GEE) — swappable: IMERG/MSWEP |
| Elevation | SRTM (for atmospheric pressure) |
| Calibration target | SNODAS gridded SWE (1 km→0.1°), 8-daily |
| Independent val. | SNOTEL 830 stations (West), point SWE |
| Snowpack depth | n | median KGE |
|---|---|---|
| peak ≥ 100 mm | 26,282 | 0.63 (64%>0.5) |
| peak ≥ 200 mm | 10,469 | 0.66 |
| peak ≥ 400 mm | 4,410 | 0.66 |
| SNOTEL independent (West) | 804 | 0.36 |
Product: snow17_conus_era5.nc — 9 param maps + test-KGE + warming + snow/snowpack masks + peak-SWE.
Live milestone board — flip the badges (todo → in prog → done) as the work advances. Last updated: 2026-07-01 — Stage-2 water balance done (424 CONUS basins, KGE 0.67 / holdout 0.56). See Stage-1 Snow → and Stage-2 Water →
Anderson, E. A. (1973). National Weather Service River Forecast System — Snow Accumulation and Ablation Model. NOAA Tech. Memo. NWS HYDRO-17.
Anderson, E. A. (2006). Snow Accumulation and Ablation Model — SNOW-17. NWS documentation (parameter ranges).
Wang, J., Hong, Y., et al. (2011). The Coupled Routing and Excess Storage (CREST) distributed hydrological model. Hydrol. Sci. J., 56(1).
Flamig, Z. L., Vergara, H., Gourley, J. J. (2020). The Ensemble Framework For Flash Flood Forecasting (EF5). Geosci. Model Dev.
Feng, D., Liu, J., Lawson, K., Shen, C. (2022). Differentiable, learnable, regionalized process-based models with multiphysical outputs. Water Resour. Res.
Kratzert, F., et al. (2023). Caravan — A global community dataset for large-sample hydrology. Scientific Data.
Yamazaki, D., et al. (2019). MERIT-Hydro: high-resolution global hydrography. Water Resour. Res.
Zeng, Z., et al. (UA-SWE) & NOHRSC SNODAS — gridded snow water equivalent products.