Reservoir Forecast Verification: The Benchmark Gap in 2026

"This forecast is 85 percent accurate" means nothing on its own. What will the reservoir storage be in the next few weeks? What should we be prepared for? Whether a forecast is useful depends on how much the truth deviates from what would have happened anyway.

The cheapest forecast available for any reservoir is to look up what the lake usually did on this date, and over the weeks that followed, in other years. Call it the seasonal average. It needs no model, and it is harder to beat than most people expect.

Any real forecast has to be tested against that baseline. Compare a forecast to an easy guess, and it looks impressive. Compare it to historical averages, and the improvement looks much smaller. But that smaller number is the one worth having, because it tells you whether the forecast actually added any real value.

Water agencies have started asking for that number. They have not agreed on what to measure against, so the numbers coming back cannot be compared with each other.

LYNXCE tested the question on Detroit Lake in Oregon, a federal dam on the North Santiam River where every measurement is public and anyone can check the work.

It was useful for about a month. Out to roughly 30 days it clearly beat the seasonal average. Past about 36 days it stopped beating it at all. A month of warning is useful for planning a drawdown or a withdrawal, and a good deal less than most people assume a reservoir forecast gives them.

It only really worked in one season. A forecast made in June still beat the seasonal average two months out. One made in December was behind after ten days. Summer at Detroit is a scheduled drawdown from a full lake, so history is a decent guide. Winter depends on which Pacific storms arrive, and history has little to say about that. A single year-round accuracy figure averages the two and describes neither.

Narrowing the forecast to similar years made it worse. Looking only at past years with a similar drought status sounds careful. It also throws away most of the history the forecast is built from, and at Detroit it cost more than it bought at every timescale tested.

A longer record produced a wider range, not a narrower one. Give the method sixteen years of history and it produces a tight, confident-looking range. Give it fifty-two and the range widens, because the longer record holds extremes the short one never saw. Nothing about the lake changed. A narrow range is usually a short record, so the useful question is how many past years went into it. Two or three is a story, not a statistic.

The uncertainty range was labelled as a percentile interval. A range described as covering the middle half of likely outcomes might lead a reader to expect reality outside it about half the time. At Detroit it landed outside about one time in six. The forecast was doing its job; the label was just implying the magnitude of historical outcomes.

1. What was it beaten against? If nobody says, treat the accuracy figure as missing. It cannot be checked or compared with anyone else's.
2. How was it tested? A forecast tested on the same years it learned from will always look good. The test has to hold back the year being predicted.
3. Does it work in this season? Ask for the seasonal breakdown. It and the annual average can point opposite ways.
4. Was each assumption checked, or just assumed? Sensible-sounding refinements sometimes make a forecast worse.
5. How many past years is each number built on? A range computed from two years is an interpolation wearing the costume of a statistic.

The World Meteorological Organization issued dedicated guidance on the subject in 2025, Guidelines on the Verification of Hydrological Forecasts (WMO-No. 1364), naming accuracy, bias, reliability, resolution and sharpness as the properties a forecast system should report. The Bureau of Reclamation's Snow Water Supply Forecasting Program asks applicants to show what their work buys against status quo snow monitoring. Forecast-Informed Reservoir Operations (FIRO) has grown from one pilot at Lake Mendocino into the screening framework Forbis and Ly (2025) describe the Corps developing from its California dams, and every viability assessment has had to show the forecasts are good enough to run a dam on.

So the demand for evidence is real. Against what?

Practical answers are already in use. Prado Dam's viability assessment used Critical Success Index for precipitation and Brier skill scores against climatology for inflow. Modi et al. (2025) used normalized mean quantile loss to connect skill to economic value across the western United States. Li et al. (2026) used ROC and correlation for terrestrial water storage. Each choice is defensible on its own. None of them can be set beside another.

Pappenberger et al. (2015) put the problem plainly: skill has no meaning in the absolute, and the reference changes the answer. The scoring rule is not at fault; a continuous ranked probability score decomposes the way Hersbach (2000) set out, but a skill score built on one is only ever a ratio against whatever reference gets chosen. Harrigan et al. (2018) showed what a study built that way looks like, across 314 UK catchments and a 50-year hindcast, and it is also where the honest magnitudes are: CRPSS of 0.75 at one day, 0.20 at one month, 0.11 at three. Arnal et al. (2024) and Baker et al. (2022) followed the design into FROSTBYTE and the Colorado, and the UK Centre for Ecology and Hydrology's Historic Weather Analogues method, in Chan et al. (2026), beats climatology by 0.13 in winter and 0.03 in summer over three-month outlooks. Modest, and strongly seasonal.

LYNXCE's position: reservoir forecasts do get verified in 2026, just against nothing in common. Two skill numbers from two studies cannot be compared, and a number quoted without its reference cannot be checked at all. Treat it as missing.

USGS, USACE and the US Drought Monitor publish everything the test needed. The method is an analogue technique in the Ensemble Streamflow Prediction line Day (1985) introduced: take the historical record of net inflow, line up the windows that start on the same day of the year, and read the spread. Keeping each window contiguous makes it a moving-block bootstrap, which is Künsch (1989), and Vogel and Shallcross (1996) tested that bootstrap against parametric alternatives for estimating reservoir storage. The bootstrap won.

The benchmark was settled before the run: day-of-year climatology of observed pool storage, leave-one-year-out, blind to the initial state. Detroit runs to a rule curve, so that benchmark is sharper here than at most sites. The expected answer went into the code before any number came out of it, at roughly 0.20 at 30 days, following Harrigan.

That is 18,801 forecasts at the 10-day horizon, scored on storage in acre-feet. Skill crosses zero at about 36 days. Persistence, scored the same way, ran from −0.07 to −3.61, so this is a hard reference.

The single row hides two reservoirs.

A forecast issued in June still beats climatology at 60 days. One issued in December has already lost at 10 days. Read operationally, this is a summer tool with some useful short-lead skill in autumn and nothing through the wet season.

One reservoir is one reservoir, though. Harrigan used 314, and nothing in these tables licenses a claim about how the method behaves anywhere else.

The method filters its analogue sample by US Drought Monitor category. That filter is the only piece of explicit hydrological judgement in the whole thing.

It costs skill at every horizon, taking roughly a third of what is left at 30 days.

Conditioning splits the pool seven ways, and the median cell falls from 52 historical windows to 6. Drought category does not say enough about the next 10 to 60 days of storage to be worth 46 windows.

Part of this travels to every US site. The Drought Monitor's first map is January 2000, so a drought-conditioned sample tops out at 26 years of anchors however long the pool record runs. Going from 30 years of Detroit data to 52 moved a conditioned band by nothing at all. The covariate is the binding constraint, and the reservoir never was.

Keep that narrow. It says this covariate fails to earn its keep at Detroit, for storage, at these horizons. Detroit is a snow-and-rain Cascade reservoir with almost nothing at the severe end of the drought ladder: D3 on 14 days in 52 years, D4 never. A water-supply reservoir in a dry basin could come out the other way.

Record length works the same way, in the direction that surprises people. Extending Detroit's analogue record from 16 years to the full 52, nothing else changed, widened the median drought-conditioned band by 17 percent, from about 35,900 to about 41,900 acre-feet, and dropped the share of cells resting on two historical windows or fewer from 28.0 percent to 18.5 percent. Truncating back to 16 years narrows that same band by about 14 percent. The short record had simply never seen the extremes the long one contained.

Padding the bands is not the answer. The failure mode that matters is an empty sample quietly returning zero, which travels on through the arithmetic as a perfectly plausible drawdown with nothing downstream able to tell the difference.

Scoring the delivered product, not the underlying idea, turned up a fourth result. This one is about labels.

A band presented as a 25th-to-75th-percentile range held the observed value 79 to 87 percent of the time. At the three horizons where the statistic is really the sample minimum and maximum, it held 95 to 97 percent. Two causes: composed bands pair high inflow against low withdrawal, a deliberately conservative design choice, and a MIN–MAX statistic is not a percentile at all.

The band is unlikely to be exceeded, so the label can be misleading. Any deliverable that plots a band should name the statistic behind it on the same axis.

The five questions in part one become five requirements in a scope of work.

1. Name the benchmark in the same sentence as the skill score. Day-of-year climatology is the sensible default for reservoir storage; persistence flatters any method.
2. Fix the hindcast design before the results are seen, and write down the expected answer. Leave-one-year-out has to drop the forecast's own year and the year it lands in, because a December forecast at long lead is scored against a target in the next calendar year.
3. Report skill by season, not only in aggregate.
4. Measure every conditioning choice instead of assuming it. Only a measurement will say whether a covariate earned what it cost.
5. Carry the sample count with every band, and refuse before substituting. Where no historical window matches, return a state instead of a value.

One transition is worth planning around. NOAA's National Water Model has run operationally at v3.1 since August 2026, and the next major version, v4.0, will be the first on the NextGen framework. Reservoirs enter it as boundary conditions rather than coming out as a forecast product. Reservoir storage forecasting stays local work on local records for the planning horizon most firms care about, a point covered further in NextGen and the End of the Monolithic National Water Model.

LYNXCE builds reservoir and water-supply forecasting tools on that basis. Every band carries the number of historical windows behind it and the statistic that produced it, and where no matching window exists the tool says so rather than returning a number that looks like an answer. Where a public reservoir can stand in for the one being modelled, the benchmarked hindcast runs first, on a design fixed in advance. Detroit Lake is that work: public data throughout, scripted end to end, every number pinned by a test so the pipeline cannot change the story quietly.

Teams evaluating a reservoir forecasting tool, or scoping a verification study, can reach us at [email protected].