TAF vs reality

A TAF is the official forecast for an airport, written by forecasters (or produced automatically) and used by pilots and dispatchers to plan flights. Here every hour of every TAF is checked against what the airport's own METAR reports showed when that hour came. Few places publish this.

Forecast vs observed flight category

IFR or worse, by lead time

POD, FAR and CSI for IFR-or-worse hours

Higher is better for POD and CSI, lower for FAR. Solid: prevailing forecast only; dashed: TEMPO/PROB groups count too.

Category hit rate and skill

Hit rate: share of hours in the right category. Heidke skill: 1 perfect, 0 no better than chance.

Airports

Best 15 by CSI

Worst 15 by CSI

What do these numbers mean?

Flight categories sum up how easy it is to land: VFR ceiling above 3,000 ft and visibility over 5 miles, MVFR ceiling 1,000–3,000 ft or 3–5 miles, IFR ceiling 500–999 ft or 1–3 miles, LIFR below 500 ft or under 1 mile. “IFR or worse” is when instrument procedures (and alternates, fuel, delays) come into play — the forecasts that matter most.

Hit rate

The share of hours where the TAF's main forecast had the right category. Easy to read but flattered by long fair-weather spells: forecasting VFR everywhere would already score high.

Heidke skill score

The hit rate after removing the hits you would get by chance with the same mix of forecasts and observations. 1 is perfect, 0 is no skill, below 0 worse than chance.

POD — probability of detection

Of the hours that really were IFR or worse, how many the TAF had forecast. 0.7 = 7 in 10 caught. hits / (hits + misses)

FAR — false alarm ratio

Of the hours the TAF forecast IFR or worse, how many turned out better. Lower is better. false alarms / (hits + false alarms)

CSI — critical success index

One number that punishes both misses and false alarms, ignoring the many easy “nothing happened” hours. hits / (hits + misses + false alarms)

Frequency bias

How often IFR was forecast compared with how often it happened. 1 = as often as observed; above 1 over-forecast (cautious), below 1 under-forecast. (hits + false alarms) / (hits + misses)

TEMPO and PROB groups

TAFs add temporary conditions on top of the main forecast: TEMPO (brief spells), PROB30/PROB40 (30–40 % chance). With “Count as forecast” an hour counts as forecast IFR if any of these groups active that hour says so. That catches more events (POD up) but cries wolf more often (FAR up).

How hours are matched

Each TAF is split into clock hours. FM changes apply from their time; BECMG changes count from the end of their period. Each hour is compared with the worst category in any METAR or SPECI during that hour, or with the routine METAR nearest the top of the hour. Every TAF covering an hour counts, at its own lead time. Hours before a TAF was issued are never scored.