In Part One, we introduced some climate model basics, including uses of climate models (not all of which are about “projecting” the future).
And we took at a look at them in their best light – on the catwalk, as it were.
Well, really, we took a look at the ensemble of climate models. We didn’t actually see a climate model at all..
The overall evaluation in Part One was the presentation of a “multi-model mean” or an ensemble. An ensemble can be the average of many models, or the average of one model run many times, or both combined.
We will return to more discussion about the curious nature of ensembles in a later post. Just as a starter, two observations from the IPCC.
IPCC AR4 in Chapter 8, Climate Models and their Evaluation, comments:
There is some evidence that the multi-model mean field is often in better agreement with observations than any of the fields simulated by the individual models (see Section 18.104.22.168.2), which supports continued reliance on a diversity of modelling approaches in projecting future climate change and provides some further interest in evaluating the multi-model mean results.
and a little later:
Why the multi-model mean field turns out to be closer to the observed than the fields in any of the individual models is the subject of ongoing research; a superficial explanation is that at each location and for each month, the model estimates tend to scatter around the correct value (more or less symmetrically), with no single model consistently closest to the observations. This, however, does not explain why the results should scatter in this way.
One interpretation of this would be:
We like ensembles because they give more accurate results, but we don’t really understand why..
A subject to come back to, now it’s time for a real model..
Step Forward Climate Model “Cici” – CCSM3
CCSM3, “Cici”, is the model from NCAR (National Center for Atmospheric Research) in the USA. Out of all the GCMs discussed in the IPCC AR4, Cici has the “best curves” – the highest resolution grid. Well, she comes from the prestigious NCAR..
The model’s vital statistics – first the atmosphere:
- top of atmosphere = 2.2 hPa (=2.2mbar), this is pretty much the top of the stratosphere, around 50km
- grid size = 1.4° x 1.4° (T85)
- number of layers vertically = 26 (L26)
second, the oceans:
- grid size = 0.3°–1° x 1°
- number of vertical layers = 40 (L40)
The vital statistics give a quick indication of the level of resolution in the model. And there are also model components for sea ice and land. The model doesn’t need the infamous “flux adjustment” which is the balancing term for energy, momentum and water between the atmosphere and oceans required in most models to keep the two parts of the model working correctly.
The CCSM3 model is described in the paper: The Community Climate System Model Version 3 (CCSM3) by W.D. Collins et al, Journal of Climate (2006). The source code and information about the model is accessible at http://www.ccsm.ucar.edu/models/.
And for those who love equations, especially lots of vector calculus, take a look at the 220 page technical document on CAM3, the atmospheric component.
It will be surprising for many to learn that just about everything on this model is out in the open.
CCSM3 Off the Catwalk – Hindcast Results
As with the multi-model means results in Part One we will take a look through a similar set of results for CCSM3.
Cici looks pretty good.
Details – The HadISST (Rayner et al., 2003) climatology of SST for 1980-1999 and the CRU (Jones et al., 1999) climatology of surface air tempeature over land for 1961–1990 are shown here. The model results are for the same period of the CMIP3 20th Century simulations. In the presence of sea ice, the SST is assumed to be at the approximate freezing point of sea water (–1.8 °C).
However, it’s hard to tell looking at two sets of absolute values, so of course we turn to the difference between model and reality.
Annual Temperature – Model Error
Model simulations of annual average temperature less observed values for Cici and for the “ensemble” or multi-model mean:
In terms of absolute error around the globe, Cici and the ensemble are very close (using the Anglotzen statistical method).
We could note that even though the values are “close”, there are areas where Cici – and the ensemble – don’t do so well. In Cici’s case southern Greenland and the Labrador Sea, which might be very important for predicting the future of the thermohaline circulation. And both are particularly bad for Antarctica, a general problem for models.
To give an idea of the variation of models, here are all of the models reviewed by the IPCC in AR4 (2007):
The top right is Cici (red circle). It’s clear that Cici is a supermodel..
Standard Deviation of Temperature
The standard deviation of temperature – “over the climatological monthly mean annual cycle ” – simulated less observed for Cici and the ensemble. We could describe it as how good is the model at working out how much temperature actually varies over the year in each location?
First, however, to make sense of the “error” of model less actual, we need to know what actual values look like:
As we would expect, the oceans show a lot less temperature variation than the land and around the tropics and sub-tropics the variation is close to zero.
Now let’s take a look at the model less actual, or “model error”:
We can see that Cici has some problems in modeling temperature variation especially under-estimating the actual variation around northern Russia and Canada and over-estimating the variation in the Middle East and Brazil. The ensemble appears to be in slightly better shape here.
Of course, these areas are where the largest temperature variation takes place.
Diurnal Range of Land Temperature
As before, first the actual values:
And now the model less actual, or “model error”:
We can see a lot of areas where the model error is quite large, usually corresponding to larger measured values. In the case of Greenland, for example, the annual average diurnal temperature range is over 20°C, while the model under-estimates this by more than 10°C. Given the legend the error might be as big as the actual value..
We can also see that on average Cici under-estimates the diurnal temperature range, and the ensemble is closer to neutral but still appears to under-estimate.
Here’s another comparison which demonstrates the problem of all the models vs observation:
The black line is the observed value. We can see that all of the models except for one are definitely under-estimating, and none of the models are particularly close to the observed values.
Now we can get to see more fundamental values.
Reflected Solar Radiation
This value is essential for calculating the basic radiation budget for the earth.
First the actual values as measured by ERBE (1985-1989):
And now the model error – model less actual:
The ensemble is definitely better than Cici. Cici has some large errors, for example, North Africa, Pacific Ocean and the Western Indian Ocean where the model error seems to be up to half of the actual value.
If we look at the values averaged by latitude the results appear a little better:
But the deviations give us a better view:
Note Cici in the solid blue line. The ensemble is proving to be the pick of the bunch..
So the model’s ability to simulate reflected solar radiation is much better by latitude than by location. But most or all of the models have significant discrepancies even when averaged over each latitude.
Outgoing Longwave Radiation
The other side of the radiation budget, first the actual ERBE measurement (1985-1989):
And now the model error – model less actual:
As with reflected SW radiation, the ensemble performs better than Cici. So while measured values are in the range of 200-300 W/m2, Cici has some areas where the (absolute) error is in excess of 30W/m2.
Looking at the OLR values averaged by latitude, the results appear a little better:
And the deviations, or model error:
Measured from CMAP, 1980-1999:
Units are in cm of rainfall per year. And now the model error – model less actual:
Once again the ensemble outshines Cici. There are some substantial errors in the areas where rainfall is high.
As with some of the previous model results, if we look at the model vs observed by latitude the picture is somewhat better:
Lastly, we will take a look at specific humidity. First the “measured”, as recalculated by ERA-40:
Observed annual mean specific humidity in g/kg, averaged zonally, 1980-1999. Note that the vertical axis is pressure on the left in mbar and km in height on the right.
And now the model error – but this time in % = (model – actual)/actual x 100:
Once again the ensemble appears to outperform Cici. And both, but especially Cici, have problems in the top half of the troposphere (around 500-200mbar) with 20-50% error in some regions in Cici’s case.
This has been a quick survey of model results for different parameters across the globe, but averaged annually, compared with observations.
In Part One, we saw the ensemble in its best light. But when we take a look at a real model, the supermodel Cici, we can see that she has a lot of areas for improvement.
There’s lots more to investigate about models, all to come in future parts of this series.
As always, comments and questions are welcome, but remember the etiquette.