SOD: Nothing you said about Ithaca mislead me about the generality of the observations you made about auto-correlation of temperature there. I associated auto-correlation most with the monthly tropical data in the Douglass-Santer controversy. Your first example turns out to be autocorrelation in daily data for one city. Comments in the same post discuss the possibility of long-term persistence (how is this technically different from autocorrelation?) in annual global temperature.

Autocorrelation in temperature series isn’t merely a statistical phenomena; it should reflect of the time scale over which temperature returns towards normal (an attractor?) during its chaotic/forced behavior. For that reason, it’s hard to justify the IPCC’s reporting 20th-century temperature rise as a linear trend without adequate discussion. They could have simply reported the MAGNITUDE of the temperature rise for the last century (1895-1905 mean vs 1995-2005 mean) and half-century. This would probably produce larger uncertainties (the difference in the means of 11 years) than the linear trend extracted from 100 years of (non-linear) data. The only thing linear about temperature rise is associated with the logarithmic increase in forcing with CO2 and exponentially rising CO2. Even the IPCC recognizes that CO2 is only a fraction of the anthropogenic forcing. So why abstract a linear trend?

]]>This type of response is typical of systems that have “blue noise” in them (increasing noise with frequency, nu < 0 in my notation).

Newer instrumentation amplifier topologies (like the AD8230) that reduce 1/f noise & drift have promise for significantly lowering your attainable noise floor for measurements of this sort.

]]>Frank,

I don’t know the answer. Although I have lots of preconceived ideas about how the results will pan out.

I hope to get into that by downloading some data from reanalysis projects and having a play. But need to get the theory sorted out clearly in my head first. I have a few books including Box & Jenkins, *Time-series analysis* and a number of papers. Lots to study.

I certainly didn’t mean to present the Ithaca temperature results as the gold standard for climate autocorrelation – more as one example that demonstrates there is such a thing as serial correlation of time-series climate data.

]]>A second factor may be the area involved. Warm dry air in Ithaca may blow into Vermont after a few days, but it might not escape from the Northeast US Mean. I think Santer/Douglass covered the whole tropics.

]]>Steve McIntyre has investigated the statistical correctness of the above Table 3.2 of AR4. He is skeptical that the IPCC has properly dealt with long term persistence.

http://climateaudit.org/2007/07/06/review-comments-on-the-ipcc-test/

]]>Several statisticians have criticized the climate science community (particularly paleoclimatologists) for statistically dubious practices. You could read:

McShane & Wyner (2010) http://www.e-publications.org/ims/submission/index.php/AOAS/user/submissionFile/6695?confirm=63ebfddf

The Wegman Report: http://republicans.energycommerce.house.gov/108/home/07142006_Wegman_Report.pdf

The NRC report on the Hockey Stick: http://books.nap.edu/openbook.php?isbn=0309102251

and lots of one-sided stuff at climateaudit.org

The big problem with climate science often is that the data is very noisy, the time span is limited and experiments can’t be repeated under carefully controlled conditions. Often, the more rigorous the statistical analysis, the greater the uncertainty and the less likely you will produce a high-impact publication. From my cynical perspective, who needs the advice of a professional statistician when Nature, Science and the IPCC are willing publish alarming results without a thorough statistical review and when your colleagues continue to cite your work? However, it is difficult to know whether such cynicism should be applied to a trivial or significant fraction of important climate science publications.

]]>http://en.wikipedia.org/wiki/Breusch%E2%80%93Godfrey_test

Note also that these days, after much experience of things going wrong out of sample, the first reaction of most econometricians to finding serial correlation in the residuals of their equations is to wonder whether the model is correctly specified. Serial correlation in the residuals implies that there is systematic variation in the dependent variable which is not being accounted for by the explanatory variables.

]]>Sorry to be absent for a bit. I assume when the IPCC corrects for first order autocorrelation in the residuals, it follows the standard procedure. This involves assuming that the measured residuals from the original specification follow an AR1 process, i.e.

e(t)=be(t-1)+u(t)

where b is the autoregressive parameter which is between 0 and 1. If you then want to test whether you have got rid of all the autoregression you can have a look at the properties of u(t). But you can see at once that if you use the equation above it includes a lagged dependent variable. And if you go back to the original equation and use the equation above to substitute out the e(t), your equation then contains a term in e(t-1) which can be replaced using the lagged version of the original equation so that temperature appears in lagged form in the RHS of the adjusted equation with residual term u(t). So either way, the process of adjusting for an AR1 residual term produces an equation with a lagged dependent variable on the RHS, invalidating the use of the DW statistic.

]]>I’m more used to instruments where the noise floor is shot noise limited (photon counting in emission spectrometry, e.g.). In that case the standard deviation decreases with increasing sampling time (Poisson statistics) until flicker noise and instrumental drift (~1/f) take over. Sample throughput usually dictates the counting time rather than minimum s.d. Well, sample throughput and other sources of uncertainty like sampling being larger than the measurement uncertainty.

]]>