The publication “Spurious correlations between recent warming and indices of
local economic activity” (S09) renewed my interest in one of the papers discussed “Quantifying the influence of anthropogenic surface processes and
inhomogeneities on gridded global climate data” (MM07). The topic of the accuracy of the surface record seems to come up quite a bit. MM07 demonstrated correlations between various measures of economic activity in a geographic area and the level of measured surface warming. S09 showed that the results of MM07 depended on the choice of satellite measurement. It also demonstrated that some correlations could be found between economic data and climate model results. This would need to be spurious since the climate models are independent of economic factors.
What is interesting about the result of my work so far is the influence on results of algorithm choices. Neither paper explained the details of a key step in their data processing, although through an email exchange Dr. McKitrick pointed me to some old software from his 2005 paper that shows the choice that he made. I think I will show that the choice in this step is not obvious, but it could possibly change at least part of the conclusions of both papers.
In MM07 they looked at the relationship between the trends in satellite measured tropospheric temperatures versus surface station measured temperatures at various locations. The idea was that the satellite record would largely be uninfluenced by economic factors and would serve as a baseline for the surface temperature trends. I note that in the paper they also discuss the fact that the satellite record appears to be somewhat influenced by local economic factors. I noticed this as well in a simplistic look, which I will describe later.
Reproducing the core MM07 data required reading through a STATA script and converting the steps to R as there are details that weren’t present in their paper. This is a great example of why having the code available is useful even if it is in a different language. I would have found it difficult to scale the economic data correctly, and to select the correct rows without this information. Maybe it would be obvious to others.
Once I got passed a little learning curve with R (this is my first time), I was able to quickly reproduce the regression results of both papers using the satellite data. I haven’t yet looked at the results reported in S09 where the output of a climate model was used.
The final data set is 440 rows of latitude and longitude locations with the decadal surface trend from HadCrut, the tropospheric trend from UAH, and a set of climate and economic factors. I will call this the “global” table.
In MM07 they only used the UAH data stating that they didn’t believe that the choice of satellite data sets would matter. Given how easy it would be to do, I think they should have also tested the RSS data. Dr. Schmidt obviously agreed that it should have been tested and created a parallel trend table using RSS. Using the trend calculated and supplied by Dr. Schmidt many of the correlations in MM07 disappear.
I note that in his blog post he says that he also tested updated UAH data as well as different surface temperature sets. The results of those tests are not reported in his paper, but I assume that they were similar to the results in MM07.
The fact that substituting RSS data for UAH data caused a change in the outcome was surprising to me. After many adjustments over the years by each it was my impression that they were very similar. I assumed that this was particularly true on decadal time scales.
In S09 Dr. Schmidt speculated that the difference in results was due to a higher trend in the RSS data. Looking at the supplied data for the 440 points in the global table I noted that the mean trend was identical between RSS and UAH at (.237K/Dec vs .232K/Dec.), so I’m not sure why he wrote that.
What was different between the RSS data and UAH data was the standard deviation. The RSS data had a standard deviation of .13 versus .19 for UAH. This is a pretty big difference relative to the observed values when both are supposed to measuring the same thing.
I had never seen anything written about this type of difference in the two data sets. As a result I wanted to look at the two data sets more closely to try to understand the difference.
As a step to doing this I decided to try to recreate the 440 rows used in both MM07 and S09 using the monthly anomaly data provided by UAH and RSS. It was then that I realized that I had a problem. Looking at the lat and long data in the rows of the global table I realized that they didn’t match the grids in either the UAH or the RSS products.
UAH and RSS produce data on a 2.5x2.5 degree grid of discrete points. For example in the UAH data set that I used latitude begins at -88.75 degrees North and in 2.5 degree increments continues to 88.75. Longitude begins at -178.75 degrees East and continues in 2.5 degree increments to 178.75.
Just as an example, in the first row of the global table the latitude is 52.5, and the longitude is 2.5. That point doesn’t appear on the satellite grid. In fact neither the latitude nor the longitude appears. So the question is which satellite grid cell do I pick in order to calculate the trend? There are in fact four grid cells which surround that data point. I can pick any of them. (Top Left , Top Right, Bottom Left, or Bottom Right)
It also occurred to me that you could average the trend across all four cells, and that might be the “correct” answer since it is my understanding that the surface data is on a 5x5 grid. I haven’t tried this yet, but I intend to do that to see if anything interesting happens. Dr. Schmidt reports that he has done this and that the results are essentially the same as in S09. I want to try it just for fun anyway. (Update 2/17 I have done this and the results are here.)
It seemed to me that the choice between the four was potentially important even given the fact that there is a lot of spatial correlation, and it fact it turned out to affect the results. There is no indication in either paper or supplied supplementary information of which choice was made. Looking at the code pointed to by Dr. McKitrick in his email it looks like he picked “Top/Right” back in 2005 when he created the data set. Dr. Schmidt didn’t respond to a question asking which choice was made in S09. (Update on 2/17 Dr. Schmidt responded and said he had picked Top/Right as well)
Looking at the UAH data (Top/Left) and (Top/Right) give results that look a lot like MM07. They are not identical but I am using updated UAH data, and it is possible that the trend calculations in R are not the same as in STATA. If I use (Bottom/Left) or (Bottom/Right) then the results aren’t as good, but there are still correlations to the economic variables.
In the case of the RSS data (Top/Left) and (Top/Right) give results that are fairly similar to MM07. (Bottom/Left ) and particularly (Bottom/Right) yield results that are more like S09.
So my conclusion so far is that the choice seems to matter to the result, and I don’t know the right way to make the choice. There are other potential choices in computing the trend data from the satellite grid, but I think this is the biggest one. I would very much have liked to see in the supplemental information how this was done in each case.
I want to note that if you simply regress the surface temperature anomaly against the economic factors you still get significant results. Other geographic factors now become important as well, which makes sense because they aren’t being canceled out by the tropospheric data. I can’t think of any reason that there should be higher surface anomalies in areas of higher economic activity. This is another indication of issues in the surface temperature record. This wasn’t discussed in either MM07 or S09. Dr. McKitrick reports that this was discussed in a 2004 paper.
Regressing either satellite trend against the other factors also results in significant correlations although to a lesser degree (if you will pardon the pun). This isn’t so surprising since the lower tropospheric measurements of the satellites might be influenced by a broad range of man made surface changes. This was discussed in MM07 but not commented on in S09.
I want to thank both Dr. McKitrick and Dr. Schmidt for being so responsive to an amateur.
R scripts to see how I arrived at these conclusions can be found here.
A follow up using 5x5 grids is here.
The following is an update after I wrote the original post.
The following is the result I got using Top/Right and the UAH data. This is similar but not identical to MM07
Mean is .2339
Residuals:
Min 1Q Median 3Q Max
-0.85006 -0.11274 -0.00614 0.11036 0.62474
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -5.3795953 3.4086411 -1.578 0.11526
myuah_trop 0.9283931 0.0664378 13.974 <>
slp 0.0055687 0.0033623 1.656 0.09841 .
dryTRUE 2.6062329 4.1729660 0.625 0.53260
dslp -0.0024560 0.0041043 -0.598 0.54990
Water -0.0242371 0.0200943 -1.206 0.22842
abslat 0.0001628 0.0009495 0.171 0.86396
g 0.0392948 0.0171258 2.294 0.02225 *
e -0.0027524 0.0004521 -6.088 2.55e-09 ***
x 0.0041382 0.0035775 1.157 0.24803
p 0.3841318 0.1165618 3.296 0.00106 **
m 0.3947718 0.1431373 2.758 0.00607 **
y -0.2987748 0.1114165 -2.682 0.00761 **
c 0.0058406 0.0025162 2.321 0.02075 *
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.1764 on 426 degrees of freedom
Multiple R-squared: 0.5441, Adjusted R-squared: 0.5302
F-statistic: 39.11 on 13 and 426 DF, p-value: <>
On RealClimate.org Dr. Schmidt has told me that he selected Top/Right when he wrote his paper. The following are the results that I get when I select Top/Right. As you can see I still show significance at the 95% level for several of the economic variables including population. The significance is less than what I saw using the UAH data as posted above.
Mean trend is .2344
Residuals:
Min 1Q Median 3Q Max
-0.992645 -0.115114 -0.008233 0.111465 0.574084
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -4.9067505 3.4937453 -1.404 0.16092
myrss_trop 0.9485121 0.0738105 12.851 <>
slp 0.0049539 0.0034460 1.438 0.15129
dryTRUE 4.0677276 4.3150450 0.943 0.34638
dslp -0.0039007 0.0042443 -0.919 0.35859
Water -0.0139312 0.0205568 -0.678 0.49834
abslat 0.0031509 0.0008996 3.503 0.00051 ***
g 0.0479709 0.0174798 2.744 0.00632 **
e -0.0023729 0.0004656 -5.097 5.21e-07 ***
x 0.0058653 0.0037066 1.582 0.11430
p 0.2687887 0.1208100 2.225 0.02661 *
m 0.3385325 0.1474768 2.295 0.02219 *
y -0.2460588 0.1148984 -2.142 0.03280 *
c 0.0066888 0.0025751 2.597 0.00972 **
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.1809 on 426 degrees of freedom
Multiple R-squared: 0.5209, Adjusted R-squared: 0.5062
F-statistic: 35.62 on 13 and 426 DF, p-value: <>
I don't know why I am getting a different result than S09. It would be nice to see the code from that paper that processes the RSS data. Maybe I'm making a mistake of some kind. The trend mean looks pretty similar however.
No comments:
Post a Comment