Note: I am very pleased to have received comments on this post, and present my responses to those comments at the end of this blog post.
In the lead up to the Copenhagen talks a group of climate scientists has put out a publication called The Copenhagen Diagnosis. For me it is a disappointing document because the scientists aren't doing what they do best, which is present science. Instead they take a chapter out of the bad skeptic book, and present evidence in a one sided and misleading way. My feeling is that they are learning the wrong lesson from everything that has happened.
The fundamental problem is that by doing things this way they are completely opening themselves up to criticism that will, to the average person, seem like the science is wrong, when it is their presentation of the science that is wrong. This causes the general public to question the whole foundation of the science on the climate issue. For people like me it causes me to read every publication, and statement from this group with complete skepticism when I would prefer to not have to dig into everything to see whether it makes sense.
I'm not going to go through the entire document. Instead I will focus on two sections where I have spent some time understanding the issues; Sea Level, and Sea Ice.
Sea Level
On page 39 of the report they make two summary statements about sea level. Which I will summarize. First sea level is rising faster than than the best estimate of the IPCC Third Assessment Report (TAR), and second that sea level rise is likely to be twice as large as the IPCC AR4 (AR4) based on a new ice sheet understanding.
Why did they choose to compare sea level rise with the Third Assessment Report? Why not the first or second, or better yet the one that was just completed AR4? Is it because most people who are reading quickly would assume that they were comparing the last few years to the most recent assessment report instead of an old one? After all they go on to say that the most recent report is seriously underestimating future sea level rise.
Even if we accept that somehow the TAR is the right baseline instead of AR4 there are problems with the graph they present. On the graph they show a gray range which they label as "IPCC Projections." They don't show anything that is a "best estimate." Looking at figure 24 from the Third Assessment Report it is a little
difficult to tell what they have used to create this range, but artistically that report had a gray range. If you use a magnifying glass and do some interpolation it might have the same high and low values as the range in their graph. But in the IPCC report that range is labeled as "the range of the average of AOGCMs for all thirty five SRES scenarios." On the Third Assessment graph there is a much wider range that is labeled "the range of all AOGCMs and scenarios including uncertainty in land-ice changes, permafrost changes and sediment deposition.
Neither of these are labeled as a "best estimate." And in fact even on their own graph the observations are at the top end of the range, not outside the range.
To make matters worse if you look at the graph you can see that observations are well above the center of the gray region by 1995. But the Third Assessment Report was published in 2001. Did the authors of the TAR really intend that sea level already be at the top of their range in a period years ahead of when the report was published? Or is this more likely just mixing apples and oranges with a comparison of General Circulation Models (GCMs) and observations. I don't think the GCMs were ever intended to be compared with observations in this manner. And the fact that the graph in the TAR doesn't include observations would back up this conclusion.
Looking to the future, what is the basis for their statement that sea level is likely to rise twice is fast as predicted in AR4? AR4 was published in 2007, so something dramatic must have happened for the conclusions of an entire chapter to be overturned. We would expect to find some serious and widely accepted peer reviewed studies that show that the AR4 consensus chapter was seriously flawed. Instead they base their conclusion on three things. First that the AR4 didn't include dynamic processes. Second that a single paper by Stefan Rahmstorf predicts higher values. And third the results of the "Delta Committee." These are really poor arguments.
On the issue of dynamic processes, it is true that the AR4 left dynamic processes out of their core range. But they go on to discuss what effect they would have. In figure 10.7 it includes an additional .1 to .2M for "Scaled-up ice sheet discharge." So in the judgment of the chapter authors, if in fact there is scaled up discharge from Greenland and Antarctica they would add .1 to .2M to their estimates. This is a long way from doubling their estimates. The Copenhagen authors present no evidence of any kind peer reviewed or other that this is incorrect.
Their second leg is the Rahmstorf 2007 paper that uses a simple linear model to forecast future sea level rises based on temperature. There are a lot of problems with that model as I have discussed in other blog posts, and as were discussed in published comments by two different groups following the publication of the paper. Even if you think that paper is interesting, it hardly can be used to overturn the consensus work of the AR4 authors who are actually experts in the field of sea level rise. (By the way they also list WBGU 2006 as a reference, but as this refers to published work, and previous IPCC reports it can hardly be used to claim that new research has overturned the AR4 prediction.)
Finally they reference the "Delta Committee." This was a government committee in the Netherlands which gave recommendations to the Dutch government on how much sea level rise to plan for. This link fills in some details about the committee, which was not a scientific assessment, and once again certainly isn't a reasonable case for saying that the estimates of the AR4 were wrong.
In summary their presentation of the past is misleading, and their statement about likely future sea level rise is not grounded in the peer reviewed literature. If they wanted to say it was their opinion that would be fine. But the way it is written a casual reader wouldn't get that message.
Sea Ice
In their summary on Sea Ice on page 31 they state that the melt in the Arctic has been larger than forecast by the AR4. This statement is true, but there are three problems. First there have been very few data points. One could just as easily say that global temperature has been well below the forecasts of the AR4. It has been pointed out repeatedly that this would not be proof of anything. Second they fail to mention in the summary that Antarctic sea ice has been above those same forecasts, although they cover the topic later in the chapter. Finally they fail to mention a recent publication by Shindell that explains that warming in the Arctic has been enhanced by black soot, which likely has been a factor in the amount of Arctic sea ice melt.
To complete the imbalance they show a graph of Arctic sea ice, but no graph of Antarctic sea ice. The Arctic sea ice graph is again a little strange. They label the "prediction" in the graph as "mean and range of IPCC models." We can see that the observations have been diverging downward, and have left the range in recent years. (Note that chose not to plot the one year recovery in 2009 even though they refer to it in the figure description.) But looking closely the divergence appears to begin in 1975. When were these models created? Once again do I believe that a model created circa 2006 was starting to be so wrong in 1975? In AR4 they don't include observations in figure 10.13 leading me to believe that you can't simply compare the observations with those models without great care in centering etc.
On the subject of the Antarctic they do devote a page to discussing the increase in sea ice. But pretty much the page is designed to explain it away because of circulation, ozone, and the fact that the ice is shrinking in some areas. I'm sure those are all good reasons, just like the black soot is a factor accelerating Arctic ice melt, which they don't mention. But to be balanced they should have presented a graph comparing Antarctic sea ice with the IPCC AR4 projections from figure 10.13. This would show a very large divergence, but on the high side.
Summary
The AR4 is a well written balanced document. I might have some objections here and there, but I learn a lot from reading it, and it gives me a good picture of where the consensus lies on climate change. When scientists stray from this type of generally balanced document, as they have with the Copenhagen Diagnosis, it makes me have to question everything that they are writing and saying, which defeats the purpose.
Response to Comments
Catherine,
I will start with the choice of report. Your premise is that they chose the TAR because there was a projection that covers the current date, while AR4 doesn't have such a projection. That is a good point, but I don't think it is quite as meaningful as you state.
The problem is that while the graph in the TAR is labeled "Sea Level Rise" it is actually a graph of absolute sea level. The graph in the CD is also absolute sea level not rate of rise. To determine the rate of sea level rise you would need to determine the slope of each of the lines in the graph. This is where you could possibly find the 1.9mm/year figure that they use although it is not sourced. If they had wanted to source a figure of rate of rise, which of course wouldn't start at zero, and then compare rate of rise during the period that would have been clear. My comments about being above the range long before 2001 have to do with absolute sea level as pictured in their graph, and since I don't think anyone was suspecting sea level would drop significantly during that period it seems unlikely that it was meant to be compared to measurements.
(After I wrote this I reflected that for me at least that graph would have been more interesting. But I think for most people two sets of nearly horizontal sea level figures with the TAR figure moving up towards the measured rate of rise wouldn't have made their rhetorical point. Especially since the rate has been decreasing in recent years, see below.)
Now let's take a look at the actual TAR graph. Between 1990 and 2010 the center of the upper and lower range, and the gray range for that matter, goes from zero to approximately .03m or 3cm. This produces an average slope over that period of 1.5mm/year, although it strains my eyes to do it. The period that they quote is 1993 to 2008 so since the curve slopes upward this might produce the 1.9mm/year figure.
They emphasize that the measured rate rate of 3.4 is 80% higher than the 1.9 figure. I have to admit that I got caught up with their graph and the ranges presented in it so I missed that point. However this recent rate of sea level rise was well understood by the authors of the chapter in AR4, and was completely taken into account in their revised sea level estimates. And even so the AR4 range is substantially similar to the TAR, as the helpfully point out, but for a different reason. So while the points they make about sea level are each technically correct, and I never said they weren't, they tend to mislead the reader into believing that the recent measured level of sea level rise would indicate that the consensus range in AR4 are likely to be understated, and it means no such thing.
You go on to say that they could have made it more dramatic by using the AR4 models, but this seems to contradict your statement that they don't cover the relevant period, so I'm not sure what you mean by that.
Your comments about the best estimate being the center are certainly consistent with other cases. But as I have pointed out it doesn't really matter which center you use, at least in this short time frame. Computing the slope using the top bar, which eventually leads to a .8M sea level rise, the recent rise in sea level is within the range, which because of the style of presentation is correctly pictured in their graph.
Catherine please add any other thoughts you would like, you caused me to look at the data a little differently, although as the scientists always like to say, my main conclusions are not affected. :-)
Anonymous,
Your post starts off well enough but then devolves into the typical sort of insults that seem to hurled around this issue constantly. Nevertheless I will answer your comment.
There have been several papers recently discussing mass loss from both Antarctica and Greenland. Particularly of note is that the GRACE satellite is showing that Antarctica may be contributing to sea level rise while the AR4 models centered around the idea that it would absorbing mass and therefore slowing sea level rise.
Nevertheless, as I pointed out in my original post, these type of ice sheet dynamics were discussed in AR4. They have provided the range that they would add for this type of thing as an additional .1 to .2 meters per year. I know of no published research changing the core conclusions on sea level rise as a result of these measurements, which in fact at current levels would be trivial. Trust me if those reports existed the authors of the CD would have referenced them.
In addition based on recent research (1) sea level rise is currently decelerating over the short term. So even if you believe that there is increased contribution from the ice sheets it is being offset from other factors. Remember that 3.4 mm/year would only yield .34m of sea level rise over the century, so there has to be an acceleration just to reach the midpoint of the estimates.
(1) A new assessment of the error budget of global mean sea level rate estimated by satellite altimetry over 1993–2008
M. Ablain1, A. Cazenave2, G. Valladeau1, and S. Guinehut1
Friday, November 27, 2009
Wednesday, November 25, 2009
Follow up on Rahmstorf 2007
I had thought in my last post that it was quite evident that the model in Rahmstorf 2007 was not well specified. In fact I was more interested in the issues dealing with the response to comments than I was in making the point clearly I suppose.
In that post I showed that the first half of the data did not do a good job of predicting the second half of the data. In fact the coefficient for the second period has half the value of the first period which would produce wildly different results for future predictions. But this by itself doesn't show the obvious which is that the linear model just doesn't work even without looking at out of sample prediction.
Here is a web page that presents the methodology for whether a linear regression is well specified.
http://www.duke.edu/~rnau/testing.htm
Remember that I am doing this against the final calculations with corrections from the corrigedums by Rahmstorf.
First I plotted predicted values against the actual values, and in fact they are not symmetrically distributed around either the diagonal or horizontal line. You can try it yourself from the code I already posted. But since there is no fixed rule for what symmetric means, my experience is that this will not be sufficient to make my point.
ocorrelat
So then I computed the Durbin Watson statistic for autocorrelation in the results.
http://en.wikipedia.org/wiki/Durbin%E2%80%93Watson_statistic
The result is .4 which according the Wikipedia page puts it in the range where it "might be cause for alarm."
The point is that the residuals are not well scattered, and they are highly autocorrelated. This should be enough for anyone to see that even a first year statistics student would know that the model isn't well specified.
In response to a comment here is the plot of the actual versus predicted values.
Here is a plot of the residuals. It doesn't take a DW statistic to see how highly autocorrelated they are.
In that post I showed that the first half of the data did not do a good job of predicting the second half of the data. In fact the coefficient for the second period has half the value of the first period which would produce wildly different results for future predictions. But this by itself doesn't show the obvious which is that the linear model just doesn't work even without looking at out of sample prediction.
Here is a web page that presents the methodology for whether a linear regression is well specified.
http://www.duke.edu/~rnau/testing.htm
Remember that I am doing this against the final calculations with corrections from the corrigedums by Rahmstorf.
First I plotted predicted values against the actual values, and in fact they are not symmetrically distributed around either the diagonal or horizontal line. You can try it yourself from the code I already posted. But since there is no fixed rule for what symmetric means, my experience is that this will not be sufficient to make my point.
ocorrelat
So then I computed the Durbin Watson statistic for autocorrelation in the results.
http://en.wikipedia.org/wiki/Durbin%E2%80%93Watson_statistic
The result is .4 which according the Wikipedia page puts it in the range where it "might be cause for alarm."
The point is that the residuals are not well scattered, and they are highly autocorrelated. This should be enough for anyone to see that even a first year statistics student would know that the model isn't well specified.
In response to a comment here is the plot of the actual versus predicted values.
Here is a plot of the residuals. It doesn't take a DW statistic to see how highly autocorrelated they are.
Monday, April 13, 2009
Published Comments on Rahmstorf 2007
After Rahmstorf 2007 (R07) was published in science there were two comments published which were each critical of the results. One comment was by Torben Schmith, Søren Johansen, and Peter Thejll (Schmith et al). The other was by Simon Holgate, Svetlana Jevrejeva, Philip Woodworth, and Simon Brewer. (Holgate et al). Dr. Rahmstorf wrote a reply (RR07) to both of these comments. A year later he followed up with a technical correction to the reply. I will touch briefly on Schmith et al. But focus on Holgate et al. In this post I will show that the response to Holgate et al. was not nearly as robust as presented. My conclusion is the same as Holgate et al. "...we do not agree that simplistic projections of the nature presented in [R07] substantially contribute to our understanding of the uncertainties in the nonlinear relationships of the climate system."
Schmith essentially said that due to the fact that there is a trend in both the temperature and sea level series that it violated the "basic assumptions of the statistical methods used." Remarkably they didn't comment on the smoothing. Even more importantly they ignored the fact that the residuals show a very high level of autocorrelation further showing that the model is mis-specified. Even so they concluded that the likelihood of the model was overstated. In RR07 Dr. Rahmstorf attempted to show that even with the trends removed a good fit remained, but this hardly addresses the main points which stand. The rest of the this post will show that the point is largely moot anyway.
The point that Holgate et al. made was simple. R07 didn't test to see if it could predict withheld data by modeling on the rest of the data. There are two problems with their presentation that confused the point when Dr. Rahmstorf responded. First they never duplicated his original result, instead using an SSA algorithm of their own devising. Second they used a different method to build the models. Using their own methods they were unable to predict the second half of the data set using the first half, and of course likewise in reverse. It is true that Dr. Rahmstorf didn't publish his code until he wrote RR07, but it seems like the should have sent him an email so they could start from the same place.
(I also note that at the beginning of his response to comments Dr. Rahmstorf said he was making "the computer code used in the analysis available for use by other researchers." What he neglects to mention is that the computer code he supplied was essentially useless without ssatrend.m. And nothing in his source code indicates where you could find that. The rest of his code is just simple regression and plotting, not much use to other researchers.)
Of course in RR07 Dr. Rahmstorf went back to his original method and reported that he could predict the second half using the first half of the data, and likewise the first half using the second half. In making his response he failed to report certain important points, and more importantly he made a significant error. A year later in October 2008 Science published his "technical correction" for that error, but as I will show that technical correction also fails.
The question is does the first half of the data predict the second half. Or, from my point of view, put more simply would a model built from the first half of the data be similar to a model built from the second half.
To evaluate this I regressed a model using the first twelve of the twenty-four five year bins. In this case the intercept is the same as the full model, but the coefficient is .44 versus .34 in the model built from the full data set. (RR07 reports .42, I'm not sure why the difference but it doesn't really matter. Also it incorrectly reports this as .42mm/year/degree but it is actually .42 cm/year/degree.) Using the second twelve of the five year bins the coefficient is only .24. (RR07 doesn't report this.) So simply put a model based on the first part of the period would have predicted nearly twice the sensitivity to temperature change that was experienced in the second set. RR07 shows graphs that it claims shows that it is making useful predictions, but based on this analysis it seems like a pretty poor match to me.
I've thrown some R code into the bottom of the source file so that the reader can project 2100 sea level with the different coefficients. But suffice it to say it makes a huge difference whether the coefficient is .42 or .24.
In any event I wonder whether the reader of these two posts has noticed the error in this methodology? The problem is that prior to being put into the bins the data had already been smoothed by the ssatrend algorithm with a 15 year window. This means that some of the original data from the second period is actually influencing the smoothed data in the first period. I don't know how Dr. Rahmstorf discovered this but as I said in October 2008 he published a technical correction. In the technical correction he noted the problem;
"This is correct, but it was illustrated by an incorrect figure (Fig. 1),in which the first half of the smoothed sea-level curve (1882 to 1941) was used to predict the sea level for 1942 to 2001. Because the smoothing procedure used a 15-year time window, the smoothed sea-level curve up to 1941 effectively contains sea-level information up to 1948. When this error is corrected and only annual sea-level measurements from 1882 to 1941 are used, the obtained fitgives a sea-level slope of 0.35 mm/year per °C"
(.35 mm/year/°C should, of course, be .35cm/year/°C)
This, he helpfully pointed out, was almost equal to the full trend of the original model even more strongly demonstrating how well a model from the first period predicted the rest of the data.
He failed to note three things. First in the response he said that he trained the data using the period 1880-1940. In the technical correction this was deftly changed to 1882-1941. It turns out the result is highly sensitive to this choice. Second based on my work I have determined that he changed the window size to 10 years from 15. Third using this exact technique the coefficient in the second period is only .22 this is unreported in the technical correction and is still much different than the trend from the first period.
In the data sets supplied by RR07 the sea level data begins in 1870, but the temperature data begins in 1880. As I've said before it isn't clear why he chose to bin the data at all, since it doesn't make any difference to the result, but the way that he did the binning in R07 starts the analysis in 1882. You can only find this out from the code, as R07 says it is using the period 1880-2001. I note that it doesn't change the results of the initial analysis.
In RR07 he says; "...but using only the first half of the data set (1880 to 1940) for deriving the statistical fit." RR07 supplied the code for R07 but not the code for the analysis in the response. This doesn't really matter because as I have duplicated he got the results in RR07 by using the approach of just regressing the first twelve bins. (This effectively started in 1882)
I have duplicated the .35 figure reported in the technical correction. To do this you have to use exactly 1882-1941. You also have to reduce the window to 10 years from 15, this is unreported in the technical correction. The result is non-robust to changes in either the exact year range or the window. If you move the window one year back this lowers the first period coefficient to .26. If you move it one year forward it raises the coefficient to .41. Combining the fact that the first and second periods don't match, to the fact that trivial changes in date ranges and window selections make large changes in the results shows that this model is not well specified. It is certainly not useful for updating sea level predictions beyond the results of AR4.
As a closing note on this analysis I want to say that it may not have been intentional, but R07, RR07, and the technical note were not nearly transparent enough. Instead it seems that they were written to make a point. It also points out, once again, the need for complete code disclosure.
Code for this analysis can be found here.
Schmith essentially said that due to the fact that there is a trend in both the temperature and sea level series that it violated the "basic assumptions of the statistical methods used." Remarkably they didn't comment on the smoothing. Even more importantly they ignored the fact that the residuals show a very high level of autocorrelation further showing that the model is mis-specified. Even so they concluded that the likelihood of the model was overstated. In RR07 Dr. Rahmstorf attempted to show that even with the trends removed a good fit remained, but this hardly addresses the main points which stand. The rest of the this post will show that the point is largely moot anyway.
The point that Holgate et al. made was simple. R07 didn't test to see if it could predict withheld data by modeling on the rest of the data. There are two problems with their presentation that confused the point when Dr. Rahmstorf responded. First they never duplicated his original result, instead using an SSA algorithm of their own devising. Second they used a different method to build the models. Using their own methods they were unable to predict the second half of the data set using the first half, and of course likewise in reverse. It is true that Dr. Rahmstorf didn't publish his code until he wrote RR07, but it seems like the should have sent him an email so they could start from the same place.
(I also note that at the beginning of his response to comments Dr. Rahmstorf said he was making "the computer code used in the analysis available for use by other researchers." What he neglects to mention is that the computer code he supplied was essentially useless without ssatrend.m. And nothing in his source code indicates where you could find that. The rest of his code is just simple regression and plotting, not much use to other researchers.)
Of course in RR07 Dr. Rahmstorf went back to his original method and reported that he could predict the second half using the first half of the data, and likewise the first half using the second half. In making his response he failed to report certain important points, and more importantly he made a significant error. A year later in October 2008 Science published his "technical correction" for that error, but as I will show that technical correction also fails.
The question is does the first half of the data predict the second half. Or, from my point of view, put more simply would a model built from the first half of the data be similar to a model built from the second half.
To evaluate this I regressed a model using the first twelve of the twenty-four five year bins. In this case the intercept is the same as the full model, but the coefficient is .44 versus .34 in the model built from the full data set. (RR07 reports .42, I'm not sure why the difference but it doesn't really matter. Also it incorrectly reports this as .42mm/year/degree but it is actually .42 cm/year/degree.) Using the second twelve of the five year bins the coefficient is only .24. (RR07 doesn't report this.) So simply put a model based on the first part of the period would have predicted nearly twice the sensitivity to temperature change that was experienced in the second set. RR07 shows graphs that it claims shows that it is making useful predictions, but based on this analysis it seems like a pretty poor match to me.
I've thrown some R code into the bottom of the source file so that the reader can project 2100 sea level with the different coefficients. But suffice it to say it makes a huge difference whether the coefficient is .42 or .24.
In any event I wonder whether the reader of these two posts has noticed the error in this methodology? The problem is that prior to being put into the bins the data had already been smoothed by the ssatrend algorithm with a 15 year window. This means that some of the original data from the second period is actually influencing the smoothed data in the first period. I don't know how Dr. Rahmstorf discovered this but as I said in October 2008 he published a technical correction. In the technical correction he noted the problem;
"This is correct, but it was illustrated by an incorrect figure (Fig. 1),in which the first half of the smoothed sea-level curve (1882 to 1941) was used to predict the sea level for 1942 to 2001. Because the smoothing procedure used a 15-year time window, the smoothed sea-level curve up to 1941 effectively contains sea-level information up to 1948. When this error is corrected and only annual sea-level measurements from 1882 to 1941 are used, the obtained fitgives a sea-level slope of 0.35 mm/year per °C"
(.35 mm/year/°C should, of course, be .35cm/year/°C)
This, he helpfully pointed out, was almost equal to the full trend of the original model even more strongly demonstrating how well a model from the first period predicted the rest of the data.
He failed to note three things. First in the response he said that he trained the data using the period 1880-1940. In the technical correction this was deftly changed to 1882-1941. It turns out the result is highly sensitive to this choice. Second based on my work I have determined that he changed the window size to 10 years from 15. Third using this exact technique the coefficient in the second period is only .22 this is unreported in the technical correction and is still much different than the trend from the first period.
In the data sets supplied by RR07 the sea level data begins in 1870, but the temperature data begins in 1880. As I've said before it isn't clear why he chose to bin the data at all, since it doesn't make any difference to the result, but the way that he did the binning in R07 starts the analysis in 1882. You can only find this out from the code, as R07 says it is using the period 1880-2001. I note that it doesn't change the results of the initial analysis.
In RR07 he says; "...but using only the first half of the data set (1880 to 1940) for deriving the statistical fit." RR07 supplied the code for R07 but not the code for the analysis in the response. This doesn't really matter because as I have duplicated he got the results in RR07 by using the approach of just regressing the first twelve bins. (This effectively started in 1882)
I have duplicated the .35 figure reported in the technical correction. To do this you have to use exactly 1882-1941. You also have to reduce the window to 10 years from 15, this is unreported in the technical correction. The result is non-robust to changes in either the exact year range or the window. If you move the window one year back this lowers the first period coefficient to .26. If you move it one year forward it raises the coefficient to .41. Combining the fact that the first and second periods don't match, to the fact that trivial changes in date ranges and window selections make large changes in the results shows that this model is not well specified. It is certainly not useful for updating sea level predictions beyond the results of AR4.
As a closing note on this analysis I want to say that it may not have been intentional, but R07, RR07, and the technical note were not nearly transparent enough. Instead it seems that they were written to make a point. It also points out, once again, the need for complete code disclosure.
Code for this analysis can be found here.
Saturday, April 11, 2009
Duplicating Rahmstorf 2007
I had quite an interesting time replicating the calculations from "A Semi-Empirical Approach to Projecting Future Sea-Level Rise" (Science 1/19/2007 R07) by Stefan Rahmstorf. The general concept in the paper is very simple. He estimates a linear equation for the rate of sea level rise based on global temperature from measured data. The difficulty came in because the paper relies on an algorithm that is not generally available. To demonstrate this it seems that although there were two critical comments on his paper published in Science neither replicated the original calculations of R07. It only became possible to replicate the results when Dr. Rahmstorf published his code as part of his response to those comments. Even then you had to do a little hunting. In this post I will show that I have duplicated those results, so that my subsequent comments make sense.
R07 is quite simple in concept. His theory is that the rate of sea level rise has a linear correlation with surface temperature. The idea is that you start with a situation where sea level is stable. Then you raise the temperature causing sea level to rise until is reaches a new equilibrium. I won't explain the whole paper, but this makes perfect sense. There are, of course, multiple inputs to sea level rise, but pretty much all of them should respond to a change in surface temperature, the question is how fast and how much.
To estimate this linear equation he uses sea level data from Church and White (IPCC), as well as the GISS global temperature record. But this is where things get a little tricky. He doesn't do a simple regression on the temperature and sea level because this data is "noisy." Instead runs both the temperature and the sea level data through a process to separate the trend from the noise. This isn't explained in the main text of the paper, but it is referred to in the text below figures 2 and 3. "Both temperature andsea-level curves were smoothed by computing nonlinear trend lines, with an embedding period of 15 years (14)." He uses the word "smoothed" but this is not a typical smoothing algorithm. Use of a typical polynomial fit will not get as "good" a result, as I discovered in early attempts.
I might have discovered this more quickly if I had read all the way to the bottom of Dr. Rahmstorf's reply to comments where a link to the code and data was published. The code and data weren't linked from the original paper which is where I started. Both published comments also didn't have the benefit of this code and data which is clear from reading them, and I guess after that everyone called it a day. But I will get to this stuff in a later post.
Reference 14 from the paper is "J. C. Moore, A. Grinsted, S. Jevrejeva, Eos 86, 226 (2005)." This paper is titled "New Tools for Analyzing Time Series Relationships and Trends." It is actually a short review article of several new mathematical techniques. At first glance it certainly isn't immediately apparent that this tells us how the "smoothing" was done. But there is a section titled "Nonlinear Trends in Sea Level and Temperature." This section refers to the use of Single Spectrum Analysis (SSA) to extract a nonlinear trend. It refers to Ghil et. al. 2002. So this is at best an indirect reference.
At this point I still hadn't discovered the code at the bottom of the reply, but I did have a lead. So I looked into SSA and discovered software at UCLA that was available. To make a long story short SSA is necessary but not sufficient to duplicate the R07 results. In fact further searches, and eventually the code led me to the fact that R07 relied on a matlab function called ssatrend.m. Dr. Rahmstorf on Real Climate indicated that this was available from Aslak Grinsted.
I wrote Dr. Grinsted who wrote me back very promptly, and sent me the source code to ssatrend.m. He also commented that he had no idea how Dr. Rahmstorf had gotten a copy of it, and that he had never meant for it to be distributed. I think that he was just concerned about it being unsupported. My own view is that it is pretty strange to use some random piece of code in a published paper without making the code your own. Especially where, as in this case, the strength of ther result depends on using this specific algorithm. It isn't available from any of the usual Matlab repositories which is the first place I looked.
In any event with a little effort I found an SSA algorithm written in R on source forge. I checked its results agains the UCLA program and they were identical so I knew the foundation was good. Then with some effort, I translated ssatrend.m into R so that I could run it on an open platform.
Finally I translated the code supplied in the reply written by Dr. Rahmstorf into R so that I could at see if I was starting from the same place. This was successful.
Using the supplied input files for sea level and temperature I get a correlation coefficient (R) of .88 exactly as reported in R07. In addition the following graphs are identical to figures two and three (top) of R07. The code is here.
At the end of the day the use of this unpublished algorithm made this much harder than it should have been as the underlying concept is very simple. I have no idea how a reviewer could have evaluated whether the use of this algorithm made sense. Having said that I don't really see any problem with it, and I look forward to trying ssatrend with other data.
But now that I can get the exact results from the paper, I have a couple of comments that go beyond what has already been written in Science.
R07 is quite simple in concept. His theory is that the rate of sea level rise has a linear correlation with surface temperature. The idea is that you start with a situation where sea level is stable. Then you raise the temperature causing sea level to rise until is reaches a new equilibrium. I won't explain the whole paper, but this makes perfect sense. There are, of course, multiple inputs to sea level rise, but pretty much all of them should respond to a change in surface temperature, the question is how fast and how much.
To estimate this linear equation he uses sea level data from Church and White (IPCC), as well as the GISS global temperature record. But this is where things get a little tricky. He doesn't do a simple regression on the temperature and sea level because this data is "noisy." Instead runs both the temperature and the sea level data through a process to separate the trend from the noise. This isn't explained in the main text of the paper, but it is referred to in the text below figures 2 and 3. "Both temperature andsea-level curves were smoothed by computing nonlinear trend lines, with an embedding period of 15 years (14)." He uses the word "smoothed" but this is not a typical smoothing algorithm. Use of a typical polynomial fit will not get as "good" a result, as I discovered in early attempts.
I might have discovered this more quickly if I had read all the way to the bottom of Dr. Rahmstorf's reply to comments where a link to the code and data was published. The code and data weren't linked from the original paper which is where I started. Both published comments also didn't have the benefit of this code and data which is clear from reading them, and I guess after that everyone called it a day. But I will get to this stuff in a later post.
Reference 14 from the paper is "J. C. Moore, A. Grinsted, S. Jevrejeva, Eos 86, 226 (2005)." This paper is titled "New Tools for Analyzing Time Series Relationships and Trends." It is actually a short review article of several new mathematical techniques. At first glance it certainly isn't immediately apparent that this tells us how the "smoothing" was done. But there is a section titled "Nonlinear Trends in Sea Level and Temperature." This section refers to the use of Single Spectrum Analysis (SSA) to extract a nonlinear trend. It refers to Ghil et. al. 2002. So this is at best an indirect reference.
At this point I still hadn't discovered the code at the bottom of the reply, but I did have a lead. So I looked into SSA and discovered software at UCLA that was available. To make a long story short SSA is necessary but not sufficient to duplicate the R07 results. In fact further searches, and eventually the code led me to the fact that R07 relied on a matlab function called ssatrend.m. Dr. Rahmstorf on Real Climate indicated that this was available from Aslak Grinsted.
I wrote Dr. Grinsted who wrote me back very promptly, and sent me the source code to ssatrend.m. He also commented that he had no idea how Dr. Rahmstorf had gotten a copy of it, and that he had never meant for it to be distributed. I think that he was just concerned about it being unsupported. My own view is that it is pretty strange to use some random piece of code in a published paper without making the code your own. Especially where, as in this case, the strength of ther result depends on using this specific algorithm. It isn't available from any of the usual Matlab repositories which is the first place I looked.
In any event with a little effort I found an SSA algorithm written in R on source forge. I checked its results agains the UCLA program and they were identical so I knew the foundation was good. Then with some effort, I translated ssatrend.m into R so that I could run it on an open platform.
Finally I translated the code supplied in the reply written by Dr. Rahmstorf into R so that I could at see if I was starting from the same place. This was successful.
Using the supplied input files for sea level and temperature I get a correlation coefficient (R) of .88 exactly as reported in R07. In addition the following graphs are identical to figures two and three (top) of R07. The code is here.
At the end of the day the use of this unpublished algorithm made this much harder than it should have been as the underlying concept is very simple. I have no idea how a reviewer could have evaluated whether the use of this algorithm made sense. Having said that I don't really see any problem with it, and I look forward to trying ssatrend with other data.
But now that I can get the exact results from the paper, I have a couple of comments that go beyond what has already been written in Science.
New consensus on sea level rise?
Recently because of some posts on stoat I have become interested in how the consensus is moving on projected sea level rise. There are clearly some scientists who now believe that 1M above 1990 by 2100 is likely. These include Stefan Rahmstorf, and Aslak Grinsted who have published papers based on empirical analysis to come to this conclusion. There are a number of issues with the paper by Dr. Rahmstorf some of which were published as comments in Science. I will post some additional comments which follow the publishing of the code and data. The paper by Dr. Grinsted is more interesting to me, but because of the time periods he uses to train and test his model much of the input data and assumptions are necessarily pretty fuzzy.
In AR4 the consensus estimates for sea level rise are on page 821 figure 10.33. Table 10.7 on the previous page breaks down the components. The range for the various scenarios including the error bars is roughly .2 meters to .6 meters. Many people seem to feel that they just threw up their hands at faster ice sheet discharge but table 10.33 includes figures for that at the bottom. They declined to add these into the projections because they couldn't assess the likelihood of this happening. It is important to note that these would have only added .09 to .17 meters to the high end of the range bringing the top to about .8 meters. (It would have slightly lowered the bottom of the range as well.)
So I wonder how far the consensus has actually changed in the last couple of years. Or are there simply some scientists who believe the consensus is low? Should the best estimate now be considered 1M? Or should we stay with the IPCC conclusions? Of course I don't attend the conferences with the types of experts who are called on to determine the consensus view. But I note that it doesn't seem to me that any of the lead authors of Chapter 10 are authors of these recent studies. (I could easily be wrong as cross checking that type of thing is difficult.)
Anyway I am going to write a couple of posts looking at these empirical papers. I am also looking at building an empirical model of my own to see if it will improve on the results of R07.
In AR4 the consensus estimates for sea level rise are on page 821 figure 10.33. Table 10.7 on the previous page breaks down the components. The range for the various scenarios including the error bars is roughly .2 meters to .6 meters. Many people seem to feel that they just threw up their hands at faster ice sheet discharge but table 10.33 includes figures for that at the bottom. They declined to add these into the projections because they couldn't assess the likelihood of this happening. It is important to note that these would have only added .09 to .17 meters to the high end of the range bringing the top to about .8 meters. (It would have slightly lowered the bottom of the range as well.)
So I wonder how far the consensus has actually changed in the last couple of years. Or are there simply some scientists who believe the consensus is low? Should the best estimate now be considered 1M? Or should we stay with the IPCC conclusions? Of course I don't attend the conferences with the types of experts who are called on to determine the consensus view. But I note that it doesn't seem to me that any of the lead authors of Chapter 10 are authors of these recent studies. (I could easily be wrong as cross checking that type of thing is difficult.)
Anyway I am going to write a couple of posts looking at these empirical papers. I am also looking at building an empirical model of my own to see if it will improve on the results of R07.
Saturday, March 14, 2009
Troposhpere Temps from Satellites and Surface Temps
In an earlier post I took at look at MM07 versus S09. One of the interesting results was that the rate of post secondary education (PSE) in a country was inversely proportional to the rate of temperature increase in that country. Taking a deeper look I saw that if you take the top quartile of grid points based on PSE from the analysis that the surface warmed more slowly than the troposphere, while in the rest of the grid points the surface warmed much faster than the troposphere. (This is using HadCRUT for the surface and RSS for the troposphere.) I suppose you could shrug this off to coincidence except that according to the Model E data supplied by Dr. Schmidt the troposhpere is supposed to be generally warming faster than the surface everywhere.
Over the 440 grid cells of the analysis the Model E troposphere warmed faster than the surface (.16 degrees per decade versus .14) This contrasts with the measured data from HadCRUT and RSS where the surface warmed faster than the troposphere (.27 versus .23).
Looking at the segment with high PSE we can start with the US. Now I know there has been a lot of sniping about the US temperature network, but I'm guessing that it is really pretty good. Out of the 440 grid points 52 are in the US. For the these grid points the troposhpere is warming faster than the surface (.26 versus .24). There are 85 grid points in the top quartile outside the US. For those points the troposphere is also warming faster than the surface (.21 versus .19).
Now the truth is that this data is very convenient for me to look at because it was already layed out by others, and I haven't looked at any other time periods to confirm that this isn't some kind of fluke.
But I think it is pretty interesting that in the countries that probably have the best surface temperature networks the actual measurements are in line with the theory as proposed by the results of Model E. The conclusion would be that perhaps climate scientists ought to be focused on troposhperic temperatures as measured by satellite, and reduce their dependence on ground based measurements. Switching to satellite measurement seems to be happening elsewhere with a good example being sea level rise.
I should add that they ought to be noticing this type of agreement with models and be pleased with the vindication, but I don't sense that they are. I have a theory as to why.
When the satellite temperatures were first introduced by UAH they were used by climate skeptics to show that there was no warming. In addition Drs. Christy and Spencer aligned themselves to some degree with the skeptics camp. Even though subsequent events have corrected the satellite trends and there now is an independent satelitte measurement from RSS this seems to have put satellite measurements out of favor in this area. This is particularly true for the UAH data.
In fact you can get a hint of this from S09. At one point Dr. Schmidt comments that the differing regression results he got by using RSS versus UAH might be caused by the "higher trend" in RSS. In fact this is uncited and he provides no results to back this up. I think he just assumed it was true, because a quick test of the trends show that for this set of grid cells over this period RSS and UAH have identical average trends. The point being that Dr. Schmidt believed so strongly that of course UAH would show less warming than RSS that he didn't even test the conclusion.
I think it would be quite interesting if the climate modeling community would look at their results relative to the troposhperic measurements from RSS/UAH and deemphasize the surface network. There is plenty of warming in the satellite measurements, and they may be a whole lot more accurate
Over the 440 grid cells of the analysis the Model E troposphere warmed faster than the surface (.16 degrees per decade versus .14) This contrasts with the measured data from HadCRUT and RSS where the surface warmed faster than the troposphere (.27 versus .23).
Looking at the segment with high PSE we can start with the US. Now I know there has been a lot of sniping about the US temperature network, but I'm guessing that it is really pretty good. Out of the 440 grid points 52 are in the US. For the these grid points the troposhpere is warming faster than the surface (.26 versus .24). There are 85 grid points in the top quartile outside the US. For those points the troposphere is also warming faster than the surface (.21 versus .19).
Now the truth is that this data is very convenient for me to look at because it was already layed out by others, and I haven't looked at any other time periods to confirm that this isn't some kind of fluke.
But I think it is pretty interesting that in the countries that probably have the best surface temperature networks the actual measurements are in line with the theory as proposed by the results of Model E. The conclusion would be that perhaps climate scientists ought to be focused on troposhperic temperatures as measured by satellite, and reduce their dependence on ground based measurements. Switching to satellite measurement seems to be happening elsewhere with a good example being sea level rise.
I should add that they ought to be noticing this type of agreement with models and be pleased with the vindication, but I don't sense that they are. I have a theory as to why.
When the satellite temperatures were first introduced by UAH they were used by climate skeptics to show that there was no warming. In addition Drs. Christy and Spencer aligned themselves to some degree with the skeptics camp. Even though subsequent events have corrected the satellite trends and there now is an independent satelitte measurement from RSS this seems to have put satellite measurements out of favor in this area. This is particularly true for the UAH data.
In fact you can get a hint of this from S09. At one point Dr. Schmidt comments that the differing regression results he got by using RSS versus UAH might be caused by the "higher trend" in RSS. In fact this is uncited and he provides no results to back this up. I think he just assumed it was true, because a quick test of the trends show that for this set of grid cells over this period RSS and UAH have identical average trends. The point being that Dr. Schmidt believed so strongly that of course UAH would show less warming than RSS that he didn't even test the conclusion.
I think it would be quite interesting if the climate modeling community would look at their results relative to the troposhperic measurements from RSS/UAH and deemphasize the surface network. There is plenty of warming in the satellite measurements, and they may be a whole lot more accurate
Follow Up on ERA-Interim from ECMWF
After looking at this post, Ryan Maue suggested that I do a further analysis of humidity trends using the ERA Interim data set of ECMWF. This is the most up to date information, although it covers a much shorter period than the ERA 40 data I used in the earlier analysis. The results are that over the 19 year period from 1989 to 2007 there are no significant trends in specific humidity (q) in this data set. Current theory would say that q should be increasing. Thus between the ERA 40 data, and the ERA Interim data there is no confirmation of the theory, and in fact many of the trends are negative, particularly over the NH mid latitudes, but not significant.
The data for this study is from the ERA interim data set downloaded from the ECMWF servers.
The results can be found here.
The data for this study is from the ERA interim data set downloaded from the ECMWF servers.
The results can be found here.
Subscribe to:
Posts (Atom)