Inconsistencies in NCDC Historical Temperature Analysis?

Inconsistencies in NCDC Historical Temperature Analysis?

Jeff Masters and I recently received an interesting email from Ken Towe who has been researching the NCDC historical temperature database and came across what appeared to be some startling inconsistencies. Namely that the average state temperature records used in the current trends analysis by the NCDC (National Climate Data Center) do not reflect the actual published records of such as they appeared in the Monthly Weather Reviews and Climatological Data Summaries of years past. Here is why.

An Example of the Inconsistency

Here is a typical example of what Ken uncovered. Below is a copy of the national weather data summary for February 1934. If we look at, say Arizona, for the month we see that the state average temperature for that month was 52.0°F.



The state-by-state climate summary for the U.S. in February 1934. It may be hard to read, but the average temperature for the state of Arizona is listed as 52.0°F From Monthly Weather Review.

However, if we look at the current NCDC temperature analysis (which runs from 1895-present) we see that for Arizona in February 1934 they have a state average of 48.9°F, not the 52.0°F that was originally published:



Here we see a screen capture of the current NCDC long-term temperature analysis for Arizona during Februaries. Note in the bar at the bottom that for 1934 they use a figure of 48.9°.

Ken looked at entire years of data from the 1920s and 1930s for numerous different states and found that this ‘cooling’ of the old data was fairly consistent across the board. In fact he produced some charts showing such. Here is an example for the entire year of 1934 for Arizona:



The chart above shows how many degrees cooler each monthly average temperature for the entire state of Arizona for each month in 1934 was compared to the current NCDC database (i.e. versus what the actual monthly temperatures were in the original Climatological Data Summaries published in 1934 by the USWB (U.S. Weather Bureau). Note, for instance, how February is 3.1°F cooler in the current database compared to the historical record. Table created by Ken Towe.

So Why the Difference in current NCDC records and past USWB records?

The basic reason for the difference is that the NCDC has begun to switch over from using what they call the ‘Traditional Climate Division Data Set’ (TCDD) to a new ‘Gridded Divisional Dataset’ (GrDD) that takes into account inconsistencies in the TCDD. Here is a summary of what was wrong with using the TCDD data sets to determine temperature averages and trends. This is a quote from ‘Transitioning from the traditional divisional dataset to the Global Historical Climatology Network-Daily gridded divisional dataset’ by Chris Fenimore, Derek Arndt, Karin Gleason, and Richard R. Heim Jr., NOAA/NESDIS/NCDC, Asheville, NC:

“The GrDD is designed to address the following general issues inherent in the TCDD:

1. For the TCDD, each divisional value from 1931-present is simply the arithmetic average of the station data within it, a computational practice that results in a bias when a division is spatially under sampled in a month (e.g., because some stations did not report) or is climatologically inhomogeneous in general (e.g., due to large variations in topography).

2. For the TCDD, all divisional values before 1931 stem from state averages published by the U.S. Department of Agriculture (USDA) rather than from actual station observations, producing an artificial discontinuity in both the mean and variance for 1895-1930 (Guttman and Quayle, 1996).

3. In the TCDD, many divisions experienced a systematic change in average station location and elevation during the 20th Century, resulting in spurious historical trends in some regions (Keim et al., 2003; Keim et al., 2005; Allard et al., 2009).

4. Finally, none of the TCDD’s station-based temperature records contain adjustments for historical changes in observation time, station location, or temperature instrumentation, inhomogeneities which further bias temporal trends (Peterson et al., 1998).”

The new GrDD is based upon 25 square kilometer (about 9.6 square miles) gridded divisions rather than the traditional regional climate divisions used in the TCDD. The original raw data of specific weather stations has not been changed, but for the purpose of the GrDD analysis the temperature assigned to a grid that happens to include a specific weather station will not match the actual station measurement since the gridded area may include several different weather stations and will also be weighted to take local topography and urbanization (heat island) issues into account. The complete transformation to the GrDD from the TCDD is slated to be complete in 2013 but has been used to some degree already since about 2007.

The impact on the long-term climate record is summarized here (from the same document cited above):

IMPACTS (PRELIMINARY FINDINGS)

Using the methods mentioned above, we identified the climate divisions, states and regions for which the data are impacted the most by this transition. In the aggregate, the GrDD dataset is slightly cooler and wetter when compared to the TCDD, but significant regional differences are apparent (Fig. 1). Typically, the greatest deviations occur in areas with large elevation differences, divisions which border Canada and Mexico, and divisions lacking a large network of stations early in the record.”

Here are two maps illustrating how the change has affected the long-term means for temperature and precipitation by traditional divisional units in the U.S.:





The report concludes with the following comment:

CONCLUSION

”The GrDD is a modern, quality-assured database that improves upon the historical monthly temperature and precipitation data that are currently available with the TCDD. Use of these data will improve our understanding of observed changes in climate across the contiguous U.S. (CONUS). Regression techniques used to derive pre-1930 divisional data in the TCDD have been replaced by real station data in the GrDD, improving comparisons made to early 20th Century data. Because of the different algorithms used, slight variances in temperature and precipitation averages may be seen throughout the data record. The average change in trend was about 0.06°F per century. The annual temperature trend in each division is between -0.3°F and +0.3°F per century and only three climate divisions had differences in their mean larger than 0.3°F per century. In terms of precipitation, the mean change in slope is slightly negative for the annual period”.

A Short History of U.S. Weather Data Collection

The first collection of weather data statistics published in the U.S. appeared in 1857 when a book titled ‘Climatology of the United States and of the Temperate Latitudes of the North American Continent’ written by Lorin Blodget. The book contains detailed monthly and annual temperature and precipitation statistics for about 200 or so locations in the U.S. collected from a variety of different sources (mostly military and the Smithsonian Institute) going back in some cases to the 1780s (and made by various individuals).



Title page of Blodget’s book that included the first published U.S. climate statistics for the U.S.

In 1874 the Smithsonian Institute relinquished its weather data collection responsibilities and passed them on to the Signal Service of the Army. The U.S. Weather Bureau (USWB) was established in 1890 under the umbrella of the U.S. Dept. of Agriculture in 1890 and assumed most of the weather data collection responsibilities. In 1895 the Weather Bureau instituted a homogenization of data collection (meaning it adopted a uniform plan of observations for all it weather sites across the country). In 1906 the USWB published its first comprehensive volume of U.S. weather data in ‘Bulletin Q: Climatology of the United States’ edited by Alfred Judson Henry. This book contains complete weather statistics for every site in the U.S. that had both temperature and precipitation (about 650 stations in all plus about another 1000 that collected precipitation data only). In 1970 the USWB became the National Weather Service (NWS).

Here is a short timeline summary of the above (note the overlapping of various agencies over time):

1820-1890: Medical Dept. of the Army military posts

1849-1874: Smithsonian Institute cooperating observers

1870-1905 (approx): Signal Service of the U.S. Army

1890-1970: U.S. Weather Bureau, Dept. of Agriculture

1970-present: National Weather Service (NOTE: the NCDC was established in the 1950s and is part of NOAA not the NWS).

There are currently about 5700 COOP, ASOS, and NWS sites that collect both temperature and precipitation data and an additional 5-6000 that collect precipitation data only.

It will be interesting to see how the changes in weather data analysis will affect the temperature and precipitation long-term averages across the contiguous U.S.

KUDOS: Ken Stowe for his diligent research in comparing the historical weather data records.

Christopher C. Burt
Weather Historian