Climategate: Not so much the emails as it is the Data- The HARRY_READ_ME.txt file

[revolutionary8]

the HARRY_READ_ME.txt file

http://www.anenglishmanscastle.com/HARRY_READ_ME.txt (http://www.anenglishmanscastle.com/HARRY_READ_ME.txt)

http://cbullitt.wordpress.com/2009/11/22/the-harry_read_me-file/

Love the ironic title. Got this from reader, Glenn. I’m out of my depth trying to read the code–and apparently so were several folks at CRU. If what he, and the techies at the links, say is true, it’s no wonder they had to spin this for 10 years–it’s all absolute bullshit.

Here’s Glenn’s take with links:

The hacked e-mails were damning, but the problems they had handling their own data at CRU are a dagger to the heart of the global warming “theory.” There is a large file of comments by a programmer at CRU called HARRY_READ_ME documenting that their data processing and modeling functions were completely out of control.

They fudged so much that NOTHING that came out of CRU can have ANY believability. If the word can be gotten out on this and understood it is the end of the global warming myth. This much bigger than the e-mails. For techie takes on this see:

http://www.tickerforum.org/cgi-ticker/akcs-www?post=118625&page=13

http://www.neuralnetwriter.cylo42.com/node/2421

To base a re-making of the global economy (i.e. cap-and-trade)on disastrously and hopelessly messed up data like this would be insanity.

Here is also a searchable data base for the climategate emails:
http://www.eastangliaemails.com

I am currently reading through the search results for "james hansen" Not pretty.

Here is a link to a blog that is following several of the above mentioned threads.
http://www.devilskitchen.me.uk/2009/11/data-horribilis-harryreadmetxt-file.html


7. Removed 4-line header from a couple of .glo files and loaded them into Matlab. Reshaped to 360r x 720c and plotted; looks OK for global temp (anomalies) data. Deduce that .glo files, after the header, contain data taken row-by-row starting with the Northernmost, and presented as '8E12.4'. The grid is from -180 to +180 rather than 0 to 360.

This should allow us to deduce the meaning of the co-ordinate pairs used to describe each cell in a .grim file (we know the first number is the lon or column, the second the lat or row - but which way up are the latitudes? And where do the longitudes break?

There is another problem: the values are anomalies, wheras the 'public' .grim files are actual values. So Tim's explanations (in _READ_ME.txt) are incorrect...

8. Had a hunt and found an identically-named temperature database file which did include normals lines at the start of every station. How handy - naming two different files with exactly the same name and relying on their location to differentiate! Aaarrgghh!! Re-ran anomdtb:

wow.

[Oyate]

What I gathered from skimming was the scenario of a programmer coming in to fix another person's screw ups. It also sounds like only a tiny portion of the overall body of evidence has been examined.

So from what I read, all of the data this programmer was looking at was utterly incomprehensible. There was just no obvious way to tell what was in all the files at the outset.

It maybe be these problems were subsequently fixed but how do we know until we get the raw data and try to model it ourselves?

Rest assured, however, that there's plenty of indisputables to destroy the "no serious question regarding global warming" line. They got hit hard in the gut on this one and they won't soon recover.

[revolutionary8]

What I gathered from skimming was the scenario of a programmer coming in to fix another person's screw ups. It also sounds like only a tiny portion of the overall body of evidence has been examined.

So from what I read, all of the data this programmer was looking at was utterly incomprehensible. There was just no obvious way to tell what was in all the files at the outset.

It maybe be these problems were subsequently fixed but how do we know until we get the raw data and try to model it ourselves?

Rest assured, however, that there's plenty of indisputables to destroy the "no serious question regarding global warming" line. They got hit hard in the gut on this one and they won't soon recover.

Yes, that is how I am reading it thus far Oyate, dude was a minion. lol. i guess they have relased 64 mb with another hundy or so to go. It is going to get interesting.

whadayamake of this?
From: Phil Jones <p.jones@xxxxxxxxx.xxx>
To: James Hansen <jhansen@xxxxxxxxx.xxx>
Subject: Differences in our series (GISS/HadCRUT3)
Date: Tue Jan 15 13:17:19 2008
Cc: gschmidt@xxxxxxxxx.xxx


Jim, Gavin,
Thanks for the summary about 2007. We're saying much the same things
about recent temps, and probably when it comes to those idiots
saying global warming is stopping - in some recent RC and CA threads. Gavin
has gone to town on this with 6,7, 8 year trends etc.
What I wanted to touch base on is the issue in this figure I
got yesterday. This is more of the same. You both attribute the differences to
your extrapolation over the Arctic (as does Stefan). I've gone along with
this, but have you produced an NH series excluding the Arctic ? Do these
agree better?
I reviewed a paper from NCDC (Tom Smith et al) about issues with
recent SSTs and the greater number of buoy type data since the late-90s
(now about 70%) cf ships. The paper shows ships are very slightly warmer
cf buoys (~0.1-0.2 for all SST). I don't think they have implemented an
adjustment for this yet, but if done it would raise global T by about 0.1
for the recent few years. The paper should be out in J. Climate soon.
The HC folks are not including SST data appearing in the Arctic for regions
where their climatology (61-90) includes years which had some sea ice. I
take it you and NCDC are not including Arctic SST data where the
climatology isn't correct? You get big positive anomalies if you do.
Some day we will have to solve both these issues. Both are difficult,
especially the latter!
Cheers
Phil

[Oyate]

Yes, that is how I am reading it thus far Oyate, dude was a minion. lol. i guess they have relased 64 mb with another hundy or so to go. It is going to get interesting.

whadayamake of this?

I really don't make anything of it except it sounds like they are trying to identify differences in temperatures from different datasets but a lot of explanatory factors could exist to explain this. So if we get a hundred-odd more mb of data, we might find they have resolved technical issues. Something tells me that might be a case or this whole thing is a lure. A trap. At the very least, there's plenty to dispute and until we can actually model their data, the arguments could go back and forth for years.

Of course this doesn't mean anything in terms of how it will play out in the media which is the field this game is being played on. We already done hurt 'em bad.

[revolutionary8]

Subject

Why are the temperature data from Hadley different from NASA? [18]Link

David and all:
One advantage (or great disadvantage if you are very busy!) of membership in GCCC is
that you are forced to investigate topics outside your areas of expertise. For some time
now, I have been puzzled as to why global temperature data from the British Hadley
Centre are different from those reported by NASA GISS, especially in the last 10 years.
GISS reports that 2005 was the warmest year (see first attachment) on record, and that
2007 tied 1998 for the second place. The Hadley group continues reporting 1998 (a strong
El Nino year) as having the highest global temperature, and then showing temperature
decreases thereafter. The two groups report their temperatures relative to different
time intervals (1951-1980 for GISS; 1961-1990 for Hadley), but much more important is
the fact that GISS data include temperatures from the heating Arctic that are excluded
by others (see second attachment). If you are interested in the topic of sun spots, the
11-year irradiance cycle, and solar forcing versus AGHGs, see the first attachment for
what NASA has to say.
We may need help on this complex topic from a "true climate scientists", such as Judith Lean!
Cheers. Yousif Kharaka
Yousif Kharaka, Research Geochemist Phone: (650) 329-4535
U. S. Geological Survey, MS 427 Fax: (650) 329-4538
345, Middlefield Road Mail: [19]ykharaka@xxxxxxxxx.xxx
Menlo Park, California 94025, USA

<GCC-Data @ NASA GISS_ GISS Surface Temperature Analysis_ 2007.pdf>
<GCC-2005 Warmest Year In A Century.pdf>

[revolutionary8]

I really don't make anything of it except it sounds like they are trying to identify differences in temperatures from different datasets but a lot of explanatory factors could exist to explain this. So if we get a hundred-odd more mb of data, we might find they have resolved technical issues. Something tells me that might be a case or this whole thing is a lure. A trap. At the very least, there's plenty to dispute and until we can actually model their data, the arguments could go back and forth for years.

Of course this doesn't mean anything in terms of how it will play out in the media which is the field this game is being played on. We already done hurt 'em bad.

I see what you are saying Oyate, but I don't really think it's "a trap" the media isn't even reporting on it, b/c their heads are spinning, the people writing the emails have clammed up, and more is coming out every day. Take a look at the data base. I can't wait till people who are familar w/ this code start plowing in to it. I think it was probably an inside job, meaning it was leaked... But that is here nor there. I think the blogosphere will cover this news, and become the trusted source. I hope so, anyway. Pretty soon, it won't matter how it plays out in the media.

From Asimov at the Ticker Forum mentioned in the links above:

Quote:
These are very promising. The vast majority in both cases are within 0.5
degrees of the published data. However, there are still plenty of values
more than a degree out.


He's trying to fit the results of his programs and data to PREVIOUS results.

Quote:
TMP has a comforting 95%+ within half a degree, though one still wonders
why it isn't 100% spot on..


Quote:
DTR fares perhaps even better, over half are spot-on, though about
7.5% are outside a half.


The percentages below is the percentage of accuracy

Quote:
However, it's not such good news for precip (PRE):
...
Percentages: 13.93 25.65 11.23 49.20

21. A little experimentation goes a short way..

I tried using the 'stn' option of anomdtb.for. Not completely sure what
it's supposed to do, but no matter as it didn't work:


Oh yea, don't forget. He's getting 0.5 and 1 degree differences in results... while they are predicting temperatures to a supposed accuracy of tenths...

[awake]

"the people can always be brought to the bidding of the leaders. That is easy. All you have to do is to tell them they are being attacked(the Earth and humanity are in danger from evil polluting capitalists who are causing the climate to change), and denounce the pacifists(deniers) for lack of patriotism(go green, reduce your carbon foot print) and exposing the country(world environment) to danger. It works the same in any country."- Hermann Goering