I've brought on Larry Ramey (and am hoping to bring on Eric Raymond) to give a walk through of what that software does and what its data set means. Ideally with both of them compiling it, running it and cross checking each other.
Larry and Eric are both pretty blunt at what they consider bovine scatology; neither of them is *likely* to be posting, but both will be looking over my posts before they go up. They also take opposite views on the issue....aside from saying that both sides should open source everything.
If, in the private review section, they can't agree on something, I'll probably put three posts out - my post, one summarizing Eric's objections, one summarizing Larry's objections and so on.
One of the reasons why some stuff in climatology is NOT open sourced - including data sets - is because it's funded by private sources and is very much commercial information and practice. Larry asked me what my response would be to someone who had a model that had a mixture of public and private data, some with restrictions on it.
My answer was "I'm sorry, but unless we can show all the steps along the way, we cannot run it - it's outside the purview of this blog." It is my hope that by showing that, yes, a bunch of talented amateurs can run the open source stuff and give a reasonable explanation of what each model is doing, that we can 'clear the air' a bit. Perhaps, about the time of the fifth Assessment Report, we can post an honest to god literature analysis.
That being said, there's a LOT to the UCAR modeling stuff. Indeed, there may be more there than two very smart people can independently confirm or analyze. It's a huge data set.
There will be periodic updates on the progress of that thread, but more articles here and there about what data sources are, signal and noise, just so there's something to read here.