Who is responsible for QA(Quality Assurance), i.e. testing, the GCM's (Global Climate Models), who SHOULD be?

Question

Who is responsible for QA(Quality Assurance), i.e. testing, the GCM's (Global Climate Models), who SHOULD be?

Answer 1

I really don't give a crap what models say...



I live in Jeddah Saudi Arabia on coast of red sea Somewhere around Latitude 21.35'

Longitude 39.10'



This isn't my first year here I have lived here for past 30 years now.



Its Summer, August, start of August temperatures are suppose to be around 45 max and 30 min. (thats in Deg C)



I'm trying to lose 3 Kgs so I go for a walk every night, this isn't something new for me I would go for a walk every night even if its 35 Deg C and 80% humidity. The most amazing thing this winter is we have temperatures of



40 Deg C Max

26 Deg C Min

And humidity of max 65% during night and 45% during day.



This might be quite hot for few but I remember the days during every July, August and start to mid of September that it was impossible to breath due to humidity, just walking from car to home or office was a killer.



I'm not really missing that but something else is missing, which is global warming.



Frankly I would say enough of lies, even if you show high temperatures around the world just because you control the media and those scientists who are payed by you, you still can't tell me that every place in the world is showing signs of cooling other than you satellites? This is the biggest lie I've ever come across in my life.



I actually have an excel sheet with me that I can use to predict temperatures around the world by tweaking simple parameters, I really can use it to show global cooling, heating or whatever you want.



Computer models remain computer models, they can never replicate the real thing.



I was reading interview of one of the designers for F22 Raptor, he said something weird that till today they use wind tunnel to do the final testing of airflow on the air craft, so someone asked him why don't you use computers for that? He said there aren't enough powerful computers available today to replicate the wind effect.



This is Lockheed Martin/Boeing a company that does nothing but make aircrafts they say they cannot replicate wind tunnel and some people who can't even figure out tomorrows weather perfectly will tell me that their computer model could predict world temperature 4-5 years in advance when it has failed to predict that its freaking too cool for Summer here?



Bunch of lies!

Answer 2

Maybe I'm missing the point, and I freely admit I don't know much about software, but exactly how would the occasional error or bug make the model inaccurate?



And if there is a serious bug in one model, wouldn't the chances be next to nil that the exact same bug would occur in another similar model? So wouldn't the use of multiple models easily eliminate the possibility that there is a bug that happens to make the predicted temperature anomalies much higher than they would be without this bug?



Or maybe I'm misunderstanding you completely. What kind of errors are you talking about and what might happen as a result of them? And why wouldn't the use of multiple models completely account for the chances of a significant random bug in a single model?

Answer 3

Well all of these models account for as many things as they can. Some other things include ocean currents, greenhouse gas concentrations, and aerosols. Answer below mine quite wrong, while you cannot say a single event was caused by global warming, it is a fact that warmer oceans cause stronger more powerful storms.

Answer 4

I do not find this at all surprising as I found a major set of calculation errors in a major SPC package specified by major auto manufactures to their suppliers. The errors are in the X- calculation and ends up requiring that one achieve a Cpk of 5+ in programs using AIAG formulas order to get a Cpk of 1.33 in theirs. I worked out a calculation converter that would allow me to figure the existence of the calculation error, but without access to their source code correcting it was impossible and they would no admit to its existence.

Answer 5

The first answer given to this question:



'It sounds like most of the AGW crowd believe that once a code is written there is no mistakes and the output is correct if it proves what they want."



is written by someone who is either intentionally lying or completely out-of-touch with climate modelling, or both. (I pick the latter, actually).



Researchers spend an ENORMOUS amount of time testing models, fixing them, et cetera. Anyone that would believe the quoted statement has probably never written a piece of code in his life, and has no business attempting to answer such a question.



There is more than one type of peer review. The one that gets talked about all the time on here is peer review as practiced at refereed journals. At organizations of any size there is also internal peer review, which is where a lot of the code-checking will go on. It may be years between when a climate modelling code development starts and when results are published. During that time there will be constant error-checking, refining of the code, internal review, etc.



EDIT: Beam, you're awfully testy about this. Did you get fired by NCAR or something? I don't know what sort of software you work on, but some software gets tested extensively, some does not. There are all sorts of different ways of testing software, with different goals in mind. People have spent many years using computers for atmospheric modelling. You're probably unaware of this, but that was pretty much the first application of large computers, back when "programming" was done with vacuum tubes rather than Visual Basic or whatever it is that you program in.



I worked for six years at a simulation software company, so yes, I have participated in software testing. The testing on GCM's goes WAY beyond anything I saw there. If you think you're such hot stuff, why don't you download the code for the Community Climate System Model (CCSM) and get to work testing it? You can get it here:



http://www.ccsm.ucar.edu/



Play with it, test it, poke holes in it, add new physics to it, just stop making the assumption that people with Ph.D.s that spend their lives doing this are stupid. I think because other groups do things differently than what you have been trained to do in your software engineering courses, you think they're wrong.



If you're advocating that more money be spent on programmers for climate research, I'm all for that. I don't do GCM development, but in my career I have done programming in Fortran, GW-Basic, VB, PV-Wave, IDL, Matlab and NCL, and I'd LOVE to have some programmer lackey doing that stuff for me. I'm just a hack at programming and if you're saying that I need to have someone writing and testing code for me, I'd love it. That way I could spend more time on the science.



Another EDIT: No Beam, I believe you are naive for thinking that software testing is only done one way or with one goal. I see you only wanting to criticize software development that I don't think you really know anything about. I gave you the link to the CCSM, use that as a test case for your own GCM software test suite--show the climate scientists how it should be done! If you're only willing to bad-mouth other people's work while doing nothing concrete of your own, why should anyone respect what you are saying?



Once more for Beam: Yes, I could tell your education was in software engineering. Bully for you. You don't think there are plenty of software engineers with all the training that you have had working on climate models? Testing is great, I'm all for testing. Just because YOU are not aware of it doesn't mean it doesn't go on. I'll even say that every climate model has bugs in the software, that's true with any large complex piece of code. Atmospheric models can be some of the largest and most complex pieces of code around, and need to be tested extensively and they are. You just don't believe it, and you don't really want to find out the details, which I'm sure you could by looking at various websites for the groups that put together the models and contacting them. You'd rather just say that you're smarter than they are and try to throw doubt on their results with nothing to back it up.



Final EDIT: The only assumption I'm making is that the software engineers and climate scientists working on climate models know a lot more about them than people that don't. And again, I don't claim any special knowledge of computer science or software engineering, I only do what I have to (I don't generate climate models). YOU'RE the expert in software testing, go do it, stop complaining about what other people are doing. The models are out there, go for it!

Answer 6

It sounds like most of the AGW crowd believe that once a code is written there is no mistakes and the output is correct if it proves what they want. If it gives them some other response then they can change the code to get it to give the answer that they want not necessarily the correct answer.

Answer 7

It is funny how people who have no information about the simulations can actually know so much about them.

Answer 8

Yes, I couldn’t agree more.



I wonder what the result would be if these GCMs were put through the same double-blind tests that drugs companies have to go through.



One team supplies the GCM. Another team supplies the parameters to go into it. And a third team runs the model to produce the result. None of the teams have any knowledge of who they are working with.



I can’t help but think that the whole Global Warming panic would disappear almost overnight if this were to happen – which is why the Global Warming Lairs are making sure it never does, of course. Hardly very scientific.



Would you happily take a drug on the strength of the claims and promises of the manufacturer?



Of course you wouldn’t. You’d insist that the drug is properly tested first. That’s science.



With Global Warming, you are expected to trust the word of the people involved. That’s religion.



As ever with Global Warming - don’t believe the hype.









:::EDIT:::



Response to Dawei.



You raise a very interesting point in your middle paragraph that applies to many aspects of Global Warming, not just computer models. What you’re saying is that, all things being equal, around 50% of the “problems” with any data will show too much warming, while around 50% of the time they will show too little warming, or even cooling.



Is that what you’re saying?



The problem with the issues you mention is that, in Global Warming, the “bugs”, “mistakes”, “errors” etc, are only ever found when they throw up anomalous data that shows too little warming. If the result of the data is the warming that was expected, then the attitude is just “That’s what we expected, so that’s fine.” And no one checks the data.



Think about it; how many times have you heard of data in the area of Global Warming being flagged as anomalously too high? It almost never happens – and when it does, it’s the sceptics that find it, not the alarmists.



But, as you point out, it should be a 50/50 split, shouldn’t it?



Take the ARGO data as a recent example. It initially showed rapid ocean cooling, so the data was checked and checked until errors were found. After correction the result was still slight cooling, so the checks continued. I’m not actually sure what the current situation is, but realclimate.org is saying that the ARGO data now shows warming – though I’ve seen a recent paper that says it’s showing cooling still. Regardless, the point is; had the data showed warming as expected, or even higher than expected, do you honestly think that the problems would have been found?



Just look at the GISS “corrections” to the global temperature data. All things being equal, the positive corrections should be matched by similar negative corrections, but that’s not the case is it? The “corrections” amount to a shocking +0.5⁰F!



So, in summary, what you’re saying is absolutely correct, but, sadly, Global Warming “science” (I use the word very loosely) is so corrupt that the errors are only ever spotted (or even looked for) if they result in data that contradicts the Global Warming hypothesis.



Are you comfortable with that situation Dawei?





As a final comment, I note with astonishment that (at the time of editing) I have 2 thumbs down for the first part of my answer. So at least two people think it would be a bad idea to have double-blind tests done on CGMs to ensure that the predictions they are making are accurate.



I rest my case!