Computational science contests. Why arent there any?

Question

I was wondering why there are no online or offline computational science contests? At least I couldn't find much by googling. I mean, like a topcoder for computational sciences. I assume one reason is providing massive computational power for few areas of study which costs lot of money. Other might be to formulate right, practical problems as online contests usually have hypothetical questions.

It is interesting that a computational scientist or engineer applies his knowledge across multiple sciences like aerospace, energy, bio, chemistry, physics, materials et all. But they somehow miss the glamor factor of consumer software.

Answer 1

I would say that there are a number of reasons why there are no computational science contests besides the potentially massive computational resources required.

1. Time limits: Writing scientific computing code is usually not something that you want to rush. A lot of emphasis is on making sure it is correct, and thorough consideration of test/corner cases. Also, if the problem being solved is too hard, then you don't want to spend too much of the contest time on waiting for code to run. Also, interesting problems would take quite a long time to solve.
2. High barrier to entry: In order to solve useful problems, you need to have a fairly detailed understanding of the physics (or whatever) as well as the math, and that already severely limits the kinds of people who would participate. For multidisciplinary problems, the set of people shrinks even further.
3. Established best practices: There is often a very small number of good ways of solving a problem (at the low numerical level), so it all comes down to using the right software libraries. Do you allow people to use libraries? If yes, then most of the code is just fluff and wrappers around it. Where do you draw the line at what is a library, and what is a "canned solution"? If you don't allow people to use libraries, then do they have to implement the entire technology stack down to the linear algebra?
4. Verifiability: How do you know that someone solved the problem correctly? You could try to ask them to solve a problem for which there is a known analytic solution. But then, why wouldn't they just figure that out? Comparing it to pre-generated numerical solutions could be problematic if their method is more accurate. Also, how do you decide on the tradeoff between speed and robustness? Shouldn't people be rewarded for writing code that can handle a wider range of problems, but is potentially slower?

It would be interesting if there were such contests. Although, if you managed to get people together to solve an interesting problem, it might be better for the scientific computing community if they just all worked together on it.

Answer 2

There is a lot of food for thought in this question. I would like to differentiate a bit with respect to the character of the contests.

The subject of contests I know is fairly inconsequential. There are spelling bees, when it comes to mathematics, there are contests in symbolic integration, marshmallow eating contests and beauty pageants. You rarely hear about vaccination contests for nurses, sentencing contests for judges, or fire fighter competitions going beyond carrying buckets. As Victor has pointed out, computational science is a serious task, which requires skill, understanding, thought, and perseverance. All these cannot take effect in a short time contest.

What you find more commonly are coding challenges like the HPC Challenge mentioned in Pedro's comment. These take place over extended periods of time, and allow the contestants to use all the tools and information they need to produce a high quality result. They are usually sponsored by corporations (for instance Google) or other big entities because they need the result or use them as a recruiting device. But they have a lot of potential to raise interest in scientific computing in young people and to give them exposure when they succeed.

A third option are prizes for already existing accomplishments. Scientific awards have seen a considerable inflation during the last years, but by many are still seen as an important indicator for scientific achievement. Compared to all the medals that are awarded for instance in pure mathematics, computational science could definitely do with some more.

Answer 3

In addition to the other comments already made, let me point out that the complexity of challenges has not deterred such contests to happen in other areas: There was a $10M prize to the first group building a vehicle to go up 100km. Clearly not something you can do in a week in your garage. Several of the other X-Prize challenges are equally difficult.

There have also been contests in areas of scientific computing that just happen to not quite be solving PDEs. For example, Netflix has sponsored a competition to predict which movies people would like, based on their ranking of previously watched movies. I believe I've heard of other similar challenges in machine learning before. In numerical methods, the 100-digit and 10-digit challenges have already been mentioned.

Answer 4

I know of one contest to predict crystal structure.

Answer 5

It happens occasionally for related subfields. For instance, in the realm of mathematics and numerical analysis SIAM's Hundred Dollar Hundred Digit Challenge challenged participants to write a codes to compute mathematical problems to 10 digits of accuracy. It a very nice idea that got a lot of good publicity and entries (not to mention, remains a good lesson to teach students who are learning numerical analysis).

I feel that large scale numerical problems are not a good vessel for these types of contests. The reason the SIAM 10 digit challenge worked out so well was because the problems were all deceptively simple, but if you brute-forced it, you wouldn't get very far. The trick was to write out a short code (often less than a page) that did the problem in a unique way.