11
$\begingroup$

As someone who has to teach courses in computational science, I am confronted with the age-old question: how do I evaluate the ability of the students to learn a subject that depends on applications that are difficult to test with "standard" testing methods (written or oral exams)? Part of the course does depend on understanding the theory and methods on an abstract level, and for that, I'd like to continue to use a written test for those concepts. However, testing understanding of practical use of these methods requires a different approach

Given the natural challenges associated not only with the proliferation of different platforms (for MATLAB, Modelica, Mathematica, and other languages) but also with Internet connectivity and test security, I would be interested in new or original methods for practically evaluating students' understanding of numerical methods. (Features promoting test security are particularly desirable.)

EDIT: I should also mention that the class I am teaching is an introductory-level course, so the students have a relatively small knowledge base to work from.

$\endgroup$
2
  • 1
    $\begingroup$ Allowing them to work on a (mini?) project of their choice, editing/critically reviewing open source codes like PLASMA/MAGMA/LAPACK/ScaLAPACK? I'm talking from the perspective of a student. $\endgroup$
    – Inquest
    Mar 6, 2012 at 16:27
  • $\begingroup$ Thanks for the comment—it reminded me that I forgot to mention that this is an introductory course, so I'm not supposed to be bringing in concepts like parallel programming and performance optimization—just a focus on the basic numerical methods and algorithms. $\endgroup$
    – aeismail
    Mar 6, 2012 at 17:36

4 Answers 4

5
$\begingroup$

Here's what I've seen as a student and teaching assistant:

  • Projects in programming and numerical methods were good in terms of bringing lots of concepts together and forcing me and other students to be creative. However, with computational projects, it's probably worth devoting a little bit of time to software carpentry skills so that students write code that's more organized. When I was an undergrad and didn't know any better, I had cut-and-paste spaghetti code that was probably hell on graders. You may want to avoid that fate by teaching them some good practices. Also, if you give everyone similar enough projects (e.g., calculate all of the thermodynamic properties in a given list for a set of compounds was one term-long project in an undergraduate thermodynamics class; later on, it was a week-long homework assignment in graduate thermodynamics), basically expect them practically copy off of each other and debug each other's code.
  • Homeworks, weekly or biweekly, were the best short-term method for learning new methods and concepts. It's easier to program something up, given a week to do it. Again, expect them to more or less copy off each other and debug each other's code.
  • Quizzes weren't really good for anything other than a couple short methods or analysis questions. You can't do any programming on quizzes, but there should also be less cheating, I mean, collaboration. You could also test pencil-and-paper coding in quizzes, which is good for basic concepts, but probably unfair for advanced concepts, or anything that requires very specialized commands, because students would have access to documentation if they were coding on a computer.
  • Exams were more or less the same as quizzes, if they were given in class, but longer, and more difficult. I have had classes give take-home exams in computational work, in which case you can ask more computationally-oriented questions and expect them to program to solve problems. However, take-home exams have the same sort of problems as homeworks and undergraduate-type projects, in which case, it's probably better if you establish a more draconian collaboration policy for take-home exams. I've had some really good take-home exams, so I think these can work well if the instructor is sufficiently creative.
  • Computational Labs are less effective than the analogous wet labs in science classes, because with a computer in front of you, it's much easier to goof off. There were a few guys in my classes who always spent time in computational labs playing online poker. These labs are probably most effective as demonstrations, or as supervised lessons on practical skills in computational science if you have enough teaching assistants to roam the lab and make sure that people are getting help and no one is screwing around online.
$\endgroup$
5
$\begingroup$

I have lectured, taught, or assisted in a variety of courses relating to numerical and computational methods, from the upper level undergraduate to the advanced graduate level. Here are the elements I have found to be helpful as an instructor:

Research Projects

For advanced classes, a research project (incorporating numerical research, usually some software development, and a write-up) are a very nice way for students to tie in their research with their academic coursework. I think that a project should be mandatory in graduate level courses, but for undergraduates they are better replaced with more directed work.

Programming Homework Assignments

The core of any computational science class is accessible programming assignments. For students with no programming experience you will need to back up your assignments with some introductory sessions to the programming environment, and ideally some sort of "help room" offered either by your department or a student organization such as SIAM. Allowing multiple frameworks and programming languages can be difficult, I accepted programs written in any language but supported only one environment, usually freely available in the University computer laboratories (operating system, editor, shell, interpreter, etc...)

Quizzes

I really like short 10-15 minute in-class quizzes once every week or every other week. It is good two-way feedback: the students see how they are doing against my expectations and against each other, and I see which concepts they are hitting and missing. This style of evaluation is not very commonly used in Europe, and I think this is a shame.

Examinations

Examinations are pencil and paper, with analysis of algorithms, code fragments, and mathematical techniques. I have never participated in a computer laboratory examination, either as a student or an instructor/evaluator. I think the closest thing I have seen is requiring a student to demonstrate their homework or project as well as answer questions about design or implementation.

The Dishonesty Constraint

Both as a student and as an instructor, I have seen enough dishonesty in the academic system to avoid relying on honor for more than 50% of a student's grade. This means that evaluations such as projects and homework, where access to external resources could lead to academic dishonesty, do not contribute to more than 50% of the course grade.

$\endgroup$
9
  • $\begingroup$ I think the final point is especially important—I will still have a written, in-class exam to cover the points that are testable in that format. That will be at least two-thirds of the grade, I think. I will also make sure that there is a write-up that accompanies the out-of-class project, so that students have to spend at least some time engaging with the material. I might also do some randomization of inputs and changing of the problem to cut down on the temptation and ability to cheat. $\endgroup$
    – aeismail
    Mar 8, 2012 at 11:24
  • $\begingroup$ @AronAhmadia: Weekly quizzes are standard in German universities across departments despite their complete independence. $\endgroup$ Mar 8, 2012 at 15:34
  • $\begingroup$ @aeismail: Americans have a very narrow (bordering on the ludicrous in some cases) understanding of plagiarism. The first year you may be able to keep students from collaborating, but the following years, the Fachschaft will have catalogued your assignments and it will become progressively more difficult to pose problems that haven't been solved to a substantial degree in previuos years. $\endgroup$ Mar 8, 2012 at 15:39
  • $\begingroup$ @Deathbreath: (to Aron) Weekly quizzes are not standard here in Aachen—at least not in Mechanical Engineering. I don't even know if I'm allowed to give them. (Not that I'd want to.) $\endgroup$
    – aeismail
    Mar 8, 2012 at 15:51
  • $\begingroup$ @Deathbreath: (to me) If students use past exams to learn the methodology, I'm happy to let them use them. I'm more concerned that they learn how to use the tools. That said, I am changing the course content from year to year, so anybody who gets a perfect score in the old stuff and a goose egg in the new stuff is probably somewhat suspect of relying too heavily on the Fachschaft. $\endgroup$
    – aeismail
    Mar 8, 2012 at 15:54
5
$\begingroup$

Some of the other answers suggest individualized projects. I'm doing that in my finite element software class and it's a lot of fun; I believe it's also really instructive for students. At the same time, it's also hugely time intensive: last time I had 18 students and it was practically a full-time job for the entire semester to supervise these projects. So one has to have a small enough class to make that work successfully.

$\endgroup$
1
2
$\begingroup$

In my honest opinion, I think the ultimate test is to apply your knowledge in unfamiliar territory. I would propose applied projects designed to test a students' ability to select effective models, discretization, approximation/solver methods, exploitable parallelism, error estimates and numerical analysis, as well as visualization methods to describe a particular physical phenomenon of computational interest. I would go further as to ask students to justify each choice based upon the problem size / expected accuracy. The key is to know which methods are appropriate under the constraints of the phenomena under investigation. Students may opt to select a phenomenon of their choosing. But if you want to make it even more challenging, assign each student a computational project in an unrelated field to their dissertation research.

$\endgroup$
1
  • $\begingroup$ My students are second-year undergraduates, so they're a long way from having a field of their dissertation research. :-) But the thought is definitely appreciated. $\endgroup$
    – aeismail
    Mar 8, 2012 at 11:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.