Priorities for learning computational methods, when should I write my own code vs. using libraries as a beginning graduate student?

Question

I am beginning my graduate studies in engineering and will be working on computational science projects. I noticed that there has been some discussion about the advantages and disadvantages of implementing your own algorithms here. Is the cost-benefit analysis of implementing your own algorithms versus using libraries different when you're a beginning graduate student?

Answer 1

In my opinion, being a beginning graduate student doesn't change the answer by David Ketcheson here to the question you've linked in your post.

Code minimal versions of algorithms you want to learn. Then set them aside. Coding your own algorithms is most useful for learning, but for research (or production) code, unless your research goals are to write software that improves upon the state-of-the-art libraries out there (if any even exist), you're better off using libraries. Libraries are likely to be better documented, more scalable, and more robust than what you code yourself, unless you are (or become) really good at coding. Also, libraries are likely to be tested and debugged for you (though of course, that depends on who wrote the library...). You will be responsible for supporting, debugging, and testing any code you write for your thesis, and to save time, it helps to minimize the amount of code you need to write.

The only other scenarios that I can think of (i.e., there may be others) are:

• There aren't any libraries out there that provide the features you need. Consider contributing to existing open-source libraries or writing your own open-source library, so that others may benefit.
• You need additional performance and can leverage special problem structure to get it. Then document and modify an existing library, or write a higher performance implementation yourself.
• A supervisor insists that you roll your own software. Well, you're on your own there, but I suggest bringing up the points above to try to convince them that you'd be better off using libraries (if that's at all possible in your situation).

Answer 2

I'd like to give some more breadth to Geoff's thoughtful answer. In particular, I want to give you a little more perspective on the value of your programming efforts as opposed to your research efforts in your early career as an academic.

You will find that being able to write software to augment your scientific research will make you a valuable member of almost any research team. However, this time will not be necessarily be considered "valuable" by your academic peers or those hiring for academic positions.

From a 2011 research survey conducted at Princeton, "A Survey of the Practice of Computational Science":

Scientists spend substantial amount of research time programming. On average, scientists estimate that 35% of their research time is spent in programming/developing software. While initially some time is spent on writing code afresh, a considerable portion of time is spent in many tedious activities. For example, researchers in Politics and Sociology who used R/Stata had to do considerable programming to retrofit census data into formats that individual packages in R/Stata understood. Some researchers in Chemical Engineering had to reverse engineer undocumented legacy code that performed flame simulation, long after the original authors had graduated, in order to adapt the code to newer fuels... Despite this, a vast majority of these researchers felt that "they spend more time programming than they should," and that research time was better spent in focusing on scientic theories or on experiments ("more concerned about physics," said one researcher).

That doesn't mean that it is not a good idea to implement or redesign a core library or applications, but if you are going to engage in any serious software development (more than 25% of your time working with code), keep these three thoughts in mind.

• Complexity and risk grow exponentially with project size and the number of developers. Until you have written or worked with larger pieces of software or teams of developers that extend beyond your lab, it will be difficult for you to gain a good appreciation of this and properly forecast effort.

• You need to be good. It takes a certain amount of maturity, both as a programmer and as an application scientist, to write useful software. You have to know what the important features are, where the numerical risks are, and be able to forecast programming effort for a given set of features and robustness. Of course, the only way to get good is to spend time on projects that you aren't the lead on or that can safely fail or be delayed, which brings me to my final point.

• Although many research laboratories and industrial positions highly value programming experience, scientific programming can act as a potential detriment to your academic career, even if your software benefits science more than your papers do. All that time you spend learning how to program well, programming, documenting your code, and making it robust translates into papers that are not being written. An advisor will not always have their student's best interests in mind here, as this is one of those cases where the student can provide work that benefits the advisor's group without benefiting the student's citation count. Seek out one or more trusted mentors in the field you are interested in and make sure you have a clear understanding of what contributions are considered valuable. academia.stackexchange.com is an excellent place to ask a follow-up question on this.

---

As a footnote: the number of one-man effort projects that significantly advance any computational field is steadily diminishing, be it an application area or something more technical such as dense linear algebra. An increasing number of the software packages that form the "bread-and-butter" of computational research are 10 years older or more. Scientific code that has not reached this level of maturity tends to have more bugs, less features, and sparse documentation. Try to avoid working with immature code that is not actively supported, regardless of how old it is.

Answer 3

I think the cost-benefit analysis depends upon the desired scientific computing literacy that you want to achieve during your studies.

For most scientist who work with computers being proficient in scientific computing is enough, this requires: a high-level (abstract) understanding of the main algorithms used, and programming skills that allow you to effectively use software libraries (building software, linking, using mailing lists).

In contrast, if you plan to become an expert in the scientific computing field, you will need a deep understanding of numerical methods, floating-point arithmetic, and computing technology. You can learn theory of these topics from books however experience through practice is needed to build and maintain advanced skills. Hence, it might be a great idea to program all you use while you learn (e.g. if you want to cook like a Chef: you learn by eating what you cook, and by cooking often!)

What is the correct level of proficiency depends on your career. See what level of skills are used by working people in your field.