Scientific computing vs numerical analysis

Question

I'm a double major in computer science and mathematics. I love both subjects. I'm thinking in taking a graduate career, perhaps in scientific computing. What's the real difference between scientific computing and numerical analysis? Are they studied as careers?

Answer 1

Wikipedia gives a good definition

Numerical analysis is the study of algorithms that use numerical approximation (as opposed to general symbolic manipulations) for the problems of mathematical analysis (as distinguished from discrete mathematics).

Numerical analysts are typically interested in proving mathematical results about their algorithms, including error bounds (how large can the error in the approximation be), convergence of iterative schemes (does the approximation approach the correct limit), order and rate of convergence (how fast does the algorithm converge), and computational complexity (bounding the number of operations required by an algorithm.) It's possible to do research in these areas without ever using a computer, and some important results even predate the development of digital computers in the 1950's.

Wikipedia also has a definition for "Scientific Computing"

Computational science (also scientific computing or scientific computation (SC)) is a rapidly growing multidisciplinary field that uses advanced computing capabilities to understand and solve complex problems. Computational science fuses three distinct elements:[1] Algorithms (numerical and non-numerical) and modeling and simulation software developed to solve science (e.g., biological, physical, and social), engineering, and humanities problems Computer and information science that develops and optimizes the advanced system hardware, software, networking, and data management components needed to solve computationally demanding problems The computing infrastructure that supports both the science and engineering problem solving and the developmental computer and information science.

Scientific computing is much more about practical aspects of getting accurate solutions out of computers. This obviously builds on the results of numerical analysis, but it also draws heavily on computer architecture and software engineering. Although research in scientific computing is often done for its own sake and to develop hardware and software that will be of use in many applications, there is also a lot of scientific computing research that is driven by the need to solve particular science and engineering problems. For example, the development of global climate models to study climate change has also moved scientific computing forward.

Numerical analysis is most commonly found in mathematics and applied mathematics departments, while scientific computing is an interdisciplinary field that can be found in computer science departments, mathematics departments, and in the various engineering and science disciplines.

Answer 2

As someone who moved from Engineering to Scientific Computing during Grad school as an incidental need of the kind of work I was doing here are my two cents:

• Numerical analysis would focus on the math and algorithms side of things. Figuring out what techniques to use to solve a particular mathamatical problem that does not have an analytical solution e.g. ODE's PDE's Matrix Manipulations Optimization etc.
• Numerical Analysis would these days often involve significant amounts of programming but still it is pretty much translating the mathematical ideas of an efficient algorithm into computer code.
• Traditionally FORTRAN was the mainstay. But you could also expect to work with C / C++ and these days Python. Some stuff might also involve packages like Mathematica or MATLAB
• Coming to Scientific Computing that's more an applied area where one tries to use computing resources to solve some scientific problem. This may involve a lot of the nuts n bolts work. e.g. Compiling codes, installing operating systems and libraries, setting up options to make scientific code work etc.
• Since a fair bit of scientific computing these days involves parallel computations you will most likely have some exposure to computing clusters, supercomputers, cloud computing etc
• In scientific computing while you may work with programming languages like C / FORTRAN etc. expect to work a lot with the "gluing" / scripting languages like bash / perl etc.
• You'll probably work a lot with Linux-ey systems and end up fairly proficient working on the command line & with tools like sed / awk etc. Some people end up being sys admins.
• Lot of scientific computing involves visualization and data storage / data retrival. Many people end up becoming experts in Big Data / Hadoop / Map Reduce etc.
• Numerical Analyisis is essentially a specialist job. You get good at math and coding and solve a specific problem very efficiently. Sometimes inventing an algorithm or two along the way. Scientific Computing is, in some sense, a generalist job. Relatively speaking. You are often using diverse tools together to solve a specific applied problem.
• A lot of scientific computing can involve working at the interfaces. e.g. Interfaces between two programs. Where you pipe data from one tool to another for processing. With some format manipulation along the way. i.e. You are trying to get diverse tools to talk to each other where the tools were not really designed to talk to each other.
• A scientific computing guy often will have to master various data formats. Many instruments will have their own proprietary formats and someone has to decode the data into a format that the numerical algorithm likes.
• Some Scientific Computing guys end up manning "helpdesks" of a very specialized nature (well paying too) where one essentially helps a generic researcher / student / Prof. use the computing resources at an institution to solve whatever problem might pop up. i.e. The scientific computing guy is the one familiar with a variety of codes and packages and able to advise a user on what tool to use to best solve the problem computationally.
• You can end up porting codes to other hardware. Or parallelizing legacy codes that were written in serial mode. Or optimizing codes to run faster. Some guys will convert codes to run in GPUs / CUDA etc. to make them run faster.
• A fair bit of scientific computing involves troubleshooting. Often codes that other people have written. To figure out why they crash on certain hardware etc.
• Often you are the middleman liasoning between specialists. e.g. I've had to work on teams with hardcore programmers, biologists that need computation but cannot code much themselves, sys admins, network gurus, technicians manning data centers etc.
• Scientific Computing guys can be asked to give significant inputs when new hardware is purchased or the architecture of a computing system is decided. On those assignments you end up working very closely with sales engineers and technologists from Dell / Cray / IBM / Infiniband / Cisco etc.

Hope this gives you some idea about the fields!

One final bit of advice (take it with a huge pinch of salt!): If you are good with math, like precision and detail and reading papers and figuring out details after significant, focused effort where individual intelligence matters and involves long periods of sustained effort then go for Numerical Analysis.

On the other hand, if you like being a generalist, switch areas, compensate for genius with hard work, be a jack of all trades, be willing to work with fuzzy and vague often conflicting recommendations, like to work with teams and deal with conflict, tight deadlines, deal with MBAs etc. then go become a Scientific Computing guy.

Again take this with a huge pinch of salt. Every person's circumstance is special. And most of us landed where we did out of sheer coincidence and not because we planned it that way. :)

Answer 3

This essay by David Bindel is very good.

An excerpt:

If numerical analysis is about the mathematical aspects of numerical method design and analysis, scientific computing is about the aspects that involve the computer. When I worry about cache architecture, or when I parallelize numerical methods, or when I build little tools to automatically generate parts of my scientific codes, I work on scientific computing. The relationship between scientific computing and numerical analysis is, I think, similar to the relationship between theoretical computer science and systems design -- they're two distinct areas, though the line between them is fuzzy and neither would thrive alone as well as they thrive together.

Answer 4

From my experiences at the beginning of scientific computing as well as the beginning of all electronic computing languages and hardware, i.e., early 60s, these are my thoughts on numerical analysis vs scientific programming.

I worked on liquid rocket combustion instability problems. Combustion instability results in pressure waves running around the rocket combustion chamber that upset wall heat transfer (can destroy engine) and can set up flow/pressure instabilities in feed systems (can also destroy engine). The ultimate model to analyze these problems would be a transient (high frequency) solution to the 2 phase Navier-Stokes equations. At that time an impossible goal. Now it is attainable.

A numerical analyst along with the Principal Investigator would be looking at the discretization of the equations into finite difference algorithms and the selection of processes to incorporate into the model and the tradeoffs between them. The analyst would be looking at what solution techniques (numerical recipes) might be selected to give the best approximation to the phenomena to be modelled recognizing the hardware limitations.

A scientific programmer would be looking for the most efficient way to implement the solutions on the computer, i.e., coding methodology and hardware limitations (memory management, etc). With the primitive hardware of the time that was a major issue. Sometimes runs took days to complete on mainframe machines.

A major issue was how to ascertain a good combustion instability analysis scheme from numerical instability caused by an inaccurate solution methodology. This required close collaboration between a numerical analyst and a scientific programmer. In fact, many times it was one person filling both roles.

Answer 5

You want to be an applied mathematician. This is a major available at many universities in their engineering school also as a PhD. Though in my experience, mathematics is the more important skill, some technical background in formal computer science can be helpful and you may not get it elsewhere. Just be careful not to go to far down the CS rabbit hole or you may wind up doing CS research which tends to pull you away from practical applications.

To answer your specific question, an applied mathematician will do research in numerical methods and analysis and possible partner with scientific researchers to do scientific computing in several areas. If you want to pursue scientific computing specifically, you may want to enter a scientific research discipline such as computational chemistry, biophysics, climate, bioinformatics, etc.