7
$\begingroup$

What is the best platform for high performance computing (HPC)?

Windows is long gone, I think. So only Unix and Linux stand the chance. What platform will strongly back my interest on computational mechanics?

The application field is large dynamic FEM problem, the approach will be Multi-physics, Multi-scale modeling. Clustering with consumer hardware will be the focus (To reduce the cost).

Improve this question
$\endgroup$
10
  • $\begingroup$ I want to build my cluster from scratch using consumer hardware parts. $\endgroup$
    – ShadowWarrior
    May 12, 2012 at 15:01
  • 3
    $\begingroup$ "Best" is a ref flag for a poll ad "engineering simulations" covers a huge amount of territory. Finite elements? CFD? Analog/digital hybrid simulations for VLSI? $\endgroup$
    – dmckee --- ex-moderator kitten
    May 12, 2012 at 16:21
  • $\begingroup$ of course FEM!!! $\endgroup$
    – ShadowWarrior
    May 12, 2012 at 16:25
  • 2
    $\begingroup$ Finite element analysis is not a small field in itself. Static? Static plus perturbations? Fully dynamic? How big are these simulations? Could you conceivably fit the whole thing into main memory or are you going to have to page to storage (which affects the specification of you storage subsystem). $\endgroup$
    – dmckee --- ex-moderator kitten
    May 12, 2012 at 16:28
  • 3
    $\begingroup$ I agree with @dmckee here that you are making the question too broad. I think it fit when the focus was on the dominant operating system (GNU/Linux, which might not be obvious if you're not experienced in HPC arena), but I don't think this question should be extended beyond that. $\endgroup$
    – Aron Ahmadia
    May 13, 2012 at 8:03

3 Answers 3

Reset to default
6
$\begingroup$

Linux takes all ten of the top 10 spots in petaflops/supercomputing, but the operating system is perhaps not the only aspect you should focus on.

Most developers will never get "time" on the highest end machines until they have something promising to show for their efforts. So you probably want to keep an eye on the evolving standards for cloud computing APIs. At this point vendors like Amazon have a lead in establishing defacto standards, but the open standards community is hard at work in promoting interoperable standards.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Your leading statement is, without proper qualification, wrong. Most of these machines are running a GNU Operating System, but the lightweight IBM kernel is proprietary and not Linux-based. Cray's kernel is Linux-based, I am not sure about the others. $\endgroup$
    – Aron Ahmadia
    May 12, 2012 at 16:04
  • $\begingroup$ Your reference to IBM might mean number 10 on the list, Roadrunner, but it's listed as a Linux operating system. Number 5 on the list TSUBAME 2.0 is described as Linux/Windows (though Linux is shown as "operating system"). There's a sublist generator that allows us to filter by "OS Family" with Linux getting 457 entries (out of the top 500). Number 13 JUGENE - Blue Gene/P Solution, an IBM system, is the highest non-Linux system (OS CNK/SLES 9). $\endgroup$
    – hardmath ♦
    May 12, 2012 at 16:49
  • 1
    $\begingroup$ The kernels running on the compute nodes for very large-scale supercomputers tend to look more like real-time/embedded Linux kernel variants, not the vanilla kernel most Linux users are familiar with. The rest of the operating system is usually based off one of the major Linux distributions, so I think your overall point stands (learn Linux, kids!). $\endgroup$
    – Aron Ahmadia
    May 12, 2012 at 19:28
2
$\begingroup$

Here is a slightly different view: Windows is not gone. On the contrary, it's making its way in the HPC again these days. Microsoft has done excellent work with their cluster tools in Windows 2008 HPC Server. Their MPI implementation (based on MPICH2) is as capable as its free sibling and the InfiniBand stack on Windows is quite performant. There is also a great choice of commercial software that is only available on Windows and is now gaining distributed computing capabilities.

That said, I would strongly disagree with anyone that can suggest "the best" HPC platform. There is no "one size fits all" platform and it depends on variety of factors. Most machines in Top 500 come with Linux or other Unix-like OS/kernel but that doesn't mean that you have to stick with Un*x for life. Big machines are all alike - learn to use MPI and one batch system and you can use them all (there is a caveat though - efficient usage of the hardware is highly platform dependent but there are some general rules).

For the real HPC user the platform is just a tool that executes his or her code in production mode. It's the development environment that counts. You can develop with MPI and/or OpenMP on any major OS, be it Windows or Mac, or Linux, or Solaris, or even some obscure OS and then run your code on the Linux supercomputer in your computing centre (or in the cloud) if you adhere to some language standards and don't use platform specific APIs. There are several people in our department that use extensively Visual Studio to develop code that runs on our enormous Linux cluser just because they don't find the development tools on Linux advanced or convenient enough. It's up to you to find the most convenient environment that suits your style and your work.

If you are building your own system then probably the price will be a serious factor and the choice of OS is almost predetermined to be Linux. It might also be your platform if your work depends on software available for Linux only. Besides, Intel provide their development tools for that OS for free if you are going to use them for personal stuff only. Linux also comes with its default GNU compiler collection and large selection of software and libraries for almost anything.

If you would like to run Windows software and you could afford the extra costs, then Windows 2008 HPC Server is a nice option. It integrates the HPC envrionment with ActiveDirectory and makes it really easy for the system to be centrally administered. Intel provides great tools for HPC development on Windows but they come at cost, even for private usage.

Since Intel development tools are available on both platforms and since they support the same variety of programming languages, it is quite easy to migrate code between the two OSes. There are also other vendors like Portland Group that provide multi-platform high performant compilers and scientific libraries.

As for the connectivity - in HPC nowadays it is either InfiniBand or multigigabit Ethernet (e.g. 10 GigE). Both provide low latency and high bandwidths which are a necessity for fast message passing. InfiniBand is faster but costs more.

Improve this answer
$\endgroup$
5
  • 4
    $\begingroup$ Microsoft has drastically decreased the size of their HPC team (most have been moved to cloud or other development). HPC Server is not particularly lightweight, either, as it requires an entire Windows image (1GB+) per compute node. $\endgroup$
    – Aron Ahmadia
    May 13, 2012 at 8:06
  • $\begingroup$ That's bad for the community but nevertheless it is still growing, at least here in Germany. As for the OS - Linux is also not particularly lightweight. Even with most software in shared /opt, a RedHat/CentOS/Scientific Linux based installation takes serveral GBs on each compute node, especially if you'd also like to use them for interactive jobs. $\endgroup$
    – Hristo Iliev
    May 13, 2012 at 8:28
  • 3
    $\begingroup$ Modern supercomputer compute nodes are diskless, as it's very difficult to get the needed density if hard drives are directly attached to the processors. Vendors don't run full Linux operating systems on them. For example, the IBM CNK implemented on BlueGene/L occupies only 1MiB of memory. $\endgroup$
    – Aron Ahmadia
    May 13, 2012 at 10:33
  • 1
    $\begingroup$ Yeah, and besides running the non-Linux dervied CNK, BlueGene systems are also powered by custom ASIC CPUs. How does that fall in the "consumer hardware" the OP will be focusing on? $\endgroup$
    – Hristo Iliev
    May 13, 2012 at 12:50
  • 1
    $\begingroup$ I think it's a pointless statement to say that Windows is "making its way in the HPC again these days". Maybe indeed its share of cluster installations (in the top 500 as well as around the globe on small research clusters) has grown from .5% to .75%, or whatever the numbers are. It's still a fact that nobody (for any practically useful definition of "nobody") uses Windows on clusters. (I assume that HPC=cluster computing.) $\endgroup$
    – Wolfgang Bangerth
    May 17, 2012 at 9:18
1
$\begingroup$

That depends on the computer architecture. When you say HPC platform, I assume a multi-processor computing cluster. Most of these would come with vendor specific flavor of UNIX, e.g. IBM AIX or HPUX. If you are building your cluster from scratch, most common choice is some flavor of Linux.

Linux is fully customizable, which is important for resource management and performance optimization. This also makes it most convenient for building/setting up large, multi-processor clusters. In addition, most scientific and engineering software and compilers are readily available for Linux through package managers, or their source is available and they are easily configured and compiled.

Which flavor of Linux exactly is usually a matter of preference.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.