In essence we can create a cluster based on a number of machines(e.g., Beouwulf style). Here the question is, do all the cluster nodes require the exact same cpu(s) specification (e.g., core, cache, etc.)? If not how the companies can scale their cluster units, they just by hundreds or thousands units at the same time?
1 Answer
$\begingroup$
$\endgroup$
Nothing requires the nodes of a cluster to be uniform. Most batch scheduling software (SLURM, LSF, SGE, PBS, etc.) allows the end user to specify requirements for the nodes that will be used to run their jobs. So if the user needs a certain amount of RAM or a certain number of cores per node, then the scheduler can select nodes to satisfy those requirements.
However, in the end, most cluster builders make them fairly uniform. Clusters are often bought all at once, up front, with a small number of node configurations. Our biggest systems at TACC tend to have one main compute node variety that makes up the bulk of the system, and 2-3 other compute node configurations to support special needs. E.g., we have 16 1TB large-memory nodes with more 32 cores on Stampede that are available in addition to the 6400 compute nodes with 32GB of RAM and 16 cores per node. We also have 128 nodes on the system that have an NVIDIA GPU.
Companies may buy all their nodes upfront or buy them in increments depending on their needs. Software can help them manage the complexity. However, most people buy in large chunks. The pricing just works out better this way. It may cost less to acquire the nodes in a big bunch from the vendor, and it certainly saves installation time to do it all at once. These systems are usually run for 3-5 years and then retired. Other systems may be purchased and installed during this time.
answered Jun 1, 2015 at 12:34