# Tag Info

37

I'll try to summarize my experiences obtained in the course of developing ViennaCL, where we have CUDA and OpenCL backends with mostly 1:1 translations of a lot of compute kernels. From your question I'll also assume that we are mostly taking about GPUs here. Performance Portability. First of all, there is no such thing as performance-portable kernels in ...

19

Ultimately, the answer to this question depends on the prices being charged for the services that you need. At some very low price, this would almost certainly be better than buying your own computer, while at some higher price you would be better off buying your own computer. The case for using a shared resource is pretty strong though and these factors ...

13

BLAS1-operations, BLAS2-operations, and sparse-operations share the same curse of low arithmetic intensity, that they perform $O(1)$ flops for each memory read (contrast this to a BLAS3-operation like gemm, which performs $O(N^3)$ flops over $O(N^2)$ reads and only becomes more and more arithmetic-intensive/compute-bound for large $N$). However, sparse ...

12

$6.60/core-month is less than a penny a core-hour. This is a good deal, and it's a better deal than you can get if you buy identical hardware yourself and pay your own power and sysadmin bill. If all you are going to do is buy one probably less powerful workstation node with sufficient RAM, then you may do better than this, but you may also complete your ... 11 Both the standard cluster and custom supercomputer (Anton) versions of molecular dynamics at D. E. Shaw Research are both deterministic and parallel invariant. That is, a test run on a single core generates the same bits as a massively parallel run. The techniques include Integer summation: Although each force term is computed in floating point, the total ... 10 A long standing favorite benchmark in high performance computing has been the HPLinpack benchmark, which measures the speed of a computer system in floating point operations per second while solving a very large, dense, linear system of equations. It is assumed that the solution takes$2/3n^{3}+2n^{2}\$ floating point operations and the tester is allowed to ...

9

Many of us in scientific computing simply have well-equipped laptops for regular software development tasks, some multicore workstations for smaller-scale testing, and access to clusters for larger runs. To give you an idea: My laptop is a Dell M3800 (4-core Intel i7, hyperthreading, 16GB of RAM). This is good enough to regularly compile my software and do ...

8

Like WolfgangBangerth, I strongly recommend that you reconsider your motivation (or your supervisor's motivation) for this goal. First, look at Center for Exascale Simulation of Combustion in Turbulence (ExaCT) web site. They have some proxy applications that may be of use to you in testing your mechanisms, and publications by authors such as John Bell, Joe ...

8

Like Brian said, the Xeon Phi cores are not at all comparable to the CUDA ones. The problem with the Phi is that it's somewhere between two horses. If you are doing highly parallel floating point calculations, NVIDIA will provide you with something like 3 times the performance at 1/4th of the price. For double precision the gap is smaller, but NVIDIA still ...

8

I realize this question was asked a while ago, but I recently needed the Freschet distance as well. I couldn't find any implementations for Python, so I wrote my own based on the paper: "Computing Discrete Frechet Distance" by "Thomas Eiter and Heikki Mannila", and thought I would share it for future reference. It's written in Cython (save as frechet.pyx) #...

7

Almost everything you can build and install in your own space. With GNU autotools, you can do something like ./configure --prefix=/path/to/your/work/space ... and then follow the usual compilation instructions. Things based on CMake and Scons have similar facilities.

7

CUDA cores aren't at all comparable to the separate processor cores in the Xeon Phi coprocessors. The Phi coprocessor cores are full fledged processors that can have their own loops, branching, etc. while the CUDA cores are all executing the same operations on various slices of your data.

7

One of the first things that you need to understand about parallel programming is the difference between shared memory multiprocessor computer systems and distributed memory clusters. A shared memory multiprocessor system is a computer in which several processor cores (which might be on one, two, or more integrated circuits) share the same memory. From ...

6

It will not scale! Use the parallel HDF5 functionality to read it in parallel with a relatively small number of readers, or do what your colleague suggests. There will be tradeoffs with both methods, and you will have to do some tuning. It depends pretty strongly on how big the file is, how many tasks you have reading it, what the underlying filesystem is (...

6

Will you be using it all the time, for a long time? In general, the economics for this are simple - if you need a valuable resource for short intermittent bursts, it will generally be cheaper to rent than own; and if you expect to use most of it for a prolonged time then it will be cheaper to own it. A simple rule of thumb actually is about the terms used -...

6

You could install BOINC on those machines. When the computers become idle, the BOINC screensaver/client requests tasks from a server and computes them. See more information about it here. This is the software used by a number of projects such as SETI@Home. You can create your own project with BOINC and then put your desktop machines to work.

6

The honest answer is that we don't know. The answer depends heavily on what is actually being run and what code the user has written. As Brian Borchers points out, there's a big difference between two benchmarks where we have all the code and supposedly know what that code is doing, but there's much disagreement about how representative this code is of what ...

6

A first step, if you "have never been up in computing", is to read the literature and see what others are doing and have done. The second step is that you will likely learn that what you want to do is not possible today -- at least unless you have access to supercomputers. I suspect that 3 billion particles is possible today, but only if you have access to ...

6

I think the question is just too subjective to answer. In the end, there are excellent C++ libraries for nearly everything that has to do with the solution of PDEs, whereas they are largely missing in the Julia environment. Examples that come to mind are PETSc/Trilinos for linear algebra, deal.II/libmesh/FEniCS for discretizations, etc. You will have to ...

5

It depends on how quad precision is implemented. If you want to implement it as "traditional" floating point numbers with sign, mantissa, and exponent (the latter two just having more than the normal 53 and 10 bits of double precision), then doing this on a processor that doesn't natively support it, is going to be pretty expensive because it will involve a ...

5

For dense distributed memory linear algebra, you can't do better than Elemental these days. Although, it does not currently handle sparse matrices, I believe. Dense-sparse operations will probably require you to do some custom coding.

5

You can use HTCondor that is designed exactly to "steal" cpu cycles from remote machines. It may be a little difficult to setup but I think this may be the best approach.

5

Let me supply a belated answer from the POV of a cluster user and a cluster administrator. A well-designed cluster will, in general, be as homogeneous as possible, with login-nodes being generally a bit beefier due to being the single point of contact of users to the cluster. A well maintained cluster will also have a good documentation of the system, for ...

5

I would say that the implementation + verification + unit testing would take you more than just 3 weeks. Although, if you are planning to invest that time, you might add that capabilities to scipy.sparse or scikits-sparse. Regarding symmetric sparse matrices, you can check Pysparse. It has Sparse Skyline format (SSS) that is used for symmetric matrices. You ...

5

You need funding, you need a better algorithm, a better implementation, or you need to change your problem. To get funding, you need to work with your adviser. This could be about using grant money or getting a grant for an XSEDE allocation. If your adviser has neither of those, it may not be a good choice to work with them even if you like their research: ...

5

This really depends on the operations you are including in your question. If you took the sparse equivalent of any level 1 BLAS or level 2 BLAS algorithm, then yes they are memory bound (not compute bound), but that is also true of level 1 BLAS and level 2 BLAS for dense matrices. To answer this we need to understand what makes an algorithm compute bound. ...

5

If you consider a 4 by 4 matrix of integers, it will be stored in memory as a unique array of integers. Since each integer is 4 bytes ( 32 bits) (sometimes not, but it's not important here) then you are using precisely 4*16 = 64 bytes. For doubles, just remember each double is usually 8 bytes (64 bits). A matrix can be stored in row major or column major ...

4

I don't recommend it, but not because of performance issues. It will be a little less performant than a traditional matrix, which are usually allocated as a big chunk of contiguous data that is indexed using a single pointer dereference and integer arithmetic . The reason for the performance hit is mostly caching differences, but once your matrix size gets ...

4

Sorry for digging up an old thread but it seems that even in 2015, Fortran is being used a lot. I just came across this (alternate link) list which basically is a list of 13 codes approved by DOE's OCLF facility to run on the 300-petaFLOPS Summit machine which will be made available to researchers in 2018. I tried to find the main language used for the code ...

4

It's possible that all the memory being used by your code is on one socket and that up to 6 cores all the tasks are running on that socket. When you get to 7+ sockets, then there are transfers between sockets to get at the memory. You may need to investigate memory affinity options for your threads. The default policy in Linux is first-touch (I think), so if ...

Only top voted, non community-wiki answers of a minimum length are eligible