4
$\begingroup$

I have a question about the memory usage within the graphics card. What does happen when the used application needs more memory than the graphic card does contain? let`s say the reserved main memory is also not enough.

The scenario I am talking about is something like:

Consider a graphics card with, let's say, 50 MB memory usable for texture information. The reserved main memory space for texture data is also around 50 MB. The application I want to run contains five data sets of each 25 MB that are needed at the same time, so you have a total of 100 MB memory, but 125 MB is needed.

$\endgroup$
  • $\begingroup$ You may want to move this question to the "gpu" group at stackoverflow. $\endgroup$ – Pedro May 25 '12 at 13:54
  • 1
    $\begingroup$ @Pedro - GPU Computing is on-topic here, particularly if the OP is interested in CUDA or OpenCL, so I'd prefer that we try to answer the question here if we can. K.K - Can you explain a little more about what situation you are considering? The CUDA compilers, for example, will refuse to compile application kernels that require too much static memory, whereas the the device library provides malloc functionality that will simply fail if not enough heap space is available. This is very similar to traditional memory handling. $\endgroup$ – Aron Ahmadia May 25 '12 at 14:42
  • $\begingroup$ Thanks for your fast response! I am just trying to understand a little bit how a graphic card does work together with the rest of the computer. there is no special setup I am referring to. It's just a general question about what is happening to data which have to be processed through the GPU if more memory is needed than is present. I can understand the "Swapping Concept" which is used for managing the main memory. Is this also used for the graphic card memory or is it just the case that data are not processed which are to big for the graphic card memory? $\endgroup$ – K.K May 25 '12 at 15:02
  • $\begingroup$ K.K., welcome to SciComp! If you'd like to follow up or clarify your question, the best course of action is to edit your question or comment upon it. Answers are typically used to respond to the question, or occasionally for long, relevant comments. $\endgroup$ – Geoff Oxberry May 25 '12 at 21:09
4
$\begingroup$

The most common case is that you cannot allocate a memory block larger than what is physically available for your GPU card. However, depending on your hardware and driver, the maximum allocatable size might be a fraction of the total physically installed memory (e.g. old AMD opencl drivers only allowed a maximum of 500 MB), so it is better if you read into the specs of the vendor (NVIDIA, AMD, or Intel).

In the case that a GPU kernel of your implementation needs to access more of the global, or local / shared memory than what one can allocate at once, then you have to manually manage the memory by allocating and freeing memory segments (smaller than maximum memory) and transfer them between host and GPU as needed.

As far as my experience goes, memory management is crucial for achieving high performance for GPU applications. The reason for this is that for most applications, transferring data through the PCIe between host and GPU can become a big bottleneck. To illustrate the PCIe problem, consider that the maximum bandwidth of PCIe 2.0 (x16 lanes) is 8 GB/s bidirectional, whereas the bandwidth from on board GPU memory to GPU chip is +140 GB/s. Moreover, all the tiny cores of a GPU can collectively process several terabytes of data per second, thus most high performance GPU kernels also need to make efficient use of shared / local and register memory (which is orders of magnitude faster than accessing global memory).

An alternative to explicit memory management, is to use extensions to opencl where a GPU kernel can directly access, from the discrete GPU chip, memory segments that reside on host (as fas as I know, AMD allows this). In such a case, the GPU would halt execution until data is received from host, these can be a very long time from the perspective of the GPU thread.

A more interesting scenario happens when the CPU and GPU are integrated in the same chip. In such a case, CPU and GPU share the host memory (consider the fusion architecture of AMD) so there should not be necessary to copy data between host and device.

$\endgroup$
1
$\begingroup$

On NVIDIA cards at least, it is possible to allocate pinned memory on the CPU for use by the graphics card. Pointers to this block of memory are in the GPU's global memory space (so they can be accessed by your kernels) but reads/writes to them must still go through the PCI-E bus. The drawback is that you cannot use too much of this memory or your OS will become very angry with you. Is this what you meant by "reserved main memory"?

It's also possible to handle compute/memory transfer at once, so if an application doesn't need all the data at once you can have the kernel process a slice of data while the next batch is transferring, do something with the result, and then continue processing and transferring.

Edit: referring to your specific example, it may also be possible to store 50 kB in texture memory, 50 kB in global (I assume this is what you meant by main?), and then find somewhere else to squeeze the last 25kB in, like constant memory. You would only be able to read these last 25 kB, so this may not work in all cases. Another option would be to use the pinned memory, as I discussed earlier. The advantage of constant memory is that the data is actually on the device, but the read-only restriction may be too onerous for some applications.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.