I frequently work with Keras and tensorflow-gpu on my Lenovo P50 workstation which has 64GB ram and a 2GB Nvidia QUadro M1000M GPU.

Today while training my model on GPU, it ran into memory error and it exit at this line:

da_ta = np.zeros((len(data),sequence_length, cls.get_vector_size(model)), dtype=K.floatx())

So my question is how to compare 64GB ram vs 2 GB gpu?
Is one better than the other?
Kindly migrate my question to the appropriate forum if this is not the right place.

Thank you.

  • 3
    $\begingroup$ Some algorithms (their implementations) work on CPUs and use RAM, some work on GPUs and then they use GPU RAM; some are even hybrid and use both CPU/GPU and thus both RAMs. If your code/library does not use GPU, you won't have any advantage of having it and vice versa. $\endgroup$ – Anton Menshov May 15 '18 at 21:39
  • 1
    $\begingroup$ For a high-performance GPU-accelerated deep-learning system (as well as many other general-purpose GPU-accelerated systems), a useful rule of thumb for balancing the two memory types is to aim for a system memory that is roughly four times the total GPU memory. Note that the NVIDIA-specified DGX systems follow this rule of thumb. $\endgroup$ – njuffa May 17 '18 at 15:12

In my opinion there is not a memory type better than other. Simply they are different things.

Normal ram are used only by the cpu, and the gpu ram is used onnly by the gpu. This is quite clear when you code direct in CUDA, i.e. if you want that the gpu use some data in normal ram you must move them to the gpu memory (host -> device). And if you want use some gpu results inside the cpu you must move them (device -> host). Note that in some languages there are "tricks" to avoid explic passage, for example Unified Memory Programming:

The underlying system manages data access and locality within a CUDA program without need for explicit memory copy calls.

Behind the scene there are movement (note is better this or manual movement this is another question....).

Keras and tensorflow are made to hide this level of detail, so you do not see it direct.

For the above motivation is not correct to compare the two memory.

If you want to have an idea about a size comparison, here is heavy dependent by the problem. I try to explain with CUDA. A gpu works with a parallel execution of a N times of the same function (read kernel) with different data. See SIMD every new data is assigned to a different thread. Now normally you have got a number of cases to elaborate >> of N so the gpu groups and organize the threads, see this, in pool of N and these pools are execute sequentially.

With some simplification you can consider how many memory every thread use (here heavy dependency by the problem) and multiply for N. In this way you have got a rough estimate of the memory used by the gpu and you can understand if 2 GB of gpu are enough for 64 gp of cpu ram.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.