Let's define scaling, as how linear the speedup of using more than one GPU or CPU is. For example, having 2 GPUs gives you 2x faster execution time.
I have noticed that in many software (e.g. Molecular Dynamics, Gromacs, AMBER, OPENMM), it scales very badly for more than 2 GPUs, but it is not a problem for CPUs (you can use hundreds of CPUs, even located on different nodes). What is stopping GPUs from achieving the same scaling as CPUs? Why CPUs even located physically far away with huge latency, can achieve better scaling than GPUs?