Take the 2-minute tour ×
Computational Science Stack Exchange is a question and answer site for scientists using computers to solve scientific problems. It's 100% free, no registration required.

Parallel computation is frequently modeled using a deterministic local rate of computation, latency overhead, and network bandwidth. In reality, these are spatially variable and non-deterministic. Studies such as Skinner and Kramer (2005) observe multi-modal distributions, but performance analysis seems to always use either deterministic or Gaussian distributions (not just inaccurate, it's inconsistent due to positive probability of negative latency).

Have higher-fidelity statistical models been developed? Do any account for cross-correlation in local compute/memory, latency, and bandwidth variability?

share|improve this question
    
Hi Jed, I only know that Little's law is often used. –  vanCompute Dec 25 '12 at 14:57
add comment

1 Answer

From the Computer Science perspective I do not think that make sense to make an general statistical model for memory access time (latency) and memory bandwidth.

It does make sense to create an statistical model for a algorithm. That is because each algorithm has a specific memory access pattern, the memory access patterns are relevant to the cache hierarchy, e.g. a algorithm with high data locality will take advantage from low level caches benefiting of really fast memory access times while other algorithms will have to go all the way to the RAM (or even worst the swap memory) and have extremely slow access times.

The general purpose values are given from the architecture point of view, you can check your architecture and search the access time from a given core to a given memory location (let's say L3 cache). Be aware that the recent architectures are Non Uniform Memory Access NUMA which will make your job a bit harder.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.