I recently moved towards computational biophysics from an experimental science background. As of now, I am learning the fundamentals and doing some basic monte carlo simulations of LJ fluid on my desktop PC. However, when I tried to run my code on the cluster, it was equally or more slower than my desktop. Later, my lab mates told me to write my code suitable for parallel computation. I have no idea how to do that. Is there any resource, book aimed for people who are not very proficient in computer science, that can teach them about basics of such programming and necessary tools. I wrote my code on c++, I am willing to move to fortran as well.
As was mentioned in the comments, A good, simple book/resource on Parallel Programming in C++ for scientific computing already gives an answer to the more general question of learning about parallel programming.
Having said that, a simple Lennard-Jones Monte Carlo program may not be a bad project to attempt, if you decide to go ahead and learn about parallel programming. The OpenMP approach is relatively easy to understand and implement. Many compilers have a -fopenmp flag, which will switch on the processing of any OpenMP
pragmas in the code. A typical
pragma is one that is applied to a
for loop (or in Fortran, a
do loop), to divide the iterations amongst the parallel threads. As an example, if you wanted to sum up $N$ numbers, and you have $P$ threads available, each thread could sum up (approximately) $N/P$ of the numbers, in parallel, and then the results can be combined at the end. You would need to learn some basic OpenMP concepts first: the difference between
shared variables, and crucially the idea of a
reduction variable, which would hold the result of the summation. But it doesn't take too long to master OpenMP at this level.
The point is that, in the simplest Monte Carlo program, the time consuming part is exactly such a loop. When you consider moving an atom, you need to compute the sum of Lennard-Jones pair potentials between it and all the other atoms, before and after the move. This assumes that you have a relatively small system, and are not using sophisticated techniques such as domain decomposition / neighbour lists to speed things up. Basically, if you can parallelize a simple summation loop, it is not a huge stretch to parallelize the single-particle potential energy calculation.
On your cluster, it is quite possible that each node consists of several cores with access to the same memory: if so, then you could use OpenMP to run on a single node, with each thread corresponding to one of those cores. So, in principle, best-case scenario, you might get a speedup of however many cores there are per node.
For larger-scale Monte Carlo, a more sophisticated approach may be needed. Parallelizing large-scale Monte Carlo codes can be done (e.g. by using domain decomposition and writing a program that uses MPI), but is more fiddly and needs more care (because of the need to ensure detailed balance) than this simple OpenMP example.
If you decide to proceed in this way, you should definitely follow up some of the resources mentioned above first. But after you've learned the basics, it's actually just a question of inserting a very few extra lines in a simple program to sum up a set of numbers, as a test, and then doing a similar thing to your code, and compiling with the appropriate option.