One of the first things that you need to understand about parallel programming is the difference between shared memory multiprocessor computer systems and distributed memory clusters.
A shared memory multiprocessor system is a computer in which several processor cores (which might be on one, two, or more integrated circuits) share the same memory. From the programmer's point of view, multiple processes created by the programmer can run on the different cores and share access to the memory. Parallel programs for scientific computing on shared memory systems are typically written using specialized language extensions such as OpenMP (which has C, C++, and Fortran versions.) The OpenMP language extensions make it easy to describe operations on arrays that are to be performed in parallel- the compiler takes care of distributing the work to the multiple processors.
A distributed memory cluster can be thought of as a large collection of computers, each of which has its own memory. In order to interact, processes running on these computers must explicitly send messages to each other. Parallel programs for scientific computing on distributed memory clusters are most commonly written using the message passing interface (MPI) library. Programming with MPI is more difficult than programming with OpenNMP because of the difficulty of deciding how to distribute the work and how processes will communicate by message passing.
Note that a program written using MPI can be run on a shared memory system using a version of the MPI library that simply passes messages between processes through the shared memory. In that sense, an MPI program is much more flexible- it can run on either type of system. In comparison, you can't effectively run an OpenMP program on a distributed memory cluster.
Shared memory systems are typically limited in the number of processor cores and the amount of storage that can be used. In practice, you'll seldom find more than 64 processor cores or about 128 gigabytes of RAM in a shared memory system, while distributed memory clusters might have tens of thousands of processor cores and terabytes of memory.
For truly large scale parallel computing you will need to learn MPI. If you are willing and able to work within the limitations of a smaller shared memory system, than learning OpenMP will be an easier way to get started in parallel computing. For example, if you're using a desktop computer with 4 or 8 cores and you want to take advantage of those cores, then OpenMP is probably the best way to get started.
Other answers have already mentioned some books to get started with MPI. For OpenMP, I'd recommend that you start with the list of resources at