ATLAS is a free BLAS/LAPACK replacement that tunes itself to the machine when compiled. MKL is the commercial library shipped by Intel. Are these two libraries comparable when it comes to performance, or does MKL have the upper hand for some tasks? If so, which ones?
MKL (from Intel) is optimized for Intel processors, and probably has the "upper hand" there in many cases. But it is also "famous" for choosing the "worst" code-paths for AMD processors, as described here.
BLAS is not monlithic. BLAS1 and BLAS2 are memory bandwidth limited, and there is not much you can do to speed them up beyond the obvious (loop unrolling, cache blocking for level 2). BLAS3 is more interesting and the prototypical benchmark here is matrix-matrix multiplication. To my knowledge GOTOBlas has always been the clear winner here, see for example this comparison or this one and this justification.
It is now years later, and we have the BLIS Project. It is the best free alternative to MKL.
Prior answers to this question have covered most of the salient points, but I want to add one comment with respect to this:
does MKL have the upper hand for some tasks?
The MKL team is in a unique position to know about future Intel instruction sets and their implementations in specific processors. Furthermore, they have access to proprietary processor simulators and pre-production hardware that no one outside of Intel can use. Thus, MKL has the upper hand with respect to degree of knowledge about future products and when they obtain this knowledge. Thus, it should not be too surprising if they produce better implementations of the BLAS than anyone else, at least early in the lifetime of a product with new features.
On the other hand, Intel has been quite open about the AVX-512 instruction set and has provided the Intel® Software Development Emulator (SDE) that allows developers to emulate AVX-512 instructions on processors that do not support these natively. Because of this, it will not be too surprising if high-quality open-source implementations of the BLAS are available for Intel processors that support AVX-512 early in the lifetime of these products.
Of course, how much difference it makes to have detailed information about a particular processor versus the fundamentals of dense linear algebra algorithms is not fully resolved. The following quote addresses this issue better than I can:
In theory, there is no difference between theory and practice. But, in practice, there is.
Full Disclosure: I work for Intel.
Profile, don't speculate! (also works as “Benchmark, don't speculate!”)
There's nothing generic one can say, it depends heavily on the tasks you want to perform (BLAS 1/2/3, for example) and the hardware you're on (obviously, the Intel MKL doesn't run on ARM processors, for example; but even among Intel processors, you can expect performance differences).
Another thing I think is worth mentioning is that AMD also offers mathematical libraries tuned to their processors, the AMD Core Math Library. It's not as feature-rich as Intel's MKL, but it does include BLAS.
I think that the main difference between vendor BLAS libraries and open source BLAS libraries is the time that it takes for open source to support the latest hardware features.
As BLAS is widely used, it is of the interest of the vendor to support the latest hardware features. For instance, consider Intel's AVX vector extensions that were introduced with the "sandy-bridge" processor on Jan 2011. MKL had AVX support even before the CPU was available but it was only recently that ATLAS (late 2011) started to rollout support for AVX.
Also, if your application really needs performance and before you start benchmarking different BLAS libraries or start hacking around optimizing anything: profile your application. It is common that human intuition is not a very effective profiler predictor, at least I know mine is not! So instead of spending time randomly optimizing, profile your application and systematically approach each bottleneck.