In vacuum without considering any existing software, there's no reason to prefer column major over row major from the code point of view. However, most mathematical literature is written in a way that groups vectors into a matrix by storing them as columns instead of rows. For example when you write the full eigenvalue equation $AX=X\Lambda$, the $X$ matrix contains all the eigenvectors written out in columns. You never really see it written the other way (though I hear that statistics folks like row vectors). Therefore, it was natural that the earliest software assumed column major format so that if you have a matrix which is a set of vectors, the storage of any single vector is contiguous. Thus, I imagine that tradition has just been carried forward to the present day, and if you want to interact with the ye olde Fortran, you want to use column major. So pretty much all highly efficient numerical linear algebra is done in column major.
The reason C is row major is somewhat of a consequence of its array syntax; you declare a 3 row by 2 column array as
double a, and later indices vary faster than earlier indices, which for 2D arrays makes it row major. Combine this with the natural Western reading order from left to right, it makes row major seem more natural.