How much space store a matrix of numbers?

Question

When we work in numerical analysis, when the matrix is too big the computer runs out of memory (i guess it's ram memory). But how much space does a matrix of numbers (integer, single, double) use? I would like to know this to be able to determine how large the matrices can I compute in my laptop.

Answer 1

If you consider a 4 by 4 matrix of integers, it will be stored in memory as a unique array of integers. Since each integer is 4 bytes ( 32 bits) (sometimes not, but it's not important here) then you are using precisely 4*16 = 64 bytes. For doubles, just remember each double is usually 8 bytes (64 bits).

A matrix can be stored in row major or column major order, and this is crucial for instance when you try to access the elements with loops, because the more you have to jump across memory positions, the slower will be your code. For instance, if you try to implement naive Gaussian elimination for small-sized matrices, you'll se that the row-oriented version may be much slower that the column-oriented one: that's depend on how things are stored, even if the algorithm is the same!

C++ is probably the best language to understand those details because it allows you to deal with memory. For instance, let's see the adresses in memory of a 2d array in C++:

#include <iostream>

int main() {

    int m[4][4] { }; //4 by4 matrix with all zeros (default value for ints) allocated on the stack
    for (unsigned int i = 0; i < 4; ++i) {
        for (unsigned int j = 0; j < 4; ++j) {
            std::cout << &m[i][j] << "\t";
        }
        std::cout << "\n";
    }
    return 0;
}

which gives the output

0x7ffeec242400  0x7ffeec242404  0x7ffeec242408  0x7ffeec24240c  
0x7ffeec242410  0x7ffeec242414  0x7ffeec242418  0x7ffeec24241c  
0x7ffeec242420  0x7ffeec242424  0x7ffeec242428  0x7ffeec24242c  
0x7ffeec242430  0x7ffeec242434  0x7ffeec242438  0x7ffeec24243c  

as you can see, we have a row-major ordering.

Notice that for real applications or large problems, we are forced to use sparse matrices, which are much more complicated. You have to take into account how to store them, how to access the elements, and the implementation is not trivial at all and it's an active area of research