# Memory usage in fortran when using an array of derived type with pointer

In this sample program I'm doing the same thing (at least I think so) in two different ways. I'm running this on my Linux pc and monitoring the memory usage with top. Using gfortran I find that in the first way (between "1" and "2") the memory used is 8.2GB, while in the second way (between "2" and "3") the memory usage is 3.0GB. With the Intel compiler the difference is even larger: 10GB versus 3GB. This seems an excessive penalty for using pointers. Why does this happen?

program test
implicit none

type nodesType
integer:: nnodes
integer,dimension(:),pointer:: nodes
end type nodesType

type nodesType2
integer:: nnodes
integer,dimension(4):: nodes
end type nodesType2

type(nodesType),dimension(:),allocatable:: FaceList
type(nodesType2),dimension(:),allocatable:: FaceList2

integer:: n,i

n = 100000000

print *, '1'
read(*,*)
allocate(FaceList(n))
do i=1,n
FaceList(i)%nnodes = 4
allocate(FaceList(i)%nodes(4))
FaceList(i)%nodes(1:4) = (/1,2,3,4/)
end do
print *, '2'
read(*,*)

do i=1,n
deallocate(FaceList(i)%nodes)
end do
deallocate(FaceList)

allocate(FaceList2(n))
do i=1,n
FaceList2(i)%nnodes = 4
FaceList2(i)%nodes(1:4) = (/1,2,3,4/)
end do
print *, '3'
read(*,*)

end program test


The background is local grid refinement. I chose the linked list to easily add and remove faces. The number of nodes is 4 by default but can become higher depending on the local refinements.

• The "first way" should be avoided as much as possible as it is prone to leaks (arrays must explicitly deallocated, as you did) besides the difference in performance you see. The only reason to use it would be for strict adherence to Fortran 95. Allocatable's in derived types were added in TR 15581 but all Fortran compilers (even ones that dont have any 2003 features) have supported them, i.e. F95 + TR15581 + TR15580 since forever. Nov 1 '12 at 15:38
• The reason to do it is that some faces might have more than 4 nodes. Nov 1 '12 at 20:02
• Then it certainly makes sense. I assumed that 4 was a constant. Nov 1 '12 at 20:31

## 3 Answers

I don't actually know how fortran compilers work, but based on the language features, I can take a guess.

Dynamic arrays in fortran come with meta-data to work with intrinsic functions like shape, size, lbound, ubound, and allocated or associated (allocatable vs pointers). For large arrays, the size of the meta-data is negligible, but for tiny arrays, like in your case, it can add up. In your case, the size 4 dynamic arrays likely have more meta-data than real data, which is leading to your memory usage balloon.

I'd strongly recommend against dynamic memory at the bottom of your structures. If you're writing a code that deals with physical systems in some number of dimensions, you can set it as a macro and recompile. If you dealing with graphs, you can statically allocate an upper bound on the number of edges or the likes. If you are dealing with a system that actually needs fine-grain dynamic memory control, then it is probably best to switch to C.

• Yeah but doesnt the metadata argument hold true for both cases? Nov 1 '12 at 15:43
• @stali no, notice that the second case requires one pointer, as opposed to the n pointers needed by the first method. Nov 1 '12 at 16:19
• I've added some background info. Your suggestion to statically allocate an upper bound is already a good improvement. The upper bound is 8, but the majority will have 4, only a small percentage will have 5,6,7 or 8. So memory is still wasted... Nov 1 '12 at 20:00
• @chris: Can you make two lists, one with 4, and one with 8 nodes? Nov 1 '12 at 20:18
• Probably. It seems to be a good compromise. Nov 2 '12 at 12:10

As maxhutch has pointed out, the problem is probably the sheer number of separate memory allocations. On top of the metadata, though, there's probably whatever additional data and alignment the memory manager needs, i.e. it is probably rounding each allocation up to some multiple of 64 bytes or more.

To avoid allocating a small chunk for each node, you could try allocating each node a portion of a pre-allocated array:

integer :: finger
indeger, dimension(8*n) :: theNodes

finger = 1
do i=1,n
FaceList(i)%nodes => theNodes(finger:finger+FaceList(i)%nnodes-1)
finger = finger + FaceList(i)%nnodes
end do


My Fortran is a bit rusty, but the above should work, if not in principle.

You'd still have the overheads of whatever your Fortran compiler thinks it needs to store for a POINTER type, but you won't have the memory manager overheads.

• this helps but only a little. The problem is that it's not a single pointer but a dynamic array of pointers: FaceList(i)%nodes(1:FaceList(i)%nnodes) => theNodes(finger:finger+FaceList(i)%nnodes-1). It also implies a sharp estimate for the size of the pre-allocated array. Nov 2 '12 at 12:31
• @chris: I'm not sure I completely understand... What do you mean by a "dynamic array of pointers"? The field nodesType%nodes is an pointer to a dynamic array. Nov 2 '12 at 13:04

Oh. This is the same problem I suffered. This question is very old, but I suggest a little different code style. My problem was allocatable statement array in derived data type, as follow code.

type :: node
real*4,dimension(:),allocatable :: var4
real*8,dimension(:),allocatable :: var8
end type node

type(node),dimension(:),allocatable :: nds

imax = 5000
allocate(nds(imax))


From some test, I confirmed that if I used the allocatable statement or pointer statement in the derived type as follow code about four cases, memory leakage happen very large. In my case, I red the file of 520MB size. But the memory usage was 4GB at release mode on intel fortran complier. That is 8 times bigger!

!(case 1) real*4,dimension(:),allocatable :: var4
!(case 2) real*4,dimension(:),pointer :: var4
!(case 3) real*4,allocatable :: var4(:,:)

!(case 4)
type :: node(k)
integer,len :: k = 4
real*4 :: var4(k)
end type node


Memory leakage doesn't happen when I use allocatable or pointer statement without derived type. In my opinion, if I declare the allocatable or pointer type variable in derived type and large allocate the derived type variable not allocatable variable in the derived type, memeory leakage occurs. To solve this problem, I changed my code that does not include derived type as follow code.

real*4,dimension(:,:),allocatable :: var4
! array index = (Num. of Nodes, Num. of Variables)


or how about this style?

integer,dimension(:),allocatable :: NumNodes ! (:)=Num. of Cell or Face or etc.
integer,dimension(:),allocatable :: Node     ! (:)=(Sum(NumNodes))


NumNodes variable means the number of nodes on each face and Node variable is the node numbers matching NumNodes variable. Maybe memory leakage is not occurred in this code style, I think.