GNU Octave: using 'parallel' with an array (apply function to each array member)

Question

I am new to GNU Octave 6.2.0 on Linux, and I am trying to use the package 'parallel' to make use of 24 threads on my machine.

I have an array (array1) that has about 500,000 rows or values (1 column) and for each member of the array I want to apply a log10 function, like this:

    test = 10*log10(array1.^2); % this works

I tried using the parallel package like so:

test = pararrayfun(24, @(array1) 10*log10(array1.^2), array1, "Vectorized", 1 "ChunksPerProc", 1); % this fails

But I am getting the error (repeating lots of times):

Could not save variable
Could not load variable

I also tried to apply the "smooth" function (from data-smoothing) to the array using a span of 100 using parallel but that too fails:

test2 = smooth(array1,100);  % this works fine

test2 = pararrayfun(24, @(array1) smooth(array1,100), array1); % this fails

Can anyone help me with the proper syntax for using 'parallel' with the above two examples ?

Thanks so much !!

Answer 1

I can not reproduce the error. Here is the snippet I am using:

pkg load parallel

vector_x = rand(500000,1);
tic
vector_y = pararrayfun(nproc, @(x) 10.*log10(x.^2), vector_x, "Vectorized", true, "ChunksPerProc", 1);
toc
tic
vector_y = pararrayfun(1, @(x) 10.*log10(x.^2), vector_x, "Vectorized", true, "ChunksPerProc", 1);
toc
tic
for i=1:length(vector_x)
    vector_y(i) = 10.*log10(vector_x(i).^2);
end
toc
tic
vector_y = 10.*log10(vector_x.^2);
toc

I was fooling around and testing some other options to do the same calculation and comparing the time spent for each one. I have a philosophical issue with arrayfun / pararrayfun (What is the use of arrayfun if a for loop is faster?), I believe that they are completely redundant, and they fool MATLAB/Octave users into false sense of efficiency. In my experience, MATLAB's JIT compiler is smart enough to vectorize simple for loops like we have here. However, apparently the Octave interpreter is not smart enough.

I am on a machine with 64 threads and here are the timing results

% pararrayfun using all available threads with ChunksPerProc = 1. Increasing this value 
% (in theory) decreases time on busy machines, but in my case it didn't help.
parcellfun: 64/64 jobs done
Elapsed time is 0.0932262 seconds.

% pararrayfun using 1 thread with ChunksPerProc = 1. Compare against using all the cores. 
% Uses half the amount of time and 1/64th of the cores.
parcellfun: 1/1 jobs done
Elapsed time is 0.0447989 seconds.

% Simple for loop.
Elapsed time is 3.23663 seconds.

% Simple vectorization to let Octave figure out the threading itself. Fastest.
Elapsed time is 0.0195971 seconds.

While I cannot answer your question (as I cannot reproduce the error), I would suggest you not to use pararrayfun for this case. Also, I am yet to encounter a single case where arrayfun/pararrayfun beats simple vectorization (or simple vectorization combined with parfor).