I have a code similar to this:

import numpy as np
import multiprocessing as mp

a = np.zeros((4, 4))  # 4x4 array containing zeros

def f(x, y):
    # uses scipy functions
    # takes long to compute
    #result = <someValue after calculation>
    global a
    a[x][y] = x+y  # simple example function

# since f takes long to compute, I want to run it in parallel
jobs = []
for x in range(4):
    for y in range(4):
        p = mp.Process(target=f, args=(x, y))

# wait for a to be filled for the next step
for j in jobs:

print a  # prints an array of zeros! 

Why are the values not assigned to a and how can I fix this? Thanks!

  • $\begingroup$ This is off-topic because it is a basic low-level programming question which is not specific to computational science, and should probably be asked (and has likely already been answered) on Stack Overflow $\endgroup$ – tel Nov 6 '13 at 23:46

Multiprocessing creates separate Python processes (i.e. UNIX or Windows process) for each mp.Process that you asked it for. These do not share memory. If you want them to operate on the same data, you must use one of the managed data types in the mp.Manager class to explicitly communicate between separate tasks.

FWIW, this is pretty slow since whatever object you put into the mp.Manager containers is pickled before it's communicated.


Not the answer you're looking for? Browse other questions tagged or ask your own question.