A Compiler for Parallel Exeuction of Numerical Python Programs on ...

<strong>Python</strong>-C API functions from multithreaded functions.I introduce an extension to mark certain <strong>for</strong> loops as parallel to provide a sharedmemorydata-parallel model similar to (but less general than) the OpenMP parallel <strong>for</strong> loopdirectives. To avoid the complexity <strong>of</strong> specifying parallel semantics <strong>for</strong> arbitrary iterators,only a parallel version <strong>of</strong> xrange iterator is defined. A special iterator, prange, is introducedthat is used to specify that a loop is parallel. The iterator prange is only a hint to thecompiler that the programmer intends to execute the loop in parallel but the compiler isnot obligated to compile the loop to a parallel C++ loop. To the <strong>Python</strong> interpreter, prangeis identical to xrange. However, un<strong>Python</strong> treats the annotations as a special parallel-loopannotation. A <strong>for</strong> loop over the parallel iterator prange has the following properties:1. Let the program point immediately be<strong>for</strong>e the loop be P. Then the program executionis serial be<strong>for</strong>e P.2. Let the program point immediately after the loop be Q. Then the program may spawnmultiple threads between P and Q. The point Q is an implicit join-point. Programexecution does not continue until all threads spawned reach Q. After Q, serial execution<strong>of</strong> the program continues.3. Each iteration <strong>of</strong> the loop is independent and may be executed in any order.4. <strong>Parallel</strong> loops may be nested but, in this model, a join point only exists at the ending<strong>of</strong> the outermost parallel loop.5. <strong>Parallel</strong> loops are a compiler directive and the compiler is not obligated to parallelizethe loop.6. If there is a data-race between any two iterations, then the output <strong>of</strong> the program isundefined.7. The body <strong>of</strong> the loop may only access variables <strong>of</strong> basic numeric types (float32, float64,int32 and int64) or NumPy arrays <strong>of</strong> basic numeric types. If any other object type isreferenced in the loop, then the loop is not parallelized by the compiler to avoid thepossibility <strong>of</strong> calling <strong>Python</strong>-C API functions from two different C threads.8. All variables defined outside the loop are treated as shared variables among threads.However, a programmer can define some variables as private by passing the keywordargument private with the value equal to a tuple containing a list <strong>of</strong> variable namesas strings.9. All variables local to the loop are treated as private.26