Parallel Processing: A KISS Approach - University of North Dakota

atch submission script applicable to your job, and submit your job. CRCRAM will then
check the allocated nodes to ensure PVM is available, scan the directory creating the data
files to process, parse the “Data_process_script” specified in the CRCRAM.ini and
replace the $1 and $2 variables with actual data file names, and call the specified program
to process the data (file by file). Since CRCRAM scans the directory creating a list of
data files to process each time CRCRAM is executed, you can increase or decrease the
number of data files for each run by adding or removing files from the directory without
having to make any changes to CRCRAM.ini or the batch submission script. Similarly,
since CRCRAM gets the list of nodes to execute on from the batch scheduler each time
CRCRAM is executed, you can change the number of nodes or processes per node in the
batch submission script and CRCRAM will adjust the load balancing accordingly.
Finally, the load balancing mechanism is master-slave and naïve. Since it does not have
any information regarding the processing time of any data file, it may very well end up
waiting for a single long-to-process file to be completed while all of the other files have
long finished. An informed scheduler would process long-to-process files first.
20