New Statistical Algorithms for the Analysis of Mass - FU Berlin, FB MI ...

5.3. QAD GRID PLATFORM SERVER 127
the current version of the worker is transferred as a zip archive and
unzipped into a new sub-directory.
5. Further commands - as stored in the database for this kind of worker -
are executed.
6. The worker is started via the worker specific command line stored in the
QAD Grid’s database during registration (see section 5.5.2.
Task Scheduling and Provision
One of the central functions of the Grid platform server is to provide jobs and
their respective details for workers. This means, there must exist two basic
functions on the platform server:
� Receive jobs from some instance and insert them into the central job
queue. The jobs can be send from some (authorized) worker or from a
Grid platform itself where a user has started an analysis that results in
a set of jobs.
� Provide jobs to workers: each authorized worker can request jobs of a
particular kind from the server (see below). A job contains all needed
information the workers needs, such as algorithm parameters, location
of data to be analyzed etc.
Job/Worker Matching
Each worker can handle exactly one particular kind of job, such as copy a file,
perform an analysis or classify an item (see section 5.4). Hence, each job and
each worker is assigned a so called job type id (JTI) tag. If a worker requests
a job it sends its JTI tag and the platform server checks if unprocessed jobs
tagged with this JTI exist. If this is the case the first job in the queue is marked
“in progress” and the parameters transferred to the requesting worker.
Requesting Data
To handle a task a worker mostly needs a dataset to e.g. perform an analysis
on. This data is stored at the central Grid server and usually at some workers
within the Grid. To get this data the worker queries the platform server to
get a list of all nodes that currently host that particular dataset. The request
includes the geographical location and the id of the needed dataset. The
resulting list includes the machine’s IP addresses ordered by the (geographical)
distance to the requesting worker. We could also have used the upload speed
of the target as order criterion but as hosting nodes must have a large upload
bandwidth the geographical location is considered to be more important to
save total network (Internet) bandwidth.
Using the resulting list the worker tries to connect to and request from the
closest node to get the needed data (see section 5.3.1). If a connection fails it
will try the next machine. If all connections fail it will request the data from
the central server.