New Statistical Algorithms for the Analysis of Mass - FU Berlin, FB MI ...

126 CHAPTER 5. COMPUTER SCIENCE GRID STRATEGIES
To distribute data the system creates tasks (see section 5.3.1) that are
inserted into the system’s job queue with an assigned priority that is lower
than analysis tasks. Therefore, if a new analysis is queued it will be handled
before the next copy task will be executed.
Remarks
To disable this feature set worker storage size to zero. This might be useful
in local area networks where data is available through the network file system
anyway and does not need to be copied.
5.3.3 Job Execution
In this section we describe how jobs are actually created and executed. This
comprises starting of workers at target machines (that will do the actual computational
work), creation and scheduling of tasks and the actual execution of
tasks, that is matching of tasks and workers. Further, we give details about
how workers get job specific data from the Grid.
Worker Injections
One of the most striking features of the QAD Grid approach is that there is
no need to (pre-)install client software on the working machines. The only
requirement is that the target machine provides SSH access. When a machine
is added to the Grid the platform server logs into that machine, copies the
client software and needed libraries to this target and executes it. The worker
then connects itself to the Grid and starts working (see section 5.4). We have
tested this with different types of Linux, MacOS and Windows systems running
a variety of SSH servers. If a target machine cannot provide SSH access
the worker can of course be started manually on that machine. Another possibility
on Windows-based systems is the usage of WMI (Windows Management
Instrumentation) which is also supported. This allows the QAD Grid Server
to log into an windows machine and - as in the SSH case - remotely execute a
program.
The main advantage of this approach is that only the network (IP) address
of a new client needs to be added to the QAD Grids client database and
there must exist a user account on that machine for the QAD Grid server to
login. Then, the server can inject the client software and start a worker on
that machine without further user interaction. These workers are then fully
controlled by the QAD platform server.
Details of this injection process are as follows:
1. The QAD Grid server establishes a SSH (Linux) or WMD (Windows)
connection to the target machine.
2. If no sub-directory “qad grid” exists within the clients temporary directory,
it is created.
3. The “qad grid/3rd party libraries” sub-directory is checked whether all
libraries needed by this worker are available. If not, missing libraries are
transferred via SCP.
4. The “qad grid/workers” sub-directory is checked for the existence of the
worker to start. If it does not exists or the available version is outdated