Project2: A Simple Distributed Computing Platform (Due at 11:59:59pm on 04/12/2012 (EST))
You are asked to develop a replicator (client) that distributes a large job over a number of computers (a server group) on a single switched LAN (our BU04 lab). In this assignment, a large (simulation) job can be divided into a number of small jobs, each of which can be assigned to one machine from the server group for execution. The execution results of the small jobs can be merged once all of them successfully terminate.
client server1 server2 server3 ...
| | | |
| | | |
| LAN | | |
The client and servers are running Network File System (NFS) so that user files are visible at $HOME directory. You may want to set up the following environment:
$HOME/[url removed, login to view]: a list of (server) hostnames which participate in the simulation.
$HOME/replicate_out: the directory that stores the small job execution result.
The simulation program "hyper_link" (binary) is provided. In this assignment, you don't need to know or care what "hyper_link" does, and actually it is a computing intensive (CPU demanding) simulator. The command line arguments of "hyper_link" are job# 100000 999 1 2 2 100, where the job number determines the number of small jobs in your simulation. To allow the client to run a large job, the job# should be given in a tuple: start, end, and step. For example, the command "hyper_link 1 100 1 100000 999 1 2 2 100" yields 100 small jobs with the job# starting from 1 to 100. Each small job produces a screen output (see example below) at the end (if finished successfully). Your code needs to redirect the output to a file and save it in $HOME/replicate_out. For example,
./hyper_link 1 100000 999 1 2 2 100
will produce a screen output looks like:
The communications between the replicator and servers are achieved through remote procedure calls in the client-server fashion.
A user interface is required for the replicator to control the server. A command line interface will be acceptable. A (working) graphic user interface (GUI) will impress the instructor and earn up to 20 bonus credits. Your interface should at least support the following operations.
show the current CPU load of a certain server (if the server is active).
show the current server status (active or inactive).
stop a certain server.
restart a certain server.
For those who are going to implement GUI, you need to create an icon for each server, and show the server status in the real time, e.g., the CPU load (with the mark of hi-threshold), active/inactive, etc.
The replicator has to make sure all small jobs are successfully finished.
If a server crashes (or not responsive), the running job (not finished yet) will be killed and rescheduled for execution.
If a server CPU load exceed the preset threshold (the higher threshold), the replicator stops the server (and therefore kills the job).
The replicate should keep polling the CPU load of the stopped server. Once the load becomes lower than the lower threshold (a preset value), the server should be reactivated to run the jobs.
The replicator can also stop any server (through user interface) if needed. Once happened, the unfinished job will be killed.
If a job terminates abnormally (e.g., being killed), the replicator has to reschedule the job execution later.
Makefile: you need to provide a Makefile that allows the instructor to compile your code by simply typing "make".
Write-up: you are required to write a README document (either in TXT or PDF format) that describes your project design detail and the execution sequence (