COMP 310 Fall 2017	HW07: Remote task execution

This project has a "compute engine" (ICompute) on the server  which is served out as a remote object.   The client then sends a "compute task" (ITask) to the server for it to execute, returning the result to the client.   The key issue here is that the server does not know a priori what the client will have it do. 

Before starting this assignment, be sure that you can run the demo from Lab08, which includes configuring your computer for use with RMI.

This assignment and all future assignments will use TCP ports in the ranges: 2001-2002 and 2099-2102.  Be sure they are all open on your computer!

This assignment page does NOT contain all the information necessary to successfully and properly complete the project! 
You MUST carefully read ALL THE DOCUMENTATION, including all Javadocs and README.txt's to fully understand the system and the project requirements.

References:

• Remote Method Invocation (RMI)
  • RMI connectivity issues

Obtaining the Provided Code

The code is available by setting the svn:externals property (under Team/Set Property) of your src folder to "provided https://svn.rice.edu/r/comp310/course/ComputeEngine/provided"

Be sure to set the working directory of both the client and server launch configurations to your the "bin" directory in your project.  
Also, set both the client and engine (server) launch configurations to be "shared" so that they will always be set correctly.
See the Eclipse Resources page for directions  

Ports that need to be opened (see the code for the values):

• In-bound on Server:
  • IRemoteTaskViewAdapter.BOUND_PORT_SERVER -- Used by IRemoteTaskViewAdapter sent from server to client to display back on server's view.
  • ICompute.BOUND_PORT-- The port the ComputeEngine is bound in the Registry.
  • IRMI_Defs.CLASS_SERVER_PORT_SERVER -- The class server the server uses.
  • IRMI_Defs.REGISTRY_PORT -- The port the Registry uses on the server.
• In-bound on Client:
  • IRemoteTaskViewAdapter.BOUND_PORT_CLIENT -- Used by IRemoteTaskViewAdapter sent from client to server to display back on client's view.
  • IRMI_Defs.CLASS_SERVER_PORT_CLIENT -- The class server the client uses.

Click for full-size image

Features

• Via the remote ICompute interface, the server will execute any ITask implementation sent to it, returning its results in a type-safe manner.
  • IMPORTANT:  The compute engine (ICompute) is NOT the same as the engine application's MVC "model".   The compute engine is just one of the objects managed by the engine model.
• A client connects to a server by retrieving the server's ICompute STUB from the server's Registry.
  • The ICompute stub should be the ONLY entity ever bound in the Registry!
  • The server does not need to ever initiate a connection to the client.  
    • Two-way communication between the client and server is established when the client first connects to the server.
• When a task arrives at the server, it's setTaskViewAdapter() method is called to attach it to the server's view by providing the task with a local view adapter, so that any status output from the task is displayed on the server's view.
• Not only are the tasks sent from the client to the server, but also, the ICompute interface provides a setTextAdapter() method enables the client to send a stub of a remote view adapter to the server so that the server can display information on the client's view plus that same method will return a stub of a remote view adapter that will enable the client to display information on the server's view.
• Object serialization:
  • Serializable interface tells the compiler that this object can be serialized, i.e. its data (not its code!) can be turned into a byte stream and sent over the network.
  • Code, if needed, is transmitted separately and automatically.   The client's java.rmi.server.codebase value is imbedded in the serialized byte stream.   The receiver's Java class loader will look at this value and find the class file server on the sender and then issue an http-type request to download the associated class file.
  • Note that the class file server does not have to be co-located with the serialized object's sender.   The class file server can be a separate machine altogether, e.g. a central class file repository.
  • transient fields are not serialized.
  • Optional readObject method in Pi2 and GetInfo is used for custom deserialization processes, here to reinitialize the transient taskView field.   The server will attach its own ITaskView adapter so the transmitted task can communicate to the server.
• IRMIUtils:
  • Has services for creating and/or locating the Registry for the client and server use, plus to obtain the local IP address in a consistent manner across multiple operating systems.
  • Has services to start and stop the RMI sub-system with minimal effort and maximum reliability.   
  • Abstracts and encapsulates the Registry obtainment process.
  • Uses techniques for composing lambdas to enable the methods to send multiple messages to multiple output targets.
  • Note:  There are some typos still running around where "IRMIUtils" (or its concrete implementation, RMIUtils) are still referred to by their old name,  "RegistryFactory"
• The client can send a message that will appear on the compute engine's view and vice versa.

Read the code documentation!  (Be sure to read the package documentation too:  In the Javadoc web page, click the "Overview" tab and then click the package name in the right-side box.)

(Package Javadocs can also be found in the package-info.java file in the package.)

Demo JARs

The provided code includes fully operational demos for both the client and the engine server as executable JAR files in the provided.demo package.   See the Javadocs for that package for complete directions on how to run the demos and their capabilities.

It is an Honor Code violation to attempt to decompile the provided JAR files.

The demos are fully compliant with the required inter-computer API as defined by the provided.compute package.   Likewise, the provided sample tasks are API compliant.   The demo client, the demo engine and/or the provided sample tasks can thus all be used to test student solutions.    This will enable each piece of the project to be tested separately.   This is NOT, however, a substitute for the required testing with other students!  

Note: The demo client is only capable of loading ITask implementations that have a public static FACTORY field of type ITaskFactory.    The sample ITasks are pre-loaded into the demo client for convenience only (loaded by its controller -- not hardwired into the view or model!).

Use the demos as part of your development process:

One great advantage of having these fully operational demos at your disposal is that you can now completely separate the development of your client from your engine server from your custom tasks.  One of the goals of the project is to demonstrate how all these parts are completely decoupled, so that is exactly how you should structure your development process! 

Important concepts:

Stubs retrieved from remote Registries -- The ICompute stub that the client retrieves via the Registry delegates its calls back to its ComputeEngine implementation on the server.

Stubs sent and returned by stubs  (stubs as factories) -- IRemoteTaskViewAdapter stubs are both sent to the server via a method in the ICompute stub in the client as well as returned by the ICompute stub as a return value.  Thus a single stub can serve as a factory for all the other stubs that are needed, eliminating the need for multitudes of objects bound in the Registry--only a single factory needs to be bound.

Entire objects being serialized and sent via stubs -- the ITask objects are being completely serialized and sent over the network from the client to the server.  They are not sent as stubs.  

Automatic remote dynamic class loading of serialized objects -- If the server does not have the code for the concrete ITask sent to it, the RMI system settings that the client and server set up automatically cause the code for the task to be dynamically loaded from across the network, from the client's class server to the server.  Likewise, this is done for the stubs that are sent back and forth.

Task factories -- Abstractions of the task instantiation process that enable dynamic loading of tasks.  The provided ITaskFactory interface is designed to encapsulate the conversion of a String (as would be passed from a user interface) to whatever type of constructor parameter, i.e. not necessarily a String, the associated ITask implementation requires.  Typically, task factories do not require constructor parameters for their own instantiation, which helps further decouple the specific tasks from the rest of the system.    Student solutions are not required to use the provided ITaskFactory but justifications on the architecture used to enable dynamic task loading should be included in the documentation and/or README.txt file.

Task factory loaders -- An encapsulation of the process of dynamically loading a task factory.  While individual task factories are are generically typed to the return type of  the tasks they produce, i.e. ITaskFactory<T>,  task factory loaders' load() methods are typed to an unbounded wildcard return type, i.e. ITaskFactory<?>.   In terms of dynamic task loading, this enables the system architecture to be decoupled from the task return type.    The provided code includes an implementation of ITaskFactoryLoader that is able to load ITask implemenations that include a field, public static ITaskFactory FACTORY, that is the singleton task factory instance for that specific type of task.   Student solutions are not required to use the provided ITaskFactoryLoader but justifications on the architecture used to enable dynamic task loading should be included in the documentation and/or README.txt file.

Custom task result formatters -- The client application should be designed to be independent of the return type of the tasks that it loads and runs but the tasks need to keep the strong typing of their return types.  The ITaskResultFormatter interface provides a means for the client application to transparently convert the strongly-typed task results to useful and informative task-specific strings that can be displayed to the user.  Since the formatting of a result is specific to each instance of a task, the ITask interface provides a factory method for the task to generate its own custom formatter.   The client application only needs to deal with the formatters as a unbounded wildcard type:  ITaskResultFormatter<?>. or as a generic type, ITaskResultFormatter<T>, in a generic method (recommended, e.g. see IClientModel.runtTask()).    Theoretically, the compute engine could also use the task's result formatter, but on the engine side, one is generally more interested in the raw task results rather than a nicely formatted output.

Local vs. remote task view adapters -- There are two kinds of task view adapters in use by the system for two very different purposes:

• ILocalTaskViewAdapter -- A non-serializable adapter used by an ITask to display text messages on the local system, e.g. the engine server when it is processing a task.  It is crucial that this adapter NOT be transmitted with a task when it is sent from one computer to another.    When an application, e.g. the engine server, receives a task, e.g. from the client, the first thing it should do is to set the task's local view adapter to that local system's ILocalTaskViewAdapter instance.   Technically, this adapter would connect the task to the local system's model, not its view, as the local model would then do whatever it takes to pass the message on to its view.
• IRemoteTaskViewAdapter -- An RMI Server object whose stubs are used by a remote system to display messages on the local system's view.  Again, technically, this adapter would connect the task to the local system's model, not its view, as the local model would then do whatever it takes to pass the message on to its view.   The compute engine interface (ICompute) has the ability for the local client system to exchange IRemoteTaskViewAdapter stubs with the remote engine server, thus enable both sides to display text messages on the other.

Requirements:

You will need to create your OWN client and engine MVC implementations (two separate applications) plus your own custom ITask implementations.   Please see the IClientModel and IEngineModel interfaces for guidelines on building one's client and server implementations.  

• Do NOT copy or modify the provided.compute or provided.rmiUtils packages!
• The IClientModel and IEngineModel interfaces are provided as a guide to help you get started by creating methods that do the necessary operations.  
  • It is recommended that you start by creating client and engine models that implement these interfaces.
  • Use of these interfaces is not required, so once you get a simple solution operational, then remove the references to these interfaces which will free you to customize your system to your needs.
• Dynamic loading of ITask implementations is provided by ITaskFactory and ITaskFactoryLoader (concrete implementation: SingletonTaskFactoryLoader).   It is NOT required that this exact mechanism be used for dynamic ITask loading -- students are free to design their own dynamic loading systems
  • Please make a note in the README.txt file if the student client solution is not capable of loading the sample tasks (recomendation: create compatible versions), though the student engine solution must be able to run the unmodifed sample tasks.

You may NOT change any of the provided code -- you will receive a commit error if you do so!

Student work:

Students are required to design and implement both client and engine server applications plus custom tasks.

• The client application must have at least the following behaviors and capabilities:
  • Is able to connect to any engine server implemtation that adheres to the task and compute engine specifications detailed by the provided.compute package and the RMI configuration as created by the provided.rmiUtils package.
    • Is able to send a status message to the the engine server immediately and automatically upon connection.
    • Is able to display a status message from the server received immediately after connection.
    • Is able to send a text message to the engine server at any time.
    • Is able to display a text message from the engine server at any time.
  • Dynamically load any "compatible" ITask implementation from anywhere in the Java classpath.
    • Where "compatible" means that the task class is compatible with the dynamic class loading architecture being used and not anything about the processing operation the task performs when it is run.
    • Is able to construct different instances of the same ITask implementation using different constructor parameters, where the value of those parameters clearly affects the processing behavior of the task when it is run.
  • Is NOT hardwired to any specific task return type.  
    • A task instance-specific ITaskResultFormatter must be used to convert a task's strongly typed return value to a String for display.
  • Is able to compose and run any two or more tasks who share a valid constructor value(s) that can be represented by the same string, i.e. the same string value can be converted into the different constructor types needed by the composed tasks.  For instance, "3.14" can be converted into a double, a string and an array of characters.
    • Any composed task must be treated by the system as an encapsulated whole, no differently than any other task, including being sent to the compute engine as a single task object.
  • Is able to shut down gracefully such that the RMI system is explicitly shutdown as the application quits.
• The engine application must have at least the following behaviors and capabilities:
  • Enables connection by any client implemtation that adheres to the task and compute engine specifications detailed by the provided.compute package and the RMI configuration as created by the provided.rmiUtils package.
    • Is able to send a status message to the client  immediately and automatically upon connection by the client.
    • Is able to display a status message from the client immediately after connection.
    • Is able to send a text message to the client at any time.
    • Is able to display a text message from the client at any time.
  • Is able to receive and run any ITask sent to it using its ICompute stub. 
    • Is able to display the raw task result value after the task is run.
    • Is able to return the raw, strongly typed task result back to the client.
    • During a task execution, is able to display any text message sent from the task through its ILocalTaskViewAdapter.
  • Is able to shut down gracefully such that any entries bound into the local Registry are unbound and the RMI system is explicitly shutdown as the application quits.
  • An engine server should be able to support multiple simultaneous connected clients, including running any task sent by any connected client at any time.
• The ITask implementations must have at least the following behaviors and capabilities:
  • At least 3 new ITasks must be implemented
    • Across the new tasks, there should be at least 3 different return value types represented. 
    • All tasks' processing behavior when run should be dynamically customizable by a parameter supplied at the task's construction.
      • At least 2 tasks must use a constructor parameter that is NOT a String.
  • All tasks must be able to run on any server implementation that adheres to the task and compute engine specifications detailed by the provided.compute package and the RMI configuration as created by the provided.rmiUtils package.
  • All tasks must generate a task-specific status message while it runs that is displayed on the server engine user interface.
  • Neither the client nor the engine server can be hardwired to a specific task return type.
• Both the client and engine server applications must run on all team member's machines and of course, the staff machines.  This includes being able to:
  • Connecting the client to the server when both are running on the same machine
  • Connecting the client to the server when they are running on different machine. All possible connection ways on team member machines must be tested!
• Inter-team testing:  Both the client and engine server applications must be tested with at least 10 people (on 10 different computers) on at least 10 different teams, i.e. against at least 10 different client & engine implementations.
• Client and engine launch configurations:  You must create and save in your repository, run (launch) configurations for both the client and the computer engine.

Note that the provided.compute package contains all the interfaces that BOTH the client and the server need to talk to each other.   Do NOT copy or modify this package!

Testing

Testing RMI programs can be tricky.   You can easily be fooled into thinking that your code runs properly if you test insufficiently.   In particular, if both ends of your test contain all the code, the remote dynamic class loading will not be tested.  Here's a more complete test:

• Always test your client and engine applications against the provided  demo client and engine, including all possible tasks.   This will help highlight and track down compatibility issues.
• Create a separate project with just the provided code in it.   Run the demo engine server from this project and test your client against it.  
• Create a separate project that is another checkout of your own code.   Delete your custom tasks from this code and then run the compute engine only.     
  • DO NOT EVER COMMIT THIS CODE!  -- you may actually want to branch your code, though this can be a hassle when you are updating your code.
  • Test the client from your main project against this engine.   This will test the remote dynamic class loading of your tasks as well as your implementations of the missing provided code.
• Be sure to do the above test across different machines!   This will test all aspects of your code.    Testing against someone else's code is NOT the same as testing against your own code!    There are errors that will ONLY show up when testing against a someone else's implementation!

Testing Requirements:

• All team members must test both the team's client and server against all other team members.
• Each team member must test both the team's client and server against at least 5 other team's client and server applications.
• The total number of other teams tested against by the whole team must be at least 5 times the number of team members, e.g. 10 other teams for a team of 2 or 15 other teams for a team of 3.
• The project's README.txt file must include a record of all successful testings by listing each team member's NetID against the NetID and team number of those with whom that team member has successfully tested both their client and server.  
  • An included Excel spreadsheet or a reference to an online spreadsheet is an acceptible record so long as it made clear in the README.txt and the staff is access to the document.
  • Testing against one's own teammates should also be included in the record.
  • The record should also include any unsuccessful tests with notes about what went wrong.   Unsuccessful test do NOT count towards the meeting the testing requirements.
  • If the required number of successful tests was not completed, the records must indicate that an exhaustive testing search across the entire class was performed in an effort to meet the testing requirements.

Most Common Errors

• Firewall ports not open -- typically shows up a "connection refused/timed out" error/li>
• Run (launch) configurations not set properly -- working directory not set, which typically shows up a security manager error on startup. 
• Editing the provided code  -- shows up as a commit error
• Transmitting an RMI server object instead of its stub.  -- typically shows up a "(de)serialization" or "(un)marshalling" error.
• Attempting to transmit a non-serializable entity -- always check if that entity should have been transmitted in the first place!   Don't rely on Eclipse's "Quick Fix"!

• Special note for teams where the members are running a mix of 32-bit and 64-bit Java computers (this is very rare these days):  You will need to use a special "svn:ignore" property to enable the different computers to separately and properly compile your code.   Please see the following directions on how to set the svn:ignore parameter to ignore the .classpath file.

Grading Guidelines

1. The at least 3 tasks implemented that must return at least 3 different types of results.   - 15 pts
2. All tasks are dynamically loaded into the client.  No tasks are hardwired into the client or engine applications. - 5 pts
3. Able to dynamically create,  send and execute a composite task.   - 3 pts
4. The tasks' results should be displayed on the client and engine's views, with task-specific formatting, using the task's ITaskResultFormatter object, appearing on the client.   - 5 pts
5. The task execution must display some sort of status message on the server when it is run.   That is, the task's execute method should utilize the ILocalTaskViewAdapter given to it by the compute engine to display a status message. - 5 pts
6. An IRemoteTaskViewAdapter stub properly created on the client and automatically transmitted to the server upon connection. - 5 pts
7. An IRemoteTaskViewAdapter stub properly created on the engine server and automatically transmitted to the client upon connection. - 5 pts
8. Connection status messages sent both ways between the client and server automatically upon connection. - 5 pts
9. Able to send string message from client's view to connected compute engine's view. - 5 pts
10. Able to send string message from compute engine's view to connected client's view.- 5 pts
11. The task should run properly when both the client and server are on the same machine.   This implies the ability to connect. - 5 pts
12. The task should run properly when the client and server are on different machines.  This implies the ability to connect. Test this between your and your partner's machines! Your code will be tested against a course staff machine. - 5 pts
13. The above described testing requirements were met.  - 15 pts
14. Run configurations created and saved in repository for both client and compute engine. - 5 pts
15. Full Javadocs and UML diagrams covering the both client and engine server applications.  - 5 pts
16. Discretionary - 7 pts

Note that the above requirements mean that you cannot change anything in the provided.compute or provided.rmiUtils packages!

© 2017 by Stephen Wong