COMP 310
Spring 2018

Lec28: Generic Extended Visitors and Data Packets

Home  Info  Canvas   Java Resources  Eclipse Resources  Piazza

One last item about RMI:  Transient Fields in Classes

HW07 overview


Warning:  the material in this lecture is conceptually very difficult.   No student is expected to understand this material right away.   We will be taking the approach of quickly showing the class the breadth of the topic to give everyone a feel for the overall issues involved and then as we encounter specific issues in our examples, we can deal with them in as a concrete, contextual manner as possible.

Advice: When studying the advanced generics discussed here, concentrate more on the ideas and relationships that the generics are trying to express and less on the syntax that is being used to implement those notions.  

First let's review a little bit on generics and push those notions on to generic list frameworks.


Generic Extended Visitors

Life gets a bit more complicated when we try to apply generics to extended visitors.   The difficulty stems from the fact that the type of the host ID (index) and the type of the host itself are not independent.  Nor are they independent of the visitors that are used to process them.  

Unfortunately, the Java generics syntax to express this is unwieldy at best and unable to do the job at worst.  The following the current state of the art in expressing the relative typing notions in extended visitors:

For instance, a particular type of host (specifically, the top-level, superclass of all concrete hosts) is defined as using an ID value of a particular type.   This is related to the fact that a host should only accept (execute) a visitor who can handle that type of host.   We want to enforce the relative covariant-contravariant relationship between the host and visitor, a characteristic of these sorts of component-framework systems, as discussed in the above linked Resources pages.   A host thus becomes defined in terms of the type of the host that its visitors can be accepted by: 

/**
 * An interface defining a host for an IExtVisitor who can visit this type of host.
 * Each concrete host will have a particular index value associated with it and call
 * the case associated with that index value on the visitor.
 * @param <I> The type of the index value used by the extended visitor.
 * @param <H> The type of the host used by the extended visitor.  Restricted to be an implementation of IExtVisitorHost<I, ? extends H>
 * 
 */
public abstract interface IExtVisitorHost<I, H extends IExtVisitorHost<I,? extends H>> extends Serializable{
  /**
   * The method that executes (accepts) the extended visitor to process this host.
   * @param <R> The return type of the visitor.
   * @param <P> The type of the input parameters of the visitor.
   * @param algo The extended visitor to execute.
   * @param params  Vararg input parameters to the visitor.
   * @return The return value from executing the visitor.
   */
  public <R, P> R execute(IExtVisitor<R, I, P, ? extends H> algo,  P... params);
}

These sorts of generic definitions are sometimes called "recursive typing" for obvious reasons.   This recursive typing necessarily affects the typing of the visitor, which must be defined in terms of a host...a host who can accept (execute) visitors whose hosts are supertypes of this visitor's host.   Got that?   

/**
 * Interface that defines an extended visitor that has specific types for its
 * return value, R, its index value, I, its input parameters, P, and its
 * host, H.   The host is restricted to being a subclass of IExtVisitorHost 
 * who takes the same index value and who accepts a visitor that takes this 
 * same host type.
 * @param <R> The type of the return value
 * @param <I> The type of the index value
 * @param <P> The type of the input parameters
 * @param <H> The type of the host, restricted to being a subclass of IExtVisitorHost<I, H>
 * @author Stephen Wong (c) 2010
 */
public abstract interface IExtVisitor<R, I, P, H extends IExtVisitorHost<I, ? super H>> extends Serializable{
	/**
	 * The parameterized case of the visitor.  The case is parameterized by the index value, idx.
	 * @param <T> The type of the host the is expected to call this method.  T is restricted to be a subclass of IExtVisitorHost<I, ? super H>
	 * @param idx The index value for the desired case
	 * @param host The host for the visitor
	 * @param params Vararg input parameters
	 * @return The value returned by the running the indexed case.
	 */
	public <T extends IExtVisitorHost<I, ? super H>> R caseAt( I idx, T host, P... params);
}

When one moves down a level to the implementation of a command-based generic extended visitor, the generics get even hairier.   The situation is complicated by the fact that at this level, these are abstract classes that are supplying strongly-typed services for their subclasses.   In the end, we want the typing to apply to the sub-class, not this abstract class:  

/**
 * Abstract implementation of IExtVisitorHost that provides defines a host by an index value and thus, 
 * a concrete execute method.    
 * 
 * Usage:   
 * public class MyExtVisitorHost extends AExtVisitorHost<MyIndex, MyExtVisitorHost> {...}
 * 
 * @param <I>  The type of the index value that is being used.
 * @param <H>  The type of the concrete SUBCLASS that extends this class.
 * @author Stephen Wong (c) 2010
 */
public abstract class AExtVisitorHost<I, H extends IExtVisitorHost<I, ? extends H>>  implements IExtVisitorHost<I, H> {
	// code body elided
}

The above declaration looks similar to those above, but the meaning of "H" has changed to refer to the subclass's type.   The same rubric applies to the abstract command-based visitor class:

/**
 * Abstract implementation of IExtVisitor that adds an invariant implementation of 
 * storing commands associated with each case in a dictionary indexed by the 
 * case's associated index value.   
 * When a particular case is called, the associated command is retrieved and 
 * executed.  The return value is the return value from the command.
 * If no associated command is found, then a default command is executed.
 * In general, command-based implementations of IExtVisitor will be concrete subclasses of this class.
 * 
 * Usage:  
 * public class MyExtVisitor extends AExtVisitor<MyReturn, MyIndex, MyReturn, MyExtVisitorHost> {...}
 * 
 * @param <R> The type of the return value
 * @param <I> The type of the index value
 * @param <P> The type of the input parameters
 * @param <H> The type of the host, restricted to being a subclass of IExtVisitorHost<I, ? super H>
 * @author Stephen Wong (c) 2010
 */
public abstract class AExtVisitor<R, I, P, H extends IExtVisitorHost<I, ? super H> > implements IExtVisitor<R,I,P, H> {
	// code body elided
}

The typing issues carry over to the commands that are used.   The commands are typed similarly to the visitors that hold them:

/**
 * Interface that defines a command used by AExtVisitor that has specific 
 * types for its return value, R, its index value, I, its input parameters, P, 
 * and its host, H.   The host is restricted to being a subclass of 
 * IExtVisitorHost who takes the same index value and who accepts a visitor 
 * that takes this same host type.
 * An IExtVisitorCmd is associated with every recognized case of an AExtVisitor,
 * including the default case.
 *
 * Usage: 
 * new IExtVisitorCmd<MyReturn, MyIndex, MyParam, MyExtVisitorHost>(){
 *     public <T extends IExtVisitorHost<Integer, ? super MyExtVisitorHost>> MyReturn apply(MyIndex index, T host, MyParam... params) {...}
 * } 
 * 
 * @param <R> The type of the return value
 * @param <I> The type of the index value
 * @param <P> The type of the input parameters
 * @param <H> The type of the host, restricted to being a subclass of IExtVisitorHost<I, ? super H>
 * @author Stephen Wong (c) 2010
 */
public abstract interface IExtVisitorCmd<R, I, P, H extends IExtVisitorHost<I,? super H>> extends Serializable{  
  /**
   * The method that is run by AExtVisitor when the case associated with this 
   * command is executed.
   * @param <T> The type of the host the is expected to call this method.  T is restricted to be a subclass of IExtVisitorHost<I, ? super H>
   * @param index The index value for the case for which this command is associated. 
   * @param host The host for the visitor
   * @param params Vararg input parameters
   * @return The value returned by the running this command.
   */
  public abstract <T extends IExtVisitorHost<I, ? super H>> R apply(I index, T host, P... params);
}
 

To see what an algorithm using generic extended visitors, please see the implementation of the deletion algorithm for the self-balancing trees.

One of the key reasons for using the recursive generic typing described above is to enable the transparent (in terms of typing) insertion of intermediate abstract classes that provide services to their subclasses.    The recursive typing allows the implementation level of the extrended visitor hierarchy to be typed relative to the classes and interfaces at that same level, rather than with respect to their superclasses as would happen if the recursive typing had not been used.


(click for full-size image)
Datapacket UML diagram

Generic Data Packets

Data packets are simply wrappers around data objects.   Think of data packets as shipping boxes for data.    Data packets enable the rest of the system to deal with an consistently typed object that provides services relevant to the transport of the data.     Here, the data packets will support a visitor design pattern, which will enable us to do type-dependent processing on the data contained inside the data packet.   Since generic data packets are a more specific application of generic extended visitors, they enable us to implement some welcome syntactical simplifications.   

In the end, we'd like to define something like DataPacket<T> where T is the type of data the data packet holds.    We could use something like integers or strings for the ID type, but we'd much rather have something whose value is definitively unique to the class T.     In Java, there is a whole "meta-programming" system called the "Java Reflection" system.   This is a series of classes that are used to represents classes, methods, etc.    In particular, the class Class<T> is a class that represents the class of T.   (Yes, it's a class that represents a class!).

Every class and interface in Java, T, has a static field called "class" that references an object of type Class<T>.   That is, String.class references an object of type Class<String>  and  JFrame.class references an object of type Class<JFrame>.    This object is unique for every class and can thus be used as the ID value in the extended visitors.    It should be noted that "T.class" is not a valid Java syntax because by run-time, the actual class of T has been type-erased (and yes, it is legal in C#!).

First we define a superclass that nails in the use of the Class object as the ID type.   The "unbounded wildcard" generic syntax "Class<?>" is read as "a Class object representing any type".  Note that this is not the same as Class<Object> which is specifically a Class object representing type Object

/**
 * Abstract data packet that defines the use of a Class object as the index type.  
 * The type of data held by the data packet defines its type and thus what case it
 * calls on its processing visitors.
 * 
 * @author Stephen Wong (c) 2010
 *
 */
public abstract class ADataPacket extends AExtVisitorHost<Class<?>, ADataPacket> {
	
	private static final long serialVersionUID = 8005386928491056679L;

	/**
	 * Constructor for this abstract superclass
	 * @param c A Class object to be used as the index value defining this type of data packet.
	 */
	public ADataPacket(Class<?> c){
		super(c);
	}
}

The next level defines the ability to hold arbitrary types of data.   Because of type erasure, the data packet is unfortunately not able to figure out what its ID value is, so that must be supplied manually.  See the example in the documentation below.   Note that one cannot simply extract the type information from the supplied data object because the type of the data packet may be the supertype of the supplied data, thus a data type extracted from the supplied data could be the wrong type to use.

The data packets that one will use will be instances of this class:

/**
 * Concrete data packet that holds a generic type of data.
 * 
 * @author Stephen Wong (c) 2010
 *
 * @param <T>  The type of the data being held.  T must be Serializable.
 */
public class DataPacket<T> extends ADataPacket{

	private static final long serialVersionUID = -57375281215880284L;
	
	/**
	 * The data being held
	 */
	private T data;
	
	/**
	 * The constructor for a data packet. ;
	 * Usage:
	 * 
	 * ADataPacket dp = new DataPacket<MyData>(MyData.class, aMyData)
	 * 
	 * @param c Must be T.class where T is the data type being used.
	 * @param data  The data to be held in the data packet
	 */
	public DataPacket(Class<T> c, T data){
		super(c);
		this.data = data;
	}
	
	/**
	 * Accessor for the held data
	 * @return  The data being held
	 */
	public T getData(){
		return data;
	}	
}

IMPORTANT:  The actual type of the data parameter above does NOT have to exactly be T!   In fact, it usually is not because T is typically a top-level interface describing the data type while the data variable is a concrete implemention of T.    The use of interfaces to describe the data types decouples the system from the data type implementations.

ALL fully serializable data types defined by a network API should be defined in terms of interfaces, NOT concrete classes unless a specific invariant concrete implementation is required, at which point the data type would be defined in terms of an abstract class.  This means that a network API is defined solely by interfaces except in very rare circumstances.

In general, c != data.getClass() and c != MyData.class ==> instead  c = IData.class where IData is the interface that data's class, MyData,  implements.  The datapacket is thus DataPacket<IData>, NOT DataPacket<MyData>.

 

The visitor for a DataPacket<T> is simplified because we know its ID type:

/**
 * Concrete visitor for processing an abstract data packet.
 * For convenience and increased type safety, the commands used by this visitor should be subclasses of
 * ADataPacketAlgoCmd<R, D, P>, where D is the type of the data that particular command processes,
 * i.e. the defining type of its associated DataPacket<D> host.
 * @author Stephen Wong (c) 2010
 *
 * @param <R>  The return type of the visitor
 * @param <P>  The vararg input parameter type of the visitor 
 */
public class DataPacketAlgo<R,P> extends AExtVisitor<R, Class<?>, P, ADataPacket> {

	private static final long serialVersionUID = 2045919428614770686L;

	/**
	 * Constructor for the class.
	 * @param defaultCmd  The default command to be used.  See the main class documentation for usage suggestions.
	 */
	public DataPacketAlgo(ADataPacketAlgoCmd<R, Object, P> defaultCmd){
		super(defaultCmd);
	}
}

The second type parameter of the ADataPacketAlgoCmd default command above is the type of data is works on. Because it is the default command, it must take type Object as its data type.

In the full generic extended visitor implementation, the commands were problematic because Java's type system did not allow us to strongly type the host input parameter to match the actual hosts used.  But since we are working in the reduced, more specific space of data packets, we can exactly determine the host's type a priori.  The application of a Template Method Pattern enables us to delegate from a type-unsafe apply method to a type-safe one:

/**
 * A DataPacketAlgo command that is designed to work on a DataPacket<D> host.
 * 
 * This convenience class both simplifies the command code but also increase type safety by restricting the host type.
 * 
 * Usage:
 * 
 * myDataPacketAlgo.addCmd(MyData.class, new ADataPacketAlgoCmd<MyReturn, MyData, MyParam>(){
 *     private static final long serialVersionUID = aGeneratedUIDvalue;
 *     
 *     public MyReturn apply(DataPacket<MyData> host, MyParam... params){
 *         // your code here
 *     }
 * }
 * 
 * Note:  In Eclipse, the auto-generation of the implemented methods of this class does not work properly.
 * The concrete apply method below is replicated by the automatic method generator because it doesn't 
 * recognize that the method already exists and is final.  Luckily, a compiler error message gets generated
 * in the attempt to override a final method.   Simply delete the extraneous auto-generated method.
 * 
 * @author Stephen Wong (c) 2010
 *
 * @param <R> The return type
 * @param <D> The data type held by the host
 * @param <P> The input parameter type
 */
public abstract class ADataPacketAlgoCmd<R, D, P> implements IExtVisitorCmd<R, Class<?>, P, ADataPacket>{

	private static final long serialVersionUID = -5627902537609466988L;


	/**
	 * The actual method called by the host visitor when the associated case is invoked.   
	 * This method simply forwards the call to the abstract apply method, performing 
	 * an unchecked cast of the host to the required DataPacket type.
	 * @param index  The Class object used to identify the host
	 * @param host The host calling the visitor
	 * @param params Vararg input parameters to be used for processing the host
	 * @return The result of this case.
	 */
	@SuppressWarnings("unchecked")
	final public <T extends IExtVisitorHost<Class<?>, ? super ADataPacket>> R apply(Class<?> index, T host, P... params) {
		return apply(index, (DataPacket<D>) host, params);
	}

	/**
	 * Abstract method that actually performs the processing of the case.   
	 * Here, the host is strongly typed to be the DataPacket type appropriate for the case (D).
	 * @param index The host ID identifying the host
	 * @param host  The DataPacket host calling the visitor
	 * @param params  Vararg input parameter to be used for processing the host
	 * @return  The result of this case.
	 */
	abstract public R apply(Class<?> index, DataPacket<D> host, P...params);

}

The Java compiler would normally issue an "Unchecked cast" warning at the casting of the host to DataPacket<D> in the upper apply method.   But the invariant execution contract of the Visitor pattern insures us that only hosts of type DataPacket<D> will call the apply method of the command.  The cast is therefore always safe and thus we can safely use the "@SuppressWarnings("unchecked")" compiler directive.

The end result is that we now have a fully type-safe apply method that we will override (implement) to perform our work on data packets.

 

There is one more convenience class that is used to represent a data packet that holds a collection of data packets.  This is useful when transporting aggregated data transmissions, say from multiple sources.   Because Java disallows the use of arrays of generic types as either subtypes or supertypes of other arrays of generic types, creating data packets of arrays of DataPacket<T> is problematic.   So instead, we would like to make a data packet that holds a Vector<ADataPacket> but here, type erasure bites us because Java cannot distinguish between vectors of holding different types at run time.  We would not be able to tell the difference between a data packet that held a vector of data packets from a data packet that held a vector of Strings, for instance.    So, we are forced to create a class that is a subclass of Vector<ADataPacket> that is no longer generic and thus has a well-defined, specific Class object, and hence ID, associated with it:

/**
 * Composite data type for use in data packets: DataPacket<VDataPacket> 
 * Note that Vector<ADataPacket> cannot be used as a data packet data type 
 * directly because type erasure prevents the distinction between 
 * Vector<A> and Vector<B> -- that is, they produce the same Class object.
 * <br/>
 * Usage:
 * 
 *      // make the vector of data packets
 *      VDataPacket vdp = new VDataPacket();
 *      vdp.add(datapacket1);
 *      vdp.add(datapacket2);
 *      vdp.add(datapacket3);
 *      // etc
 *      // make the composite data packet
 *      ADataPacket vd = new DataPacket<VDataPacket>(VDataPacket.class, vdp);
 * 
 * This class is a vector of the abstract data packets, ADataPacket.  If a vector of more specific 
 * types of data packets is desired, a custom class that is a sub-class of the desired
 * Vector type should be used instead of this class.   Composites made as such would be 
 * distinguishable as per their held types.
 * @author Stephen Wong (c) 2010
 *
 */
public class VDataPacket extends Vector<ADataPacket> {

	private static final long serialVersionUID = -860544422905072718L;

	/**
	 * Convenience method that creates a command that maps a visitor over the vector of data packets.
	 * 
	 * Since the result returned by the returned command is a vector of R, 
	 * the returned command must be wrapped in another command
	 * before it can be used in a recursive algorithm, which would require a return of type R.
	 * @param <R>  The return type of the given visitor.
	 * @param <P>  The vararg input parameter type of the original visitor. 
	 * @param algo The visitor to be mapped over all the stored data packets.
	 * @return An ADataPacketAlgoCmd whose results are a vector of results from applying the given visitor to each data packet element.
	 */
	public static <R, P> ADataPacketAlgoCmd<Vector<R>, VDataPacket, P> makeMapCmd(final DataPacketAlgo<R,P> algo) {
		return new ADataPacketAlgoCmd<Vector<R>, VDataPacket, P>(){

			private static final long serialVersionUID = -5855856243603215928L;

			public Vector<R> apply(Class<?> index, DataPacket<VDataPacket> host, P... params) {
				Vector<R> vResult = new Vector<R>();
				for(ADataPacket d: host.getData()){
					vResult.add(d.execute(algo, params));
				}
				return vResult;
			}
		};
	}
}

With the above infrastructure, we are poised to send data packets wherever we want, containing whatever we want and being processed however we want!   And now for the real kicker:

Not only do data packets carry information in the data they contain, but the TYPE of the data that they contain is also information in of itself and in a Visitor pattern scenario, it is the type of the data that will determine the actions that will take place in response to receiving the data packet.   The data is just support information for that type-driven action!

Important implementation note:  The  Class<?> ID object used to define a DataPacket is not necessarily correspond to the exact type, T, of the data being placed into the DataPacket.   That is, the ID object is not always of type Class<T>.  Often one wants the type of the ID to be the supertype of T and thus have the data processed as per the superclass of T, not as a special subclass case.   This is why DataPacket does not automatically infer it's ID value from the data being stored.

(At this point, it is perfectly normal for you to feel like this.)

 


© 2018 by Stephen Wong