COMP 310	Mixed Data Dictionaries and Inter-Module Communications

Mixed Data Dictionaries

The problem with dictionaries is that they are typed to fixed <key, value> types.   One can only put one type of object into a dictionary.    If one wants to put a variety of types of data into the same dictionary, e.g. for configuration information or other common data stores, the superclass of all possible held data types must be used to define the dictionary.    This causes two main problems with type-safety in the application:

1. The retrieved data must be downcast to be used because it comes out with a type that is too abstract to be useful.
2. The application must keep track of which keys are related to which types of data.    Nothing prevents incorrect casting at run-time.

What we'd really like a dictionary that enables us to store mixed types of data safely together.   The problem is that Java's type erasure causes the type information to be lost at runtime.   

The solution is thus to encode the key with the type information of the data to which it refers.   Then, the dictionary can a) restrict any data being stored to only the type or subtype of that the key specifies and b) because of a), can unequivocally retrieve the data with the specific type as defined by the key being used.   

See the documentation for the mixedData package.

While a MixedDataKey is Serializable, it is often more practical to build the key when and where it is used by using the three values needed to specify the key instead of sending a serialized key instance.

Here's a breakdown of the information needed to generate a key. These three values completely and uniquely determine a key, that is, any MixedDataKey instance made with the same three values will be equal.

• UUID id = This value insures that the key will be unique from all other keys with the same two other values. In a typical scenario, all the keys in use share the same id value. For instance, a single randomly-generated UUID value could be used to uniquely identify the instance of the Game Server. The Game Server would then give that common UUID value to all commands it sends out so that they would be tied to that instance of the Game Server. 
• String desc = This is a "human-friendly" name that you can use to identify the particular value in the dictionary to which this key maps, e.g. "MapView adapter". This helps you keep track of what each key is for.  In practice, this is the main part of the key that one uses to differentiate one value from another in the dictionary.
• Class<?> type = The is the Java type of the value to which this key maps in the dictionary. Only values of this type can be associated with this key. While it is technically possible to use the same description above for different value types, it is not recommended because of the semantic confusion it would cause. Technically, this type specification duplicates the generic typing of the MixedDataKey itself but is needed in Java due to type erasure; in run-time generics languages such as C#, this parameter is not needed at all.

Type-erasure Problems when Storing Generic Types in a Mixed-Data Dictionary (MDD)

It turns out that saving generic types to the MDD is problematic due to Java's type-erasure (boo, hiss!).    

In particular, type erasure prevents on from getting the class object of a generic type, e.g. suppose one has a generic class, MyClass<T>, then Java won't let you get the class object for it, e.g. MyClass<SomeClass>.class will not compile.   This is because at run-time, only the raw MyClass exists, not the fully generic type.    Thus, we cannot make the MDD key that we want because the key requires the corresponding Class object for the data being stored.

The type-safe work-around is to create a custom, non-generic subclass for the desired type, e.g.:

class MyCustomClass extends MyClass<SomeClass> {
}

Now, MyCustomClass.class is a perfectly valid operation and creates the Class object we want for the MDD key.

Notes:

• The irony is that we need the Class object at all is because of type-erasure!
• This issue would also come up when trying to make DataPackets of generic types.

Establishing communications between isolated modules using common data storage

A very common problem that arises in enterprise systems is the need for highly decoupled modules to be able to communicate with each other.   These modules tend be tightly sandboxed, with very limited access to their host systems.    Examples of such communications include passing large amounts of data from multiple producers to multiple consumers and enabling arbitrary components in a component-framework system to inter-operate.   Since it is impossible to predict a priori what communications needs any arbitrary modules might need, one must abstract the issue and present a highly abstract interface that can handle any situation.   

In an object-oriented system, entities communicate by calling methods on objects.   Those object may be each other or they might be common shared objects.   In highly decoupled scenarios using MVC-like architectures, decoupled communications occurs by calling methods on adapters whose back-side implementations are hidden from the caller.  

One of the most popular techniques to establish communications between isolated entities in enterprise applications is to use a common data storage to hold shared data or objects.    All that one needs to do is to have one entity place the desired shared data or objects into the common data storage for other entities to retrieve.  Once the shared information is retrieved, communications can be established and/or data will have been transferred.   Cloud systems typically have massive shared databases and in-memory "memcaches" just for this purpose.

In the example below, we will consider the smaller but analogous situation of isolated commands installed into an extended visitor that need to communicate with each other. 

Important note:   It is critical to understand the difference between how the commands for "well-known message" types can interact with the local system vs. how commands associated with "unknown message" types, i.e. command that may have originated from a different system, can interact with the same local system.

• Commands for "well-known" message types:  A message type is "well-known" if the receiver of the message (i.e. the machine whose visitor will process it), requires intimate, internal access to the local system in order to process that message type.   Thus, no outside entity can possibly tell the local system how to process the message and the local system is thus responsible for always providing that capability.    Thus the commands associated with well-known message types fundamentally have very intimate access to the local system and are not necessarily bound by any standardized (e.g. remote API-determined) adapters given to the command.
• Commands for "non-well-known" message types:  These commands may have originated from outside the local system.   They are therefore fundamentally outside of the encapsulation of the local system and are thus sandboxed by the adapter(s) given to them by the local system when they were installed.   The only access non-well-known message type commands have to the local system is through the adapter interface given to them by the local system (ICmd2ModelAdpt, below).   This is analogous to how the adapters in an MVC architecture isolate and sandbox the model and view.  This class of commands CANNOT freely access the local system!   If their adapter does not allow it, it cannot be done.    

Thus, the command's adapter interface (ICmd2ModelAdpt below) must supply methods along the lines of  put(key, object) and get(key) to put and get objects to/from the local data storage.  Note that the adapter does NOT directly expose the actual data storage to the commands in order protect its implementation; it only exposes the necessary functionality to interact with the data storage.

Command-to-command communications can thus easily be accomplished by simply having one command put an object, typically an adapter to micro-models or views, into the local data storage while other commands who desire that particular communications, will subsequently retrieve it.

Mixed Data Dictionaries as common data storage

Common data storage presents two problems with which typical dictionaries have difficulty:

1. Type-safety:  Since the specific types of data that need to be stored in the common data storage are unknown and/or unspecified, a typical dictionary has to be typed to the most general data type possible, typically Object.  But this means that for any specific piece of data being stored in the dictionary, when retrieved, must be downcast to its actual type.   This fundamentally type-unsafe operation has no guarantees of being performed without error.   
2. Multi-tenancy isolation:  The modules saving data in the common storage may represent multiple unrelated sets modules who need to be decoupled from each other.   Think multiple users in a system all saving data to a common storage.   This "multi-tenant" situation is problematic because a typical dictionary requires that all its keys be unique--but if the different tenants of the system are truly decoupled from each other, then no guarantees can be made that they won't accidentally pick the same key for their stored data, thus causing a conflict with potentially disastrous results.   This problem is further compounded by the lack of type safety, meaning that a module could possibly pull out a data item of the wrong type because another unrelated module accidentally used the same key value to store it.

IMixedDataDictionary solves both these problems:

1. Type-safety:  The special MixedDataKey object holds the type information of the stored data, enabling the IMixedDataDictionary's get() operation to always return a strongly and properly typed result.  That is, if one has a mixed data dictionary, mdd, into which one stores an object of type T, a MixedDataKey<T> key must have been used to do the storage of a value of type T, i.e.     mdd.put(key, value).   And when that key is used to retrieve the stored data, the mixed data dictionary automatically returnes it with its proper type, no downcasting is needed:   T myData = mdd.get(key).
2. Multi-tenancy isolation:  To create a MixedDataKey<T> requires 3 items:   A UUID value, a String descriptive value, and a Class<T> object.  The String descriptive value and the Class<T> object are needed to provde a user-friendly key value and the data typing information respectively.    But since another unrelated module may be using the same descriptive value, e.g. "view adapter", and the same class type, e.g. "MapPanel", these two values are insufficient to uniquely identify the key across the system.   The inclusion of the UUID value accomplishes this uniqueness.    A single UUID value is all that is needed across a set of related modules to ensure that their data keys are unique from any keys made by any other unrelated modules in the system, even if they have the same string descriptors and class types.   Note that only a single UUID value only needs to be specified across a set of related modules, there is no need to specify the UUID on a per-key basis because one can assume that a related set of modules is capable of keeping its own set of descriptors and class types from colliding with themselves.

Thus, in the example above, the adapter to the commands, ICmd2ModelAdpt,  only needs to expose the type-safe put and get methods, which will connect to the back-end IMixedDataDictionary:

• <T> T put(MixedDataKey<T> key, T value) -- the return value here on the adapter does not have to echo the return value from the dictionary.
• <T> T get(MixedDataKey<T> key)
• boolean containsKey(MixedDataKey<?> key) -- optional, if this functionality is desired.

With this simple addition to the command's interface to the local system, type-safe and isolated data storage and inter-command communications can be achieved.

For an example of how common data storage can be used to enable command-to-command communications in a distributed application, see ChatApp and Final Proect Communications Pathways.

© 2017 by Stephen Wong