Implementing Portable Serialization
As an alternative to the existing serialization methods, Hazelcast offers a language/platform independent Portable serialization that has the following advantages:
-
support for multi-version of the same object type
-
fetching individual fields without having to rely on reflection
-
queries and indexing support without deserialization and/or reflection
In order to support these features, a serialized Portable object contains
meta information like the version and the concrete location of the each field in the binary data.
This way, Hazelcast navigates in the byte[] and deserializes only
the required field without actually deserializing the whole object.
This improves the Query performance.
With multi-version support, you can have two cluster members where each has different versions of the same object. Hazelcast stores both meta information and uses the correct one to serialize and deserialize Portable objects depending on the member. This is very helpful when you are doing a rolling upgrade without shutting down the cluster.
Portable serialization is totally language independent and is used as the binary protocol between Hazelcast server and clients.
Portable Serialization Example Code
Here is example code for Portable implementation of a Foo class.
public class Foo implements Portable {
final static int ID = 5;
private String foo;
public String getFoo() {
return foo;
}
public void setFoo( String foo ) {
this.foo = foo;
}
@Override
public int getFactoryId() {
return 1;
}
@Override
public int getClassId() {
return ID;
}
@Override
public void writePortable( PortableWriter writer ) throws IOException {
writer.writeString( "foo", foo );
}
@Override
public void readPortable( PortableReader reader ) throws IOException {
foo = reader.readString( "foo" );
}
}Similar to IdentifiedDataSerializable, a Portable Class must provide classId and factoryId.
The Factory object creates the Portable object given the classId.
An example Factory could be implemented as follows:
public class MyPortableFactory implements PortableFactory {
@Override
public Portable create( int classId ) {
if ( Foo.ID == classId )
return new Foo();
else
return null;
}
}Registering the Portable Factory
The last step is to register the Factory to the SerializationConfig.
Below are the programmatic and declarative configurations for this step.
Config config = new Config();
config.getSerializationConfig().addPortableFactory( 1, new MyPortableFactory() );<hazelcast>
...
<serialization>
<portable-version>0</portable-version>
<portable-factories>
<portable-factory factory-id="1">
com.hazelcast.examples.PortableFactory
</portable-factory>
</portable-factories>
</serialization>
...
</hazelcast>hazelcast:
serialization:
portable-version: 0
portable-factories:
- factory-id: 1
class-name: com.hazelcast.examples.PortableFactoryNote that the id that is passed to the SerializationConfig is the same
as the factoryId that the Foo class returns.
When using a client/server deployment, you also need to register the implemented factory on
the client side. For a Java client, the process is the same as described above to be performed
in the client configuration, e.g., hazelcast-client.xml/yaml.
For the other Hazelcast clients, see the following for details:
Versioning for Portable Serialization
More than one version of the same class may need to be serialized and deserialized. For example, a client may have an older version of a class and the member to which it is connected may have a newer version of the same class.
Portable serialization supports versioning. It is a global versioning, meaning that all portable classes that are serialized through a member get the globally configured portable version.
You can declare the version in the XML or YAML configuration file using
the portable-version element, as shown below.
<hazelcast>
...
<serialization>
<portable-version>1</portable-version>
<portable-factories>
<portable-factory factory-id="1">
PortableFactoryImpl
</portable-factory>
</portable-factories>
</serialization>
...
</hazelcast>hazelcast:
serialization:
portable-version: 1
portable-factories:
- factory-id: 1
class-name: PortableFactoryImplYou can also use the interface
VersionedPortable
which enables to upgrade the version per class, instead of global versioning.
If you need to update only one class, you can use this interface.
In this case, your class should implement VersionedPortable instead of Portable,
and you can give the desired version using the method VersionedPortable.getClassVersion().
You should consider the following when you perform versioning:
-
It is important to change the version whenever an update is performed in the serialized fields of a class, for example by incrementing the version.
-
If a client performs a Portable deserialization on a field and then that Portable is updated by removing that field on the cluster side, this may lead to a problem.
-
Portable serialization does not use reflection and hence, fields in the class and in the serialized content are not automatically mapped. Field renaming is a simpler process. Also, since the class ID is stored, renaming the Portable does not lead to problems.
-
Types of fields need to be updated carefully. Hazelcast performs basic type upgradings, such as
inttofloat.
Example Portable Versioning Scenarios
Assume that a new member joins to the cluster with a class that has been modified and class' version has been upgraded due to this modification.
-
If you modified the class by adding a new field, the new member’s
putoperations include that new field. If this new member tries to get an object that was put from the older members, it getsnullfor the newly added field. -
If you modified the class by removing a field, the old members get
nullfor the objects that are put by the new member. -
If you modified the class by changing the type of a field, the error
IncompatibleClassChangeErroris generated unless the change was made on a built-in type or the byte size of the new type is less than or equal to the old one. The following are examples of allowed type conversions:-
long→int,byte,char,short -
int→byte,char,short
-
If you have not modify a class at all, it works as usual.
Ordering Consistency for writePortable
Independent of the member-member or member-client communications, the method writePortable() of
the classes that implement Portable should be consistent.
This means, the fields listed under the method writePortable() should be
in the same order for all involved members and/or clients.
Let’s consider the following Employee class:
class EmployeePortable implements Portable {
private String name;
private int age;
public EmployeePortable() {
}
public EmployeePortable(int age, String name) {
this.age = age;
this.name = name;
}
public int getFactoryId() {
return 666;
}
public int getClassId() {
return 2;
}
public void writePortable(PortableWriter writer) throws IOException {
writer.writeString("n", name);
writer.writeInt("a", age);
}
public void readPortable(PortableReader reader) throws IOException {
name = reader.readString("n");
age = reader.readInt("a");
}
public int getAge() {
return age;
}
}As you see in the above example, first the name and then the age is written.
This order should be preserved in other members or clients.
Null Portable Serialization
Be careful with serializing null nested portable inside another portable object. Hazelcast lazily creates a class definition of portable internally when you use this serialization for the first time. This class definition is stored and used later for deserializing that portable class.
Firstly, you cannot call the writePortable method with a null object.
You should use writeNullPortable instead, as shown below:
public class BaseObject implements Portable {
private Foo foo;
@Override
public int getFactoryId() {
return FACTORY_ID;
}
@Override
public int getClassId() {
return BASEOBJECT_CLASS_ID;
}
@Override
public void writePortable(PortableWriter writer) throws IOException {
if (foo == null) {
writer.writeNullPortable("foo", FACTORY_ID, FOO_CLASS_ID);
} else {
writer.writePortable("foo", foo);
}
}
@Override
public void readPortable(PortableReader reader) throws IOException {
foo = reader.readPortable("foo");
}
}
public static class Foo implements Portable {
private String bar;
//....
@Override
public int getFactoryId() {
return FACTORY_ID;
}
@Override
public int getClassId() {
return FOO_CLASS_ID;
}
@Override
public void writePortable(PortableWriter writer) throws IOException {
writer.writeString("b", bar);
}
@Override
public void readPortable(PortableReader reader) throws IOException {
bar = reader.readString("b");
}
}Secondly, when you try to serialize a null portable when there is no class definition at the moment, Hazelcast throws a serialization exception saying "Cannot write null portable without explicitly registering class definition!".
There are two solutions to get rid of this exception. You can either put a non-null portable class of the same type before any other operation or manually register a class definition in the serialization configuration as shown below:
Config config = new Config();
final ClassDefinition classDefinition = new ClassDefinitionBuilder(FACTORY_ID, FOO_CLASS_ID)
.addStringField("b").build();
config.getSerializationConfig().addClassDefinition(classDefinition);
Hazelcast.newHazelcastInstance(config);DistributedObject Serialization
Putting a DistributedObject (Hazelcast Semaphore, Queue, etc.) in a cluster member and
getting it from another one is not a straightforward operation.
Passing the ID and type of the DistributedObject can be a solution.
For deserialization, you can get the object from HazelcastInstance.
For instance, if your object is an instance of IQueue, you can either use
HazelcastInstance.getQueue(id) or Hazelcast.getDistributedObject.
You can use the HazelcastInstanceAware interface in the case of
a deserialization of a Portable DistributedObject if it gets an ID to be looked up.
HazelcastInstance is set after deserialization, so you first need to store the ID and
then retrieve the DistributedObject using the setHazelcastInstance method.
| See the Serialization Configuration Wrap-Up section for a full description of Hazelcast Serialization configuration elements. |
