A newer version of IMDG is available.

Want to try Hazelcast Platform?

We’ve combined the in-memory storage of IMDG with the stream processing power of Jet to bring you the all new Hazelcast Platform.

Implementing Portable Serialization

As an alternative to the existing serialization methods, Hazelcast offers a language/platform independent Portable serialization that has the following advantages:

support for multi-version of the same object type
fetching individual fields without having to rely on reflection
queries and indexing support without deserialization and/or reflection

In order to support these features, a serialized Portable object contains meta information like the version and the concrete location of the each field in the binary data. This way, Hazelcast navigates in the byte[] and deserializes only the required field without actually deserializing the whole object. This improves the Query performance.

With multi-version support, you can have two cluster members where each has different versions of the same object. Hazelcast stores both meta information and uses the correct one to serialize and deserialize Portable objects depending on the member. This is very helpful when you are doing a rolling upgrade without shutting down the cluster.

Portable serialization is totally language independent and is used as the binary protocol between Hazelcast server and clients.

Portable Serialization Example Code

Here is example code for Portable implementation of a Foo class.

public class Foo implements Portable {
    final static int ID = 5;

    private String foo;

    public String getFoo() {
        return foo;
    }

    public void setFoo( String foo ) {
        this.foo = foo;
    }

    @Override
    public int getFactoryId() {
        return 1;
    }

    @Override
    public int getClassId() {
        return ID;
    }

    @Override
    public void writePortable( PortableWriter writer ) throws IOException {
        writer.writeUTF( "foo", foo );
    }

    @Override
    public void readPortable( PortableReader reader ) throws IOException {
        foo = reader.readUTF( "foo" );
    }
}

Similar to IdentifiedDataSerializable, a Portable Class must provide classId and factoryId. The Factory object creates the Portable object given the classId.

An example Factory could be implemented as follows:

public class MyPortableFactory implements PortableFactory {

    @Override
    public Portable create( int classId ) {
        if ( Foo.ID == classId )
        return new Foo();
        else
        return null;
    }
}

Registering the Portable Factory

The last step is to register the Factory to the SerializationConfig. Below are the programmatic and declarative configurations for this step.

Config config = new Config();
config.getSerializationConfig().addPortableFactory( 1, new MyPortableFactory() );

XML
YAML

<hazelcast>
    ...
    <serialization>
        <portable-version>0</portable-version>
        <portable-factories>
            <portable-factory factory-id="1">
                com.hazelcast.examples.PortableFactory
            </portable-factory>
        </portable-factories>
    </serialization>
    ...
</hazelcast>

hazelcast:
  serialization:
    portable-version: 0
    portable-factories:
      - factory-id: 1
        class-name: com.hazelcast.examples.PortableFactory

Note that the id that is passed to the SerializationConfig is the same as the factoryId that the Foo class returns.

When using a client/server deployment, you also need to register the implemented factory on the client side. For a Java client, the process is the same as described above to be performed in the client configuration, e.g., hazelcast-client.xml/yaml. For the other Hazelcast clients, see the following for details:

Versioning for Portable Serialization

More than one version of the same class may need to be serialized and deserialized. For example, a client may have an older version of a class and the member to which it is connected may have a newer version of the same class.

Portable serialization supports versioning. It is a global versioning, meaning that all portable classes that are serialized through a member get the globally configured portable version.

You can declare the version in the XML or YAML configuration file using the portable-version element, as shown below.

XML
YAML

<hazelcast>
    ...
    <serialization>
        <portable-version>1</portable-version>
        <portable-factories>
            <portable-factory factory-id="1">
                PortableFactoryImpl
            </portable-factory>
        </portable-factories>
    </serialization>
    ...
</hazelcast>

hazelcast:
  serialization:
    portable-version: 1
    portable-factories:
      - factory-id: 1
        class-name: PortableFactoryImpl

You can also use the interface VersionedPortable which enables to upgrade the version per class, instead of global versioning. If you need to update only one class, you can use this interface. In this case, your class should implement VersionedPortable instead of Portable, and you can give the desired version using the method VersionedPortable.getClassVersion().

You should consider the following when you perform versioning:

It is important to change the version whenever an update is performed in the serialized fields of a class, for example by incrementing the version.
If a client performs a Portable deserialization on a field and then that Portable is updated by removing that field on the cluster side, this may lead to a problem.
Portable serialization does not use reflection and hence, fields in the class and in the serialized content are not automatically mapped. Field renaming is a simpler process. Also, since the class ID is stored, renaming the Portable does not lead to problems.
Types of fields need to be updated carefully. Hazelcast performs basic type upgradings, such as int to float.

Example Portable Versioning Scenarios

Assume that a new member joins to the cluster with a class that has been modified and class' version has been upgraded due to this modification.

If you modified the class by adding a new field, the new member’s put operations include that new field. If this new member tries to get an object that was put from the older members, it gets null for the newly added field.
If you modified the class by removing a field, the old members get null for the objects that are put by the new member.
If you modified the class by changing the type of a field, the error IncompatibleClassChangeError is generated unless the change was made on a built-in type or the byte size of the new type is less than or equal to the old one. The following are examples of allowed type conversions:
- long → int, byte, char, short
- int→ byte, char, short

If you have not modify a class at all, it works as usual.

Ordering Consistency for `writePortable`

Independent of the member-member or member-client communications, the method writePortable() of the classes that implement Portable should be consistent. This means, the fields listed under the method writePortable() should be in the same order for all involved members and/or clients.

Let’s consider the following Employee class:

class EmployeePortable implements Portable {

    private String name;
    private int age;

    public EmployeePortable() {
    }

    public EmployeePortable(int age, String name) {
        this.age = age;
        this.name = name;
    }

    public int getFactoryId() {
        return 666;
    }

    public int getClassId() {
        return 2;
    }

    public void writePortable(PortableWriter writer) throws IOException {
        writer.writeUTF("n", name);
        writer.writeInt("a", age);
    }

    public void readPortable(PortableReader reader) throws IOException {
        name = reader.readUTF("n");
        age = reader.readInt("a");
    }

    public int getAge() {
        return age;
    }
}

As you see in the above example, first the name and then the age is written. This order should be preserved in other members or clients.

Null Portable Serialization

Be careful with serializing null nested portable inside another portable object. Hazelcast lazily creates a class definition of portable internally when the user first serializes. This class definition is stored and used later for deserializing that portable class.

Firstly, writePortable method cannot be called with a null object. writeNullPortable should be used instead as shown below:

public class BaseObject implements Portable {

    private Foo foo;

    @Override
    public int getFactoryId() {
        return FACTORY_ID;
    }

    @Override
    public int getClassId() {
        return BASEOBJECT_CLASS_ID;
    }

    @Override
    public void writePortable(PortableWriter writer) throws IOException {
        if (foo == null) {
            writer.writeNullPortable("foo", FACTORY_ID, FOO_CLASS_ID);
        } else {
            writer.writePortable("foo", foo);
        }
    }

    @Override
    public void readPortable(PortableReader reader) throws IOException {
        foo = reader.readPortable("foo");
    }
}

public static class Foo implements Portable {

    private String bar;

    //....

    @Override
    public int getFactoryId() {
        return FACTORY_ID;
    }

    @Override
    public int getClassId() {
        return FOO_CLASS_ID;
    }

    @Override
    public void writePortable(PortableWriter writer) throws IOException {
        writer.writeUTF("b", bar);
    }

    @Override
    public void readPortable(PortableReader reader) throws IOException {
        bar = reader.readUTF("b");
    }
}

Secondly, when the user tries to serialize a null portable when there is no class definition at the moment, Hazelcast throws a serialization exception saying "Cannot write null portable without explicitly registering class definition!".

There are two solutions to get rid of this exception. Either put a non-null portable class of the same type before any other operation, or manually register a class definition in serialization configuration as shown below.

Config config = new Config();
final ClassDefinition classDefinition = new ClassDefinitionBuilder(FACTORY_ID, FOO_CLASS_ID)
                       .addUTFField("b").build();
config.getSerializationConfig().addClassDefinition(classDefinition);
Hazelcast.newHazelcastInstance(config);

DistributedObject Serialization

Putting a DistributedObject (Hazelcast Semaphore, Queue, etc.) in a cluster member and getting it from another one is not a straightforward operation. Passing the ID and type of the DistributedObject can be a solution. For deserialization, you can get the object from HazelcastInstance. For instance, if your object is an instance of IQueue, you can either use HazelcastInstance.getQueue(id) or Hazelcast.getDistributedObject.

You can use the HazelcastInstanceAware interface in the case of a deserialization of a Portable DistributedObject if it gets an ID to be looked up. HazelcastInstance is set after deserialization, so you first need to store the ID and then retrieve the DistributedObject using the setHazelcastInstance method.

See the Serialization Configuration Wrap-Up section for a full description of Hazelcast Serialization configuration elements.

Edit this page Report an issue