Other Network Configurations
All network related configurations are performed via the network element
in the Hazelcast XML configuration file or the class NetworkConfig
when using programmatic configuration. Following subsections describe the
available configurations that you can perform under the network element.
Public Address
public-address overrides the public address of a member. By default, a member
selects its socket address as its public address. But behind a network address translation (NAT),
two endpoints (members) may not be able to see/access each other.
If both members set their public addresses to their defined addresses on NAT,
then that way they can communicate with each other. In this case, their public addresses
are not an address of a local network interface but a virtual address defined by NAT.
It is optional to set and useful when you have a private cloud.
Note that, the value for this element should be given in the format host IP address:port number.
See the following examples.
Declarative Configuration:
<hazelcast>
...
<network>
<public-address>11.22.33.44:5555</public-address>
</network>
...
</hazelcast>hazelcast:
network:
public-address: 11.22.33.44:5555Programmatic Configuration:
Config config = new Config();
config.getNetworkConfig()
.setPublicAddress( "11.22.33.44:5555" );Port
You can specify the ports that Hazelcast uses to communicate between cluster members.
Its default value is 5701. The following are example configurations.
Declarative Configuration:
<hazelcast>
...
<network>
<port port-count="20" auto-increment="true">5701</port>
</network>
...
</hazelcast>hazelcast:
network:
port:
auto-increment: true
port-count: 20
port: 5701Programmatic Configuration:
Config config = new Config();
config.getNetworkConfig().setPort( 5701 )
.setPortAutoIncrement( true ).setPortCount( 20 );According to the above example, Hazelcast tries to find free ports between 5701 and 5720.
port has the following attributes.
-
port-count: By default, Hazelcast tries 100 ports to bind. Meaning that, if you set the value of port as 5701, as members are joining to the cluster, Hazelcast tries to find ports between 5701 and 5801. You can choose to change the port count in the cases like having large instances on a single machine or willing to have only a few ports to be assigned. The parameterport-countis used for this purpose, whose default value is 100. -
auto-increment: In some cases you may want to choose to use only one port. In that case, you can disable the auto-increment feature ofportby settingauto-incrementtofalse. Theport-countattribute is not used when auto-increment feature is disabled.
Outbound Ports
By default, Hazelcast lets the system pick up an ephemeral port during socket bind operation. But security policies/firewalls may require you to restrict outbound ports to be used by Hazelcast-enabled applications. To fulfill this requirement, you can configure Hazelcast to use only defined outbound ports. The following are example configurations.
Declarative Configuration:
<hazelcast>
...
<network>
<outbound-ports>
<!-- ports between 33000 and 35000 -->
<ports>33000-35000</ports>
<!-- comma separated ports -->
<ports>37000,37001,37002,37003</ports>
<ports>38000,38500-38600</ports>
</outbound-ports>
</network>
...
</hazelcast>hazelcast:
network:
outbound-ports:
- 33000-35000
- 37000,37001,37002,37003
- 38000,38500-38600Programmatic Configuration:
...
NetworkConfig networkConfig = config.getNetworkConfig();
// ports between 35000 and 35100
networkConfig.addOutboundPortDefinition("35000-35100");
// comma separated ports
networkConfig.addOutboundPortDefinition("36001, 36002, 36003");
networkConfig.addOutboundPort(37000);
networkConfig.addOutboundPort(37001);
...| You can use port ranges and/or comma separated ports. |
As shown in the programmatic configuration, you use the method addOutboundPort
to add only one port. If you need to add a group of ports, then use the method addOutboundPortDefinition.
In the declarative configuration, the element ports can be used for both single and
multiple port definitions. When you set this element to 0 or *,
your operating system (not Hazelcast) selects a free port from the ephemeral range.
Reuse Address
When you shutdown a cluster member, the server socket port goes into the
TIME_WAIT state for the next couple of minutes. If you start the member right after
shutting it down, you may not be able to bind it to the same port because it is in the
TIME_WAIT state. If you set the reuse-address element to true, the TIME_WAIT state
is ignored and you can bind the member to the same port again.
The following are example configurations.
Declarative Configuration:
<hazelcast>
...
<network>
<reuse-address>true</reuse-address>
</network>
...
</hazelcast>hazelcast:
network:
reuse-address: trueProgrammatic Configuration:
...
NetworkConfig networkConfig = config.getNetworkConfig();
networkConfig.setReuseAddress( true );
...Join
The join configuration element is used to discover Hazelcast members and enable them
to form a cluster. Hazelcast provides Multicast, TCP/IP, AWS, Kubernetes, Azure, GCP, Eureka, and more.
These mechanisms are explained the Discovery Mechanisms section.
This section describes all the sub-elements and attributes of join element.
The following are example configurations.
Declarative Configuration:
<hazelcast>
...
<network>
<join>
<multicast enabled="false">
<multicast-group>224.2.2.3</multicast-group>
<multicast-port>54327</multicast-port>
<multicast-time-to-live>32</multicast-time-to-live>
<multicast-timeout-seconds>2</multicast-timeout-seconds>
<trusted-interfaces>
<interface>192.168.1.102</interface>
</trusted-interfaces>
</multicast>
<tcp-ip enabled="false">
<required-member>192.168.1.104</required-member>
<member>192.168.1.104</member>
<members>192.168.1.105,192.168.1.106</members>
</tcp-ip>
<aws enabled="false">
<access-key>my-access-key</access-key>
<secret-key>my-secret-key</secret-key>
<region>us-west-1</region>
<host-header>ec2.amazonaws.com</host-header>
<security-group-name>hazelcast-sg</security-group-name>
<tag-key>type</tag-key>
<tag-value>hz-members</tag-value>
</aws>
<discovery-strategies>
<discovery-strategy ... />
</discovery-strategies>
</join>
</network>
...
</hazelcast>hazelcast:
network:
join:
multicast:
enabled: false
multicast-group: 224.2.2.3
multicast-port: 54327
multicast-time-to-live: 32
multicast-timeout-seconds: 2
trusted-interfaces:
- 192.168.1.102
tcp-ip:
enabled: false
required-member: 192.168.1.104
member-list:
- 192.168.1.104
- 192.168.1.105,192.168.1.106
aws:
enabled: false
access-key: my-access-key
secret-key: my-secret-key
region: us-west-1
host-header: ec2.amazonaws.com
security-group-name: hazelcast-sg
tag-key: type
tag-value: hz-nodes
discovery-strategies:
discovery-strategy:
...Programmatic Configuration:
Config config = new Config();
NetworkConfig network = config.getNetworkConfig();
JoinConfig join = network.getJoin();
join.getTcpIpConfig().addMember( "10.45.67.32" ).addMember( "10.45.67.100" )
.setRequiredMember( "192.168.10.100" ).setEnabled( true );The join element has the following sub-elements and attributes.
multicast element
The multicast element includes parameters to fine tune the multicast join mechanism.
-
enabled: Specifies whether the multicast discovery is enabled or not,trueorfalse. -
multicast-group: The multicast group IP address. Specify it when you want to create clusters within the same network. Values can be between 224.0.0.0 and 239.255.255.255. Its default value is 224.2.2.3. -
multicast-port: The multicast socket port that the Hazelcast member listens to and sends discovery messages through. Its default value is 54327. -
multicast-time-to-live: Time-to-live value for multicast packets sent out to control the scope of multicasts. See more information, see The Linux Documentation Project. -
multicast-timeout-seconds: Only when the members are starting up, this timeout (in seconds) specifies the period during which a member waits for a multicast response from another member. For example, if you set it as 60 seconds, each member waits for 60 seconds until a leader member is selected. Its default value is 2 seconds. -
trusted-interfaces: Includes IP addresses of trusted members. When a member wants to join to the cluster, its join request is rejected if it is not a trusted member. You can give an IP addresses range using the wildcard (*) on the last digit of IP address, e.g., 192.168.1.* or 192.168.1.100-110.
| If you prefer to use the multicast mechanism, make sure that your network is enclosed and secure. See the Multicast section. |
tcp-ip element
The tcp-ip element includes parameters to fine tune the TCP/IP join mechanism.
-
enabled: Specifies whether the TCP/IP discovery is enabled or not. Values can betrueorfalse. -
required-member: IP address of the required member. Cluster is only formed if the member with this IP address is found. -
member: IP address(es) of one or more well known members. Once members are connected to these well known ones, all member addresses are communicated with each other. You can also give comma separated IP addresses using thememberselement.tcp-ipelement also accepts theinterfaceparameter. See the Interfaces element description. -
connection-timeout-seconds: Defines the connection timeout in seconds. This is the maximum amount of time Hazelcast is going to try to connect to a well known member before giving up. Setting it to a too low value could mean that a member is not able to connect to a cluster. Setting it to a too high value means that member startup could slow down because of longer timeouts, for example when a well known member is not up. Increasing this value is recommended if you have many IPs listed and the members cannot properly build up the cluster. Its default value is 5 seconds.
aws element
The aws element includes parameters to allow the members to form a cluster
on the Amazon EC2 and ECS environments.
For details, please check the Hazelcast AWS Discovery plugin documentation.
azure element
The azure element includes parameters to allow the members to form a cluster on the Azure VM machines.
For details, please check the Hazelcast Azure Discovery plugin documentation.
gcp element
The gcp element includes parameters to allow the members to form a cluster on the GCP Compute VM instances.
For details, please check the Hazelcast GCP Discovery plugin documentation.
kubernetes element
The kubernetes element includes parameters to allow the members to form a cluster on the Kubernetes environment.
For details, please check the Hazelcast Kubernetes Discovery plugin documentation.
discovery-strategies element
The discovery-strategies element configures internal or external discovery
strategies based on the Hazelcast Discovery SPI. For further information, see
the Discovery SPI section and the vendor documentation of
the used discovery strategy.
Interfaces
You can specify which network interfaces that Hazelcast should use.
Servers mostly have more than one network interface, so you may want to list
the valid IPs. Range characters "*" and "-" can be used for simplicity.
For instance, 10.3.10.* refers to IPs between 10.3.10.0 and 10.3.10.255.
Interface 10.3.10.4-18 refers to IPs between 10.3.10.4 and 10.3.10.18
(4 and 18 included). If network interface configuration is enabled
(it is disabled by default) and if Hazelcast cannot find a matching interface,
then it prints a message on the console and does not start on that member.
The following are example configurations.
Declarative Configuration:
<hazelcast>
...
<network>
<interfaces enabled="true">
<interface>10.3.16.*</interface>
<interface>10.3.10.4-18</interface>
<interface>192.168.1.3</interface>
</interfaces>
</network>
...
</hazelcast>hazelcast:
network:
interfaces:
enabled: true
interfaces:
- 10.3.16.*
- 10.3.10.4-18
- 192.168.1.3Programmatic Configuration:
Config config = new Config();
NetworkConfig network = config.getNetworkConfig();
InterfacesConfig interfaceConfig = network.getInterfaces();
interfaceConfig.setEnabled( true )
.addInterface( "192.168.1.3" );IPv6 Support
Hazelcast supports IPv6 addresses seamlessly (This support is switched off by default, see the note at the end of this section).
All you need is to define IPv6 addresses or interfaces in the network
configuration. The only current limitation is that you cannot define
wildcard IPv6 addresses in the TCP/IP join configuration (tcp-ip element).
Interfaces configuration does not have this limitation,
you can configure wildcard IPv6 interfaces in the same way as IPv4 interfaces.
<hazelcast>
...
<network>
<port auto-increment="true">5701</port>
<join>
<multicast enabled="false">
<multicast-group>FF02:0:0:0:0:0:0:1</multicast-group>
<multicast-port>54327</multicast-port>
</multicast>
<tcp-ip enabled="true">
<member>[fe80::223:6cff:fe93:7c7e]:5701</member>
<interface>192.168.1.0-7</interface>
<interface>192.168.1.*</interface>
<interface>fe80:0:0:0:45c5:47ee:fe15:493a</interface>
</tcp-ip>
</join>
<interfaces enabled="true">
<interface>10.3.16.*</interface>
<interface>10.3.10.4-18</interface>
<interface>fe80:0:0:0:45c5:47ee:fe15:*</interface>
<interface>fe80::223:6cff:fe93:0-5555</interface>
</interfaces>
</network>
...
</hazelcast>hazelcast:
network:
port:
auto-increment: true
port: 5701
join:
multicast:
enabled: false
multicast-group: FF02:0:0:0:0:0:0:1
multicast-port: 54327
tcp-ip:
enabled: true
member: [fe80::223:6cff:fe93:7c7e]:5701
interface: 192.168.1.0-7
interface: 192.168.1.*
interface: fe80:0:0:0:45c5:47ee:fe15:493a
interfaces:
enabled: true
interfaces:
- 10.3.16.*
- 10.3.10.4-18
- fe80:0:0:0:45c5:47ee:fe15:*
- fe80::223:6cff:fe93:0-5555JVM has two system properties for setting the preferred protocol stack
(IPv4 or IPv6) as well as the preferred address family types (inet4 or inet6).
On a dual stack machine, IPv6 stack is preferred by default, you can change this
through the java.net.preferIPv4Stack=<true|false> system property. When querying
name services, JVM prefers IPv4 addresses over IPv6 addresses and returns an IPv4
address if possible. You can change this through java.net.preferIPv6Addresses=<true|false>
system property.
See also additional details on IPv6 support in Java.
IPv6 support has been switched off by default, since some platforms have issues
using the IPv6 stack. Some other platforms such as Amazon AWS have no support at all.
To enable IPv6 support, just set configuration property hazelcast.prefer.ipv4.stack
to false. See the System Properties appendix for details.
|
Member Address Provider SPI
| This SPI is not intended to provide addresses of other cluster members with which the Hazelcast instance forms a cluster. To do that, see the previous sections above. |
By default, Hazelcast chooses the public and bind address. You can influence on the
choice by defining a public-address in the configuration or by using other
properties mentioned above. In some cases, though, these properties are not
enough and the default address picking strategy chooses wrong addresses.
This may be the case when deploying Hazelcast in some cloud environments,
such as AWS, when using Docker or when the instance is deployed behind a NAT
and the public-address property is not enough (see the Public Address section).
In these cases, it is possible to configure the bind and public address
in a more advanced way. You can provide an implementation of the
com.hazelcast.spi.MemberAddressProvider interface which provides
the bind and public address. The implementation may then choose these
addresses in any way - it may read from a system property or file or
even invoke a web service to retrieve the public and private address.
The details of the implementation depend heavily on the environment in which Hazelcast is deployed. As such, we now demonstrate how to configure Hazelcast to use a simplified custom member address provider SPI implementation. An example implementation is shown below:
public static final class SimpleMemberAddressProvider implements MemberAddressProvider {
@Override
public InetSocketAddress getBindAddress() {
// determine the address using some configuration, calling an API, ...
return new InetSocketAddress(hostname, port);
}
@Override
public InetSocketAddress getPublicAddress() {
// determine the address using some configuration, calling an API, ...
return new InetSocketAddress(hostname, port);
}
}Note that if the bind address port is 0 then it uses a port as configured
in the Hazelcast network configuration (see the Port section).
If the public address port is set to 0 then it broadcasts the same port that
it is bound to. If you wish to bind to any local interface, you may return
new InetSocketAddress((InetAddress) null, port) from the getBindAddress() address.
The following configuration examples contain properties that are provided to the
constructor of the provider class. If you do not provide any properties, the class
may have either a no-arg constructor or a constructor accepting a single
java.util.Properties instance. On the other hand, if you do provide properties
in the configuration, the class must have a constructor accepting a single
java.util.Properties instance.
Declarative Configuration:
<hazelcast>
...
<network>
<member-address-provider enabled="true">
<class-name>SimpleMemberAddressProvider</class-name>
<properties>
<property name="prop1">prop1-value</property>
<property name="prop2">prop2-value</property>
</properties>
</member-address-provider>
<!-- other network configurations -->
</network>
...
</hazelcast>hazelcast:
network:
member-address-provider:
enabled: true
class-name: SimpleMemberAddressProvider
properties:
prop1: prop1-value
prop2: prop2-value
...Programmatic Configuration:
Config config = new Config();
MemberAddressProviderConfig memberAddressProviderConfig = config.getNetworkConfig().getMemberAddressProviderConfig();
memberAddressProviderConfig
.setEnabled(true)
.setClassName(MemberAddressProviderWithStaticProperties.class.getName());
Properties properties = memberAddressProviderConfig.getProperties();
properties.setProperty("prop1", "prop1-value");
properties.setProperty("prop2", "prop2-value");
config.getNetworkConfig().getJoin().getAutoDetectionConfig().setEnabled(false);
// perform other configuration
Hazelcast.newHazelcastInstance(config);