[Bug] [seatunnel engine zeta] Persistence Configuration of IMap

[lwy205]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

I add a config in hazelcast-master.yaml jus like version 2.3.12 ,but it dose not work and get an error while start seatunnel master service

SeaTunnel Version

2.3.13

SeaTunnel Config

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

hazelcast:
  cluster-name: seatunnel
  network:
    rest-api:
      enabled: false
      endpoint-groups:
        CLUSTER_WRITE:
          enabled: true
        DATA:
          enabled: true
    join:
      tcp-ip:
        enabled: true
        member-list:
          - master1:5801
          - master2:5801
          - node4:5802
          - node5:5802
#          - localhost:5801
#          - localhost:5802
    port:
      auto-increment: false
      port: 5801
  properties:
    hazelcast.invocation.max.retry.count: 20
    hazelcast.tcp.join.port.try.count: 30
    hazelcast.logging.type: log4j2
    hazelcast.operation.generic.thread.count: 50
    hazelcast.heartbeat.failuredetector.type: phi-accrual
    hazelcast.heartbeat.interval.seconds: 2
    hazelcast.max.no.heartbeat.seconds: 180
    hazelcast.heartbeat.phiaccrual.failuredetector.threshold: 10
    hazelcast.heartbeat.phiaccrual.failuredetector.sample.size: 200
    hazelcast.heartbeat.phiaccrual.failuredetector.min.std.dev.millis: 100

[root@master1 config]# vim hazelcast-master.yaml
[root@master1 config]# cat hazelcast-master.yaml
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

hazelcast:
  cluster-name: seatunnel
  network:
    rest-api:
      enabled: false
      endpoint-groups:
        CLUSTER_WRITE:
          enabled: true
        DATA:
          enabled: true
    join:
      tcp-ip:
        enabled: true
        member-list:
          - master1:5801
          - master2:5801
          - node4:5802
          - node5:5802
#          - localhost:5801
#          - localhost:5802
    port:
      auto-increment: false
      port: 5801
  properties:
    hazelcast.invocation.max.retry.count: 20
    hazelcast.tcp.join.port.try.count: 30
    hazelcast.logging.type: log4j2
    hazelcast.operation.generic.thread.count: 50
    hazelcast.heartbeat.failuredetector.type: phi-accrual
    hazelcast.heartbeat.interval.seconds: 2
    hazelcast.max.no.heartbeat.seconds: 180
    hazelcast.heartbeat.phiaccrual.failuredetector.threshold: 10
    hazelcast.heartbeat.phiaccrual.failuredetector.sample.size: 200
    hazelcast.heartbeat.phiaccrual.failuredetector.min.std.dev.millis: 100

  map:
      engine*:
         map-store:
           enabled: true
           initial-mode: EAGER
           factory-class-name: org.apache.seatunnel.engine.server.persistence.FileMapStoreFactory
           properties:
             type: hdfs
             namespace: /seatunnel/imap/
             clusterName: seatunnel
             storage.type: hdfs
             fs.defaultFS: hdfs://newcluster
             seatunnel.hadoop.dfs.nameservices: newcluster
             seatunnel.hadoop.dfs.ha.namenodes.newcluster: master1,master2
             seatunnel.hadoop.dfs.namenode.rpc-address.newcluster.master1: master1:8020
             seatunnel.hadoop.dfs.namenode.rpc-address.newcluster.master2: master2:8020
             seatunnel.hadoop.dfs.client.failover.proxy.provider.newcluster: org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider

Running Command

./bin/seatunnel-cluster.sh -d -r master

Error Exception

2026-03-24 20:06:33,209 WARN  [c.h.c.CPSubsystem             ] [main] - [master1]:5801 [seatunnel] [5.1] CP Subsystem is not enabled. CP data structures will operate in UNSAFE mode! Please note that UNSAFE mode will not provide strong consistency guarantees.
2026-03-24 20:06:33,376 INFO  [.c.c.DefaultClassLoaderService] [main] - start classloader service with cache mode
2026-03-24 20:06:33,384 INFO  [o.a.s.e.s.CoordinatorService  ] [main] - [master1]:5801 [seatunnel] [5.1] Start pending job schedule thread
2026-03-24 20:06:33,776 WARN  [o.a.h.u.NativeCodeLoader      ] [main] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2026-03-24 20:06:34,301 WARN  [c.h.s.i.p.ProxyService        ] [main] - [master1]:5801 [seatunnel] [5.1] Error while initializing proxy: IMap{name='engine_checkpoint_monitor'}
java.lang.NullPointerException: null
        at com.hazelcast.spi.impl.operationservice.impl.Invocation.<init>(Invocation.java:209) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.operationservice.impl.PartitionInvocation.<init>(PartitionInvocation.java:51) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.operationservice.impl.PartitionInvocation.<init>(PartitionInvocation.java:62) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.operationservice.impl.OperationServiceImpl.invokeOnPartition(OperationServiceImpl.java:328) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.map.impl.proxy.MapProxySupport.waitUntilLoaded(MapProxySupport.java:760) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.map.impl.proxy.MapProxyImpl.waitUntilLoaded(MapProxyImpl.java:112) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.map.impl.proxy.MapProxySupport.initializeMapStoreLoad(MapProxySupport.java:330) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.map.impl.proxy.MapProxySupport.initialize(MapProxySupport.java:265) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.map.impl.proxy.MapProxyImpl.initialize(MapProxyImpl.java:112) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.proxyservice.impl.ProxyRegistry.doCreateProxy(ProxyRegistry.java:266) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.proxyservice.impl.ProxyRegistry.createProxy(ProxyRegistry.java:235) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.proxyservice.impl.ProxyRegistry.getOrCreateProxyFuture(ProxyRegistry.java:202) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.proxyservice.impl.ProxyRegistry.getOrCreateProxy(ProxyRegistry.java:182) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.proxyservice.impl.ProxyServiceImpl.getDistributedObject(ProxyServiceImpl.java:153) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.instance.impl.HazelcastInstanceImpl.getDistributedObject(HazelcastInstanceImpl.java:355) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.instance.impl.HazelcastInstanceImpl.getMap(HazelcastInstanceImpl.java:174) ~[seatunnel-starter.jar:2.3.13]
        at org.apache.seatunnel.engine.server.checkpoint.monitor.CheckpointMonitorService.<init>(CheckpointMonitorService.java:55) ~[seatunnel-starter.jar:2.3.13]
        at org.apache.seatunnel.engine.server.SeaTunnelServer.startMaster(SeaTunnelServer.java:191) ~[seatunnel-starter.jar:2.3.13]
        at org.apache.seatunnel.engine.server.SeaTunnelServer.init(SeaTunnelServer.java:159) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.servicemanager.impl.ServiceManagerImpl.initService(ServiceManagerImpl.java:235) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.servicemanager.impl.ServiceManagerImpl.initServices(ServiceManagerImpl.java:211) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.servicemanager.impl.ServiceManagerImpl.start(ServiceManagerImpl.java:103) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.NodeEngineImpl.start(NodeEngineImpl.java:250) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.instance.impl.Node.start(Node.java:458) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.instance.impl.HazelcastInstanceImpl.<init>(HazelcastInstanceImpl.java:124) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.instance.impl.HazelcastInstanceFactory.constructHazelcastInstance(HazelcastInstanceFactory.java:217) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.instance.impl.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:196) ~[seatunnel-starter.jar:2.3.13]
        at org.apache.seatunnel.engine.server.SeaTunnelServerStarter.initializeHazelcastInstance(SeaTunnelServerStarter.java:70) ~[seatunnel-starter.jar:2.3.13]
        at org.apache.seatunnel.engine.server.SeaTunnelServerStarter.createHazelcastInstance(SeaTunnelServerStarter.java:52) ~[seatunnel-starter.jar:2.3.13]
        at org.apache.seatunnel.core.starter.seatunnel.command.ServerExecuteCommand.execute(ServerExecuteCommand.java:83) ~[seatunnel-starter.jar:2.3.13]
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) ~[seatunnel-starter.jar:2.3.13]
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelServer.main(SeaTunnelServer.java:34) ~[seatunnel-starter.jar:2.3.13]
2026-03-24 20:06:34,301 ERROR [c.h.s.i.s.i.ServiceManagerImpl] [main] - [master1]:5801 [seatunnel] [5.1] Error while initializing service: null
java.lang.NullPointerException: null
        at com.hazelcast.spi.impl.operationservice.impl.Invocation.<init>(Invocation.java:209) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.operationservice.impl.PartitionInvocation.<init>(PartitionInvocation.java:51) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.operationservice.impl.PartitionInvocation.<init>(PartitionInvocation.java:62) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.operationservice.impl.OperationServiceImpl.invokeOnPartition(OperationServiceImpl.java:328) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.map.impl.proxy.MapProxySupport.waitUntilLoaded(MapProxySupport.java:760) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.map.impl.proxy.MapProxyImpl.waitUntilLoaded(MapProxyImpl.java:112) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.map.impl.proxy.MapProxySupport.initializeMapStoreLoad(MapProxySupport.java:330) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.map.impl.proxy.MapProxySupport.initialize(MapProxySupport.java:265) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.map.impl.proxy.MapProxyImpl.initialize(MapProxyImpl.java:112) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.proxyservice.impl.ProxyRegistry.doCreateProxy(ProxyRegistry.java:266) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.proxyservice.impl.ProxyRegistry.createProxy(ProxyRegistry.java:235) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.proxyservice.impl.ProxyRegistry.getOrCreateProxyFuture(ProxyRegistry.java:202) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.proxyservice.impl.ProxyRegistry.getOrCreateProxy(ProxyRegistry.java:182) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.proxyservice.impl.ProxyServiceImpl.getDistributedObject(ProxyServiceImpl.java:153) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.instance.impl.HazelcastInstanceImpl.getDistributedObject(HazelcastInstanceImpl.java:355) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.instance.impl.HazelcastInstanceImpl.getMap(HazelcastInstanceImpl.java:174) ~[seatunnel-starter.jar:2.3.13]
        at org.apache.seatunnel.engine.server.checkpoint.monitor.CheckpointMonitorService.<init>(CheckpointMonitorService.java:55) ~[seatunnel-starter.jar:2.3.13]
        at org.apache.seatunnel.engine.server.SeaTunnelServer.startMaster(SeaTunnelServer.java:191) ~[seatunnel-starter.jar:2.3.13]
        at org.apache.seatunnel.engine.server.SeaTunnelServer.init(SeaTunnelServer.java:159) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.servicemanager.impl.ServiceManagerImpl.initService(ServiceManagerImpl.java:235) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.servicemanager.impl.ServiceManagerImpl.initServices(ServiceManagerImpl.java:211) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.servicemanager.impl.ServiceManagerImpl.start(ServiceManagerImpl.java:103) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.spi.impl.NodeEngineImpl.start(NodeEngineImpl.java:250) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.instance.impl.Node.start(Node.java:458) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.instance.impl.HazelcastInstanceImpl.<init>(HazelcastInstanceImpl.java:124) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.instance.impl.HazelcastInstanceFactory.constructHazelcastInstance(HazelcastInstanceFactory.java:217) ~[seatunnel-starter.jar:2.3.13]
        at com.hazelcast.instance.impl.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:196) ~[seatunnel-starter.jar:2.3.13]
        at org.apache.seatunnel.engine.server.SeaTunnelServerStarter.initializeHazelcastInstance(SeaTunnelServerStarter.java:70) ~[seatunnel-starter.jar:2.3.13]
        at org.apache.seatunnel.engine.server.SeaTunnelServerStarter.createHazelcastInstance(SeaTunnelServerStarter.java:52) ~[seatunnel-starter.jar:2.3.13]
        at org.apache.seatunnel.core.starter.seatunnel.command.ServerExecuteCommand.execute(ServerExecuteCommand.java:83) ~[seatunnel-starter.jar:2.3.13]
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) ~[seatunnel-starter.jar:2.3.13]
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelServer.main(SeaTunnelServer.java:34) ~[seatunnel-starter.jar:2.3.13]
2026-03-24 20:06:34,339 INFO  [c.h.i.d.Diagnostics           ] [main] - [master1]:5801 [seatunnel] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2026-03-24 20:06:34,344 INFO  [c.h.c.LifecycleService        ] [main] - [master1]:5801 [seatunnel] [5.1] [master1]:5801 is STARTING
2026-03-24 20:06:34,385 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-3] - [master1]:5801 [seatunnel] [5.1] [master2]:5801 is added to the blacklist.
2026-03-24 20:06:34,385 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-1] - [master1]:5801 [seatunnel] [5.1] [node5]:5802 is added to the blacklist.
2026-03-24 20:06:34,385 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-2] - [master1]:5801 [seatunnel] [5.1] [node4]:5802 is added to the blacklist.
2026-03-24 20:06:35,375 INFO  [c.h.i.c.ClusterService        ] [main] - [master1]:5801 [seatunnel] [5.1] 

Members {size:1, ver:1} [
        Member [master1]:5801 - 71303e1f-1974-4916-a767-71e8fa586f57 [master node] [active master] this
]

2026-03-24 20:06:35,384 INFO  [o.a.s.e.s.CoordinatorService  ] [pool-4-thread-1] - [master1]:5801 [seatunnel] [5.1] This node become a new active master node, begin init coordinator service
2026-03-24 20:06:35,390 INFO  [.h.i.p.i.PartitionStateManager] [pool-4-thread-1] - [master1]:5801 [seatunnel] [5.1] Initializing cluster partition table arrangement...
2026-03-24 20:06:35,390 INFO  [c.h.c.LifecycleService        ] [main] - [master1]:5801 [seatunnel] [5.1] [master1]:5801 is STARTED
2026-03-24 20:06:37,431 INFO  [c.h.i.s.t.TcpServerConnection ] [hz.main.IO.thread-in-3] - [master1]:5801 [seatunnel] [5.1] Initialized new cluster connection between /10.110.211.18:5801 and /10.110.211.19:47062
2026-03-24 20:06:42,533 INFO  [c.h.i.c.ClusterService        ] [hz.main.priority-generic-operation.thread-0] - [master1]:5801 [seatunnel] [5.1] 

Members {size:2, ver:2} [
        Member [master1]:5801 - 71303e1f-1974-4916-a767-71e8fa586f57 [master node] [active master] this
        Member [master2]:5801 - e12366cb-bd58-45e6-8d97-ad8f2723a800 [master node]
]

2026-03-24 20:06:42,801 INFO  [c.h.i.p.i.MigrationManager    ] [hz.main.migration] - [master1]:5801 [seatunnel] [5.1] Repartitioning cluster data. Migration tasks count: 271
2026-03-24 20:06:43,093 INFO  [o.a.s.e.s.CoordinatorService  ] [pool-4-thread-1] - [master1]:5801 [seatunnel] [5.1] Loaded event handlers: [org.apache.seatunnel.api.event.LoggingEventHandler@7e0bd9e1]
2026-03-24 20:06:44,014 INFO  [c.h.i.p.i.MigrationManager    ] [hz.main.migration] - [master1]:5801 [seatunnel] [5.1] All migration tasks have been completed. (repartitionTime=Tue Mar 24 20:06:42 CST 2026, plannedMigrations=271, completedMigrations=271, remainingMigrations=0, totalCompletedMigrations=271)
2026-03-24 20:06:44,935 INFO  [c.h.i.s.t.TcpServerConnection ] [hz.main.IO.thread-in-0] - [master1]:5801 [seatunnel] [5.1] Initialized new cluster connection between /10.110.211.18:5801 and /10.110.211.23:35674
2026-03-24 20:06:48,437 INFO  [c.h.i.s.t.TcpServerConnection ] [hz.main.IO.thread-in-1] - [master1]:5801 [seatunnel] [5.1] Initialized new cluster connection between /10.110.211.18:5801 and /10.110.211.24:53420
2026-03-24 20:06:53,531 INFO  [c.h.i.c.ClusterService        ] [hz.main.generic-operation.thread-8] - [master1]:5801 [seatunnel] [5.1] 

Members {size:4, ver:4} [
        Member [master1]:5801 - 71303e1f-1974-4916-a767-71e8fa586f57 [master node] [active master] this
        Member [master2]:5801 - e12366cb-bd58-45e6-8d97-ad8f2723a800 [master node]
        Member [node4]:5802 - 999362e6-3ad2-4137-ba7f-1404c74b35a8 [worker node]
        Member [node5]:5802 - e4c253cc-27df-4333-a381-4f1c0534dbd2 [worker node]
]

Zeta or Flink or Spark Version

zeta

Java or Scala Version

1.8.0_221

Screenshots

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[dybyte]

You might find #10570 helpful, as it discusses a similar issue.

[dybyte]

Closed by #10610 .

[DanielLeens]

I checked the current status of this one. This is the same initialization-order problem around engine_checkpoint_monitor that was discussed in the related checkpoint-monitor issue. The important update is that PR #10610 has already been merged into dev on April 3, 2026.

So from the current repo state, the bug was real and the dev branch already contains the fix. If anyone still reproduces it after picking up a build that includes #10610, that would be a fresh follow-up case rather than the original unresolved issue.