ClusterTopologyServerNotFoundException

classic Classic list List threaded Threaded
14 messages Options
kvenkatramtreddy kvenkatramtreddy
Reply | Threaded
Open this post in threaded view
|

ClusterTopologyServerNotFoundException

Hi, We have setup 3 node cluster on Linux server with 2 core CPU and 8 GB RAM. We have configured Ignite persistence and replicated caches. Please find attached configuration in the email.example-ignite-sample.xml We are receiving following error when we start single node. class org.apache.ignite.IgniteException: Failed to map keys for cache (all partition nodes left the grid) [topVer=AffinityTopologyVersion [topVer=6, minorTopVer=0], cache=users] at org.apache.ignite.internal.processors.cache.GridCacheAdapter.containsKey(GridCacheAdapter.java:633) at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.containsKey(IgniteCacheProxyImpl.java:972) at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.containsKey(GatewayProtectedCacheProxy.java:816) at com.ibm.PushNotificationService.isAssignedUser(PushNotificationService.java:239) at com.ibm.PushNotificationService.run(PushNotificationService.java:102) at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626) at com.ibm.ws.concurrent.internal.SubmittedTask.run(SubmittedTask.java:276) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: class org.apache.ignite.internal.cluster.ClusterTopologyServerNotFoundException: Failed to map keys for cache (all partition nodes left the grid) [topVer=AffinityTopologyVersion [topVer=6, minorTopVer=0], cache=users] at org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.serverNotFoundError(GridPartitionedSingleGetFuture.java:711) at org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.mapKeyToNode(GridPartitionedSingleGetFuture.java:332) at org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.map(GridPartitionedSingleGetFuture.java:216) at org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.init(GridPartitionedSingleGetFuture.java:208) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.getAsync0(GridDhtAtomicCache.java:1391) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$1600(GridDhtAtomicCache.java:130) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$16.apply(GridDhtAtomicCache.java:469) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$16.apply(GridDhtAtomicCache.java:467) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.asyncOp(GridDhtAtomicCache.java:756) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.getAsync(GridDhtAtomicCache.java:467) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.containsKeyAsync(GridCacheAdapter.java:641) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.containsKey(GridCacheAdapter.java:630) ... 11 more My requirement is that, it should run on available nodes, when other nodes available it should be replicated.

Sent from the Apache Ignite Users mailing list archive at Nabble.com.
ezhuravlev ezhuravlev
Reply | Threaded
Open this post in threaded view
|

Re: ClusterTopologyServerNotFoundException

Hi,

Looks like your cluster is unstable and part of the log you provided isn't enough to find the root cause of this.

Could you provide logs from all nodes in the cluster for investigation?

Thanks,
Evgenii

2018-03-31 7:17 GMT+03:00 kvenkatramtreddy <[hidden email]>:
Hi, We have setup 3 node cluster on Linux server with 2 core CPU and 8 GB RAM. We have configured Ignite persistence and replicated caches. Please find attached configuration in the email.example-ignite-sample.xml We are receiving following error when we start single node. class org.apache.ignite.IgniteException: Failed to map keys for cache (all partition nodes left the grid) [topVer=AffinityTopologyVersion [topVer=6, minorTopVer=0], cache=users] at org.apache.ignite.internal.processors.cache.GridCacheAdapter.containsKey(GridCacheAdapter.java:633) at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.containsKey(IgniteCacheProxyImpl.java:972) at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.containsKey(GatewayProtectedCacheProxy.java:816) at com.ibm.PushNotificationService.isAssignedUser(PushNotificationService.java:239) at com.ibm.PushNotificationService.run(PushNotificationService.java:102) at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626) at com.ibm.ws.concurrent.internal.SubmittedTask.run(SubmittedTask.java:276) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: class org.apache.ignite.internal.cluster.ClusterTopologyServerNotFoundException: Failed to map keys for cache (all partition nodes left the grid) [topVer=AffinityTopologyVersion [topVer=6, minorTopVer=0], cache=users] at org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.serverNotFoundError(GridPartitionedSingleGetFuture.java:711) at org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.mapKeyToNode(GridPartitionedSingleGetFuture.java:332) at org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.map(GridPartitionedSingleGetFuture.java:216) at org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.init(GridPartitionedSingleGetFuture.java:208) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.getAsync0(GridDhtAtomicCache.java:1391) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$1600(GridDhtAtomicCache.java:130) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$16.apply(GridDhtAtomicCache.java:469) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$16.apply(GridDhtAtomicCache.java:467) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.asyncOp(GridDhtAtomicCache.java:756) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.getAsync(GridDhtAtomicCache.java:467) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.containsKeyAsync(GridCacheAdapter.java:641) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.containsKey(GridCacheAdapter.java:630) ... 11 more My requirement is that, it should run on available nodes, when other nodes available it should be replicated.

Sent from the Apache Ignite Users mailing list archive at Nabble.com.

kvenkatramtreddy kvenkatramtreddy
Reply | Threaded
Open this post in threaded view
|

Re: ClusterTopologyServerNotFoundException

Hi,

Thank you very much.

We are receiving this error only when one node was running. Exception is
gone as soon other nodes are started and joined the cluster.

I have attached the single node where we received the error.

exception.log
<http://apache-ignite-users.70518.x6.nabble.com/file/t1700/exception.log>  

Thanks & Regards,
Venkat



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Pavel Vinokurov Pavel Vinokurov
Reply | Threaded
Open this post in threaded view
|

Re: ClusterTopologyServerNotFoundException

Hi Venkat,

Could you setup constistentId (IgniteConfiguration#setConsistentId)  for 3 server nodes and add these ids to the baseline topology 
using ignite.cluster().setBaselineTopology(nodes);

Please setup the baseline and activate the cluster as described in https://apacheignite.readme.io/docs/cluster-activation.

Thanks,
Pavel

2018-04-01 15:58 GMT+03:00 kvenkatramtreddy <[hidden email]>:
Hi,

Thank you very much.

We are receiving this error only when one node was running. Exception is
gone as soon other nodes are started and joined the cluster.

I have attached the single node where we received the error.

exception.log
<http://apache-ignite-users.70518.x6.nabble.com/file/t1700/exception.log>

Thanks & Regards,
Venkat



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/



--

Regards

Pavel Vinokurov

siva siva
Reply | Threaded
Open this post in threaded view
|

Re: ClusterTopologyServerNotFoundException

We also getting same exception ,

updated 2.3 to 2.5 version, and using baseline topology

i am trying with one client and server node ,after starting the server and
client node no issues.If we stop and start the server node , getting that
exception

* class
org.apache.ignite.internal.cluster.ClusterTopologyServerNotFoundException:
Failed to map keys for cache (all partition nodes left the grid)
[topVer=AffinityTopologyVersion [topVer=14,*



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Stanislav Lukyanov Stanislav Lukyanov
Reply | Threaded
Open this post in threaded view
|

RE: ClusterTopologyServerNotFoundException

Hi,

 

Have you tried setting consistentId and perform the activation as Pavel suggested?

I’d also suggest to go through the usage scenarios described at https://apacheignite.readme.io/docs/baseline-topology#section-usage-scenarios.

 

Thanks,

Stan

 

 

From: [hidden email]
Sent: 11 июня 2018 г. 14:16
To: [hidden email]
Subject: Re: ClusterTopologyServerNotFoundException

 

We also getting same exception ,

 

updated 2.3 to 2.5 version, and using baseline topology

 

i am trying with one client and server node ,after starting the server and

client node no issues.If we stop and start the server node , getting that

exception

 

* class

org.apache.ignite.internal.cluster.ClusterTopologyServerNotFoundException:

Failed to map keys for cache (all partition nodes left the grid)

[topVer=AffinityTopologyVersion [topVer=14,*

 

 

 

--

Sent from: http://apache-ignite-users.70518.x6.nabble.com/

 

dkarachentsev dkarachentsev
Reply | Threaded
Open this post in threaded view
|

Re: ClusterTopologyServerNotFoundException

In reply to this post by siva
Hi,

Could you please provide a reproducer? I don't get such exception.

Thanks!
-Dmitry



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
prudhvibiruda prudhvibiruda
Reply | Threaded
Open this post in threaded view
|

Re: ClusterTopologyServerNotFoundException

In reply to this post by Pavel Vinokurov
Hi,
Even I am getting the same exception with cachemode.replicated.
But my requirement is that my ignite node shouldn't wait for other nodes in
the cluster.
In our case , even when one node is down , the other should be working.So
that's why we didn't define the baseline topology.
Can you give any alternate solution for this.
<http://apache-ignite-users.70518.x6.nabble.com/file/t2790/Capture.png>



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: ClusterTopologyServerNotFoundException

Hello!

Unfortunately there is not enough information. What is partition mapping of this cache? What is the state of cluster?

Regards.
--
Ilya Kasnacheev


вт, 25 февр. 2020 г. в 11:09, prudhvibiruda <[hidden email]>:
Hi,
Even I am getting the same exception with cachemode.replicated.
But my requirement is that my ignite node shouldn't wait for other nodes in
the cluster.
In our case , even when one node is down , the other should be working.So
that's why we didn't define the baseline topology.
Can you give any alternate solution for this.
<http://apache-ignite-users.70518.x6.nabble.com/file/t2790/Capture.png>



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
prudhvibiruda prudhvibiruda
Reply | Threaded
Open this post in threaded view
|

Re: ClusterTopologyServerNotFoundException

This post was updated on .
Hi ,
Sorry for the late reply.
Please find these attached screenshots of my ignite configuration and also
the error we are getting.
We are using spring boot in our project.
Our requirement is that :
1.) We need ignite to store data on disk.
2.)When one node is down then the other nodes should continue doing the
persistence on to the disk.That's why we used CacheMode.Replicated. And since our operation shouldn't stop when any node is down, so we didn't define baseline topology.
Please help on what are we missing ? Why are we getting that exception ?
What are your suggestions?
<http://apache-ignite-users.70518.x6.nabble.com/file/t2790/Capture1.png
<http://apache-ignite-users.70518.x6.nabble.com/file/t2790/Capture2.png
<http://apache-ignite-users.70518.x6.nabble.com/file/t2790/Capture3.png



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: ClusterTopologyServerNotFoundException

Hello!

Do you have baseline topology? What does it contain?

I'm pretty sure Ignite will define baseline topology for persistent cluster, and you will have to adjust it to match your actual nodes.

Regards,
--
Ilya Kasnacheev


пн, 16 мар. 2020 г. в 09:27, prudhvibiruda <[hidden email]>:
Hi ,
Sorry for the late reply.
Please find these attached screenshots of my ignite configuration and also
the error we are getting.
We are using spring boot in our project.
Our requirement is that :
1.) We need ignite to store data on disk.
2.)When node is down then the other node should continue doing the
persistence on to the disk.
Please help on what are we missing ? Why are we getting that exception ?
What are your suggestions?
<http://apache-ignite-users.70518.x6.nabble.com/file/t2790/Capture1.png>
<http://apache-ignite-users.70518.x6.nabble.com/file/t2790/Capture2.png>
<http://apache-ignite-users.70518.x6.nabble.com/file/t2790/Capture3.png>



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
prudhvibiruda prudhvibiruda
Reply | Threaded
Open this post in threaded view
|

Re: ClusterTopologyServerNotFoundException

Hi,
We didn't explicitly define baseline topology. Because we don't want any of
our nodes (all our nodes should be server nodes) waiting for other nodes.
So at present with in our cluster we have only one server node.
So you are saying that we should customize this baseline topology? But why
is this error coming when we have only one node in our cluster?

Thanks for the quick reply,
Prudhvi



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: ClusterTopologyServerNotFoundException

Hello!

I no longer understand your architecture. First you say you want a replicated highly-available cache, then you say you only have one server node. Can you please elaborate?

Regards.
--
Ilya Kasnacheev


вт, 17 мар. 2020 г. в 10:45, prudhvibiruda <[hidden email]>:
Hi,
We didn't explicitly define baseline topology. Because we don't want any of
our nodes (all our nodes should be server nodes) waiting for other nodes.
So at present with in our cluster we have only one server node.
So you are saying that we should customize this baseline topology? But why
is this error coming when we have only one node in our cluster?

Thanks for the quick reply,
Prudhvi



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
prudhvibiruda prudhvibiruda
Reply | Threaded
Open this post in threaded view
|

Re: ClusterTopologyServerNotFoundException

Hi ,

Please don't be confused.
Our plan is to use more  server nodes further. But as of now we have only
one server node.  
Since we are planning for more server nodes we are using
CacheMode.Replicated.
We don't want our nodes not to start just because other server nodes are not
working , that's why we didn't define baseline topology.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/