Partitions stuck in MOVING state after upgrade to 2.7

classic Classic list List threaded Threaded
4 messages Options
dilaz03 dilaz03
Reply | Threaded
Open this post in threaded view
|

Partitions stuck in MOVING state after upgrade to 2.7

Hello.

Partitions on Ignite cluster stuck forever in MOVING state after
restart. Cluster version is 2.7.0#20181130-sha1:256ae401. I don't see
any errors in logs about moving partitions and and I don't understand
why Ignite try to move partitions. May be I can enable some additional
logs for troubleshooting this issue?

Thank you.

dilaz03 dilaz03
Reply | Threaded
Open this post in threaded view
|

Re: Partitions stuck in MOVING state after upgrade to 2.7

I see next logs on node startup after hard shutdown:
...
[exchange-worker-#40] DEBUG o.a.i.i.p.c.p.GridCacheDatabaseSharedManager -
Restored partition state (from WAL) [grp=test_events, p=1021,
state=MOVINGupdCntr=303]
...

Some partitions are in OWNING state, but many partitions are MOVING. And I
should call 'cache -rlp' for clearing this state. If I shutdown node after
deactivation then all is correct:
...
[exchange-worker-#40] DEBUG o.a.i.i.p.c.p.GridCacheDatabaseSharedManager -
Restored partition state (from page memory) [grp=test_events, p=0,
state=OWNINGupdCntr=568124]
...

I think I don't understand something about new version. How should I restore
node after crash?

Thank you.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
dilaz03 dilaz03
Reply | Threaded
Open this post in threaded view
|

Re: Partitions stuck in MOVING state after upgrade to 2.7

There is problem with Kubernetes, because scheduler can restart Ignite node
at any time.

Thank you.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
dilaz03 dilaz03
Reply | Threaded
Open this post in threaded view
|

Re: Partitions stuck in MOVING state after upgrade to 2.7


I think I can comment this lines in
org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition:
            if (grp.walEnabled())
                ctx.wal().log(new
PartitionMetaStateRecord(grp.groupId(), id, state(), updateCounter()));

PS. https://issues.apache.org/jira/browse/IGNITE-10226

On 1/10/19 7:33 PM, dilaz03 wrote:
> There is problem with Kubernetes, because scheduler can restart Ignite node
> at any time.
>
> Thank you.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/