Partition Loss Policy mismatch blocks server restart

classic Classic list List threaded Threaded
6 messages Options
Conrad Mukai (cmukai) Conrad Mukai (cmukai)
Reply | Threaded
Open this post in threaded view
|

Partition Loss Policy mismatch blocks server restart

I set up a cluster of server nodes with the following cacheConfiguration:

 

        <property name="cacheConfiguration">

            <list>

                <!-- Partitioned cache example configuration (Atomic mode). -->

                <bean class="org.apache.ignite.configuration.CacheConfiguration">

                    <property name="name" value="default"/>

                    <property name="atomicityMode" value="ATOMIC"/>

                    <property name="backups" value="0"/>

                </bean>

            </list>

        </property>

 

Apparently an application uploaded a client configuration with a cache group and now I cannot restart the cluster with the original configuration. I get the following error:

 

Caused by: class org.apache.ignite.IgniteCheckedException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

 

First question is how can I restart my cluster. The second question is this really due to the client configuration, and if so why is it possible for a client to break the entire service restart?

 

Thanks in advance,

Conrad

 

Evgeniy Rudenko Evgeniy Rudenko
Reply | Threaded
Open this post in threaded view
|

Re: Partition Loss Policy mismatch blocks server restart

Hi Conrad

All caches in the group should have the same partitionLossPolicy. Cache with different partitionLossPolicy should not be allowed to join the group.

Could you tell what version of Ignite you are using. Also could you attach full logs and full xml configurations to check.

On Mon, Oct 21, 2019 at 1:16 AM Conrad Mukai (cmukai) <[hidden email]> wrote:

I set up a cluster of server nodes with the following cacheConfiguration:

 

        <property name="cacheConfiguration">

            <list>

                <!-- Partitioned cache example configuration (Atomic mode). -->

                <bean class="org.apache.ignite.configuration.CacheConfiguration">

                    <property name="name" value="default"/>

                    <property name="atomicityMode" value="ATOMIC"/>

                    <property name="backups" value="0"/>

                </bean>

            </list>

        </property>

 

Apparently an application uploaded a client configuration with a cache group and now I cannot restart the cluster with the original configuration. I get the following error:

 

Caused by: class org.apache.ignite.IgniteCheckedException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

 

First question is how can I restart my cluster. The second question is this really due to the client configuration, and if so why is it possible for a client to break the entire service restart?

 

Thanks in advance,

Conrad

 



--
Best regards,
Evgeniy
Conrad Mukai (cmukai) Conrad Mukai (cmukai)
Reply | Threaded
Open this post in threaded view
|

Re: Partition Loss Policy mismatch blocks server restart

Here is a log of the failed start up:

 

cp: can't stat '/opt/ignite/apache-ignite/libs/optional/ignite-spring': No such file or directory

cp: can't stat '/opt/ignite/apache-ignite/libs/optional/ignite-indexing': No such file or directory

[21:54:08]    __________  ________________

[21:54:08]   /  _/ ___/ |/ /  _/_  __/ __/

[21:54:08]  _/ // (7 7    // /  / / / _/

[21:54:08] /___/\___/_/|_/___/ /_/ /___/

[21:54:08]

[21:54:08] ver. 2.7.6#20190911-sha1:21f7ca41

[21:54:08] 2019 Copyright(C) Apache Software Foundation

[21:54:08]

[21:54:08] Ignite documentation: http://ignite.apache.org

[21:54:08]

[21:54:08] Quiet mode.

[21:54:08]   ^-- Logging to file '/opt/ignite/apache-ignite/work/log/ignite-7e6a9d33.log'

[21:54:08]   ^-- Logging by 'Log4JLogger [quiet=true, config=/opt/ignite/apache-ignite/config/ignite-log4j.xml]'

[21:54:08]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false or "-v" to ignite.{sh|bat}

[21:54:08]

[21:54:08] OS: Linux 3.10.0-957.1.3.el7.x86_64 amd64

[21:54:08] VM information: OpenJDK Runtime Environment 1.8.0_212-b04 IcedTea OpenJDK 64-Bit Server VM 25.212-b04

[21:54:08] Configured plugins:

[21:54:08]   ^-- None

[21:54:08]

[21:54:08] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]]]

[21:54:08] Message queue limit is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides.

[21:54:08] Security status [authentication=off, tls/ssl=off]

[21:54:08] Automatically adjusted max WAL archive size to 8.0 GiB (to override, use DataStorageConfiguration.setMaxWalArhiveSize)

[2019-10-20 21:54:10,107][ERROR][main][IgniteKernal] Exception during start processors, node will be stopped and close connections

class org.apache.ignite.IgniteCheckedException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

                at org.apache.ignite.internal.processors.cache.GridCacheUtils.validateCacheGroupsAttributesMismatch(GridCacheUtils.java:1052)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.validateCacheGroupConfiguration(ClusterCachesInfo.java:1965)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onStart(ClusterCachesInfo.java:152)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnStart(GridCacheProcessor.java:762)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.onReadyForRead(GridCacheProcessor.java:737)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetastorageReadyForRead(GridCacheDatabaseSharedManager.java:409)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:675)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:4730)

                at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1048)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)

                at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)

                at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1076)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:962)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:861)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:731)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:700)

                at org.apache.ignite.Ignition.start(Ignition.java:348)

                at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)

[2019-10-20 21:54:10,115][ERROR][main][IgniteKernal] Got exception while starting (will rollback startup routine).

class org.apache.ignite.IgniteCheckedException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

                at org.apache.ignite.internal.processors.cache.GridCacheUtils.validateCacheGroupsAttributesMismatch(GridCacheUtils.java:1052)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.validateCacheGroupConfiguration(ClusterCachesInfo.java:1965)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onStart(ClusterCachesInfo.java:152)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnStart(GridCacheProcessor.java:762)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.onReadyForRead(GridCacheProcessor.java:737)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetastorageReadyForRead(GridCacheDatabaseSharedManager.java:409)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:675)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:4730)

                at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1048)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)

                at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)

                at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1076)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:962)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:861)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:731)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:700)

                at org.apache.ignite.Ignition.start(Ignition.java:348)

                at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)

[21:54:10] Ignite node stopped OK [uptime=00:00:03.043]

class org.apache.ignite.IgniteException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

                at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1029)

                at org.apache.ignite.Ignition.start(Ignition.java:351)

                at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)

Caused by: class org.apache.ignite.IgniteCheckedException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

                at org.apache.ignite.internal.processors.cache.GridCacheUtils.validateCacheGroupsAttributesMismatch(GridCacheUtils.java:1052)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.validateCacheGroupConfiguration(ClusterCachesInfo.java:1965)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onStart(ClusterCachesInfo.java:152)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnStart(GridCacheProcessor.java:762)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.onReadyForRead(GridCacheProcessor.java:737)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetastorageReadyForRead(GridCacheDatabaseSharedManager.java:409)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:675)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:4730)

                at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1048)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)

                at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)

                at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1076)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:962)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:861)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:731)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:700)

                at org.apache.ignite.Ignition.start(Ignition.java:348)

                ... 1 more

Failed to start grid: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

 

Here is the configuration file used at start up:

 

<?xml version="1.0" encoding="UTF-8"?>

 

<!--

  Licensed to the Apache Software Foundation (ASF) under one or more

  contributor license agreements.  See the NOTICE file distributed with

  this work for additional information regarding copyright ownership.

  The ASF licenses this file to You under the Apache License, Version 2.0

  (the "License"); you may not use this file except in compliance with

  the License.  You may obtain a copy of the License at

 

       http://www.apache.org/licenses/LICENSE-2.0

 

  Unless required by applicable law or agreed to in writing, software

  distributed under the License is distributed on an "AS IS" BASIS,

  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  See the License for the specific language governing permissions and

  limitations under the License.

-->

 

<!--

    Ignite Spring configuration file to startup Ignite cache.

 

    This file demonstrates how to configure cache using Spring. Provided cache

   will be created on node startup.

 

    Use this configuration file when running HTTP REST examples (see 'examples/rest' folder).

 

    When starting a standalone node, you need to execute the following command:

    {IGNITE_HOME}/bin/ignite.{bat|sh} examples/config/example-cache.xml

 

    When starting Ignite from Java IDE, pass path to this file to Ignition:

    Ignition.start("examples/config/example-cache.xml");

-->

<beans xmlns="http://www.springframework.org/schema/beans"

       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

       xsi:schemaLocation="

        http://www.springframework.org/schema/beans

        http://www.springframework.org/schema/beans/spring-beans.xsd">

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

 

        <property name="dataStorageConfiguration">

            <bean class="org.apache.ignite.configuration.DataStorageConfiguration">

                <property name="defaultDataRegionConfiguration">

                    <bean class="org.apache.ignite.configuration.DataRegionConfiguration">

                        <property name="persistenceEnabled" value="true"/>

                        <property name="name" value="data_region_data_loom"/>

 

                        <!-- initial size -->

                        <property name="initialSize" value="#{10L * 1024 * 1024 * 1024}"/>

                        <!-- max size -->

                        <property name="maxSize" value="#{50L * 1024 * 1024 * 1024}"/>

                       <!-- Enabling RANDOM_2_LRU eviction for this region.  -->

                        <property name="pageEvictionMode" value="RANDOM_2_LRU"/>

                    </bean>

                </property>

 

                <!-- Set the page size -->

                <property name="pageSize" value="#{16L * 1024}"/>

 

                <!-- Changing WAL Mode.-->

                <property name="walMode" value="FSYNC"/>

                <!-- Enable write throttling. -->

                <property name="writeThrottlingEnabled" value="true"/>

 

                <!--  WAL Archiving -->

                <property name="walPath" value="/var/lib/apache-ignite-wal"/>

                <property name="walArchivePath" value="/var/lib/apache-ignite-wal/archive"/>

            </bean>

        </property>

 

        <property name="cacheConfiguration">

            <list>

                <!-- Partitioned cache example configuration (Atomic mode). -->

                <bean class="org.apache.ignite.configuration.CacheConfiguration">

                    <property name="name" value="default"/>

                    <property name="atomicityMode" value="ATOMIC"/>

                    <property name="backups" value="0"/>

                </bean>

            </list>

        </property>

 

        <!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->

        <property name="discoverySpi">

            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

                <property name="ipFinder">

                    <!--

                        Ignite provides several options for automatic discovery that can be used

                        instead os static IP based discovery. For information on all options refer

                        to our documentation: http://apacheignite.readme.io/docs/cluster-config

                    -->

                    <!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->

                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">

                    <!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">-->

                        <property name="addresses">

                            <list>

                                <!--

                                Explicitly specifying address of a local node to let it start and

                                operate normally even if there is no more nodes in the cluster.

                                You can also optionally specify an individual port or port range.

                                -->

                                <value>10.207.86.89</value>

 

                                <!--

                                IP Address and optional port range of a remote node.

                                You can also optionally specify an individual port.

                                -->

                                <value>10.207.86.99:47500..47509</value>

                                <value>10.207.86.51:47500..47509</value>

                            </list>

                        </property>

                    </bean>

                </property>

            </bean>

        </property>

    </bean>

</beans>

 

Thanks,

Conrad

 

 

From: Evgeniy Rudenko <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Monday, October 21, 2019 at 3:10 AM
To: "[hidden email]" <[hidden email]>
Subject: Re: Partition Loss Policy mismatch blocks server restart

 

Hi Conrad

 

All caches in the group should have the same partitionLossPolicy. Cache with different partitionLossPolicy should not be allowed to join the group.

 

Could you tell what version of Ignite you are using. Also could you attach full logs and full xml configurations to check.

 

On Mon, Oct 21, 2019 at 1:16 AM Conrad Mukai (cmukai) <[hidden email]> wrote:

I set up a cluster of server nodes with the following cacheConfiguration:

 

        <property name="cacheConfiguration">

            <list>

                <!-- Partitioned cache example configuration (Atomic mode). -->

                <bean class="org.apache.ignite.configuration.CacheConfiguration">

                    <property name="name" value="default"/>

                    <property name="atomicityMode" value="ATOMIC"/>

                    <property name="backups" value="0"/>

                </bean>

            </list>

        </property>

 

Apparently an application uploaded a client configuration with a cache group and now I cannot restart the cluster with the original configuration. I get the following error:

 

Caused by: class org.apache.ignite.IgniteCheckedException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

 

First question is how can I restart my cluster. The second question is this really due to the client configuration, and if so why is it possible for a client to break the entire service restart?

 

Thanks in advance,

Conrad

 



--

Best regards,
Evgeniy

Evgeniy Rudenko Evgeniy Rudenko
Reply | Threaded
Open this post in threaded view
|

Re: Partition Loss Policy mismatch blocks server restart

But where do you define AlphaCaseTelProtobufCache? It is missed in the attached config.

This is expected behaviour if it is some new cache and you are adding it in the code of server node, because you already have a cache with different PartitionLossPolicy in your running cluster's group. Just change it's loss policy to READ_WRITE_SAFE and you will be able to join server node.

On Mon, Oct 21, 2019 at 7:53 PM Conrad Mukai (cmukai) <[hidden email]> wrote:

Here is a log of the failed start up:

 

cp: can't stat '/opt/ignite/apache-ignite/libs/optional/ignite-spring': No such file or directory

cp: can't stat '/opt/ignite/apache-ignite/libs/optional/ignite-indexing': No such file or directory

[21:54:08]    __________  ________________

[21:54:08]   /  _/ ___/ |/ /  _/_  __/ __/

[21:54:08]  _/ // (7 7    // /  / / / _/

[21:54:08] /___/\___/_/|_/___/ /_/ /___/

[21:54:08]

[21:54:08] ver. 2.7.6#20190911-sha1:21f7ca41

[21:54:08] 2019 Copyright(C) Apache Software Foundation

[21:54:08]

[21:54:08] Ignite documentation: http://ignite.apache.org

[21:54:08]

[21:54:08] Quiet mode.

[21:54:08]   ^-- Logging to file '/opt/ignite/apache-ignite/work/log/ignite-7e6a9d33.log'

[21:54:08]   ^-- Logging by 'Log4JLogger [quiet=true, config=/opt/ignite/apache-ignite/config/ignite-log4j.xml]'

[21:54:08]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false or "-v" to ignite.{sh|bat}

[21:54:08]

[21:54:08] OS: Linux 3.10.0-957.1.3.el7.x86_64 amd64

[21:54:08] VM information: OpenJDK Runtime Environment 1.8.0_212-b04 IcedTea OpenJDK 64-Bit Server VM 25.212-b04

[21:54:08] Configured plugins:

[21:54:08]   ^-- None

[21:54:08]

[21:54:08] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]]]

[21:54:08] Message queue limit is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides.

[21:54:08] Security status [authentication=off, tls/ssl=off]

[21:54:08] Automatically adjusted max WAL archive size to 8.0 GiB (to override, use DataStorageConfiguration.setMaxWalArhiveSize)

[2019-10-20 21:54:10,107][ERROR][main][IgniteKernal] Exception during start processors, node will be stopped and close connections

class org.apache.ignite.IgniteCheckedException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

                at org.apache.ignite.internal.processors.cache.GridCacheUtils.validateCacheGroupsAttributesMismatch(GridCacheUtils.java:1052)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.validateCacheGroupConfiguration(ClusterCachesInfo.java:1965)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onStart(ClusterCachesInfo.java:152)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnStart(GridCacheProcessor.java:762)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.onReadyForRead(GridCacheProcessor.java:737)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetastorageReadyForRead(GridCacheDatabaseSharedManager.java:409)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:675)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:4730)

                at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1048)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)

                at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)

                at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1076)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:962)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:861)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:731)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:700)

                at org.apache.ignite.Ignition.start(Ignition.java:348)

                at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)

[2019-10-20 21:54:10,115][ERROR][main][IgniteKernal] Got exception while starting (will rollback startup routine).

class org.apache.ignite.IgniteCheckedException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

                at org.apache.ignite.internal.processors.cache.GridCacheUtils.validateCacheGroupsAttributesMismatch(GridCacheUtils.java:1052)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.validateCacheGroupConfiguration(ClusterCachesInfo.java:1965)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onStart(ClusterCachesInfo.java:152)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnStart(GridCacheProcessor.java:762)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.onReadyForRead(GridCacheProcessor.java:737)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetastorageReadyForRead(GridCacheDatabaseSharedManager.java:409)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:675)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:4730)

                at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1048)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)

                at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)

                at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1076)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:962)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:861)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:731)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:700)

                at org.apache.ignite.Ignition.start(Ignition.java:348)

                at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)

[21:54:10] Ignite node stopped OK [uptime=00:00:03.043]

class org.apache.ignite.IgniteException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

                at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1029)

                at org.apache.ignite.Ignition.start(Ignition.java:351)

                at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)

Caused by: class org.apache.ignite.IgniteCheckedException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

                at org.apache.ignite.internal.processors.cache.GridCacheUtils.validateCacheGroupsAttributesMismatch(GridCacheUtils.java:1052)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.validateCacheGroupConfiguration(ClusterCachesInfo.java:1965)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onStart(ClusterCachesInfo.java:152)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnStart(GridCacheProcessor.java:762)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.onReadyForRead(GridCacheProcessor.java:737)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetastorageReadyForRead(GridCacheDatabaseSharedManager.java:409)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:675)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:4730)

                at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1048)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)

                at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)

                at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1076)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:962)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:861)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:731)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:700)

                at org.apache.ignite.Ignition.start(Ignition.java:348)

                ... 1 more

Failed to start grid: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

 

Here is the configuration file used at start up:

 

<?xml version="1.0" encoding="UTF-8"?>

 

<!--

  Licensed to the Apache Software Foundation (ASF) under one or more

  contributor license agreements.  See the NOTICE file distributed with

  this work for additional information regarding copyright ownership.

  The ASF licenses this file to You under the Apache License, Version 2.0

  (the "License"); you may not use this file except in compliance with

  the License.  You may obtain a copy of the License at

 

       http://www.apache.org/licenses/LICENSE-2.0

 

  Unless required by applicable law or agreed to in writing, software

  distributed under the License is distributed on an "AS IS" BASIS,

  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  See the License for the specific language governing permissions and

  limitations under the License.

-->

 

<!--

    Ignite Spring configuration file to startup Ignite cache.

 

    This file demonstrates how to configure cache using Spring. Provided cache

   will be created on node startup.

 

    Use this configuration file when running HTTP REST examples (see 'examples/rest' folder).

 

    When starting a standalone node, you need to execute the following command:

    {IGNITE_HOME}/bin/ignite.{bat|sh} examples/config/example-cache.xml

 

    When starting Ignite from Java IDE, pass path to this file to Ignition:

    Ignition.start("examples/config/example-cache.xml");

-->

<beans xmlns="http://www.springframework.org/schema/beans"

       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

       xsi:schemaLocation="

        http://www.springframework.org/schema/beans

        http://www.springframework.org/schema/beans/spring-beans.xsd">

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

 

        <property name="dataStorageConfiguration">

            <bean class="org.apache.ignite.configuration.DataStorageConfiguration">

                <property name="defaultDataRegionConfiguration">

                    <bean class="org.apache.ignite.configuration.DataRegionConfiguration">

                        <property name="persistenceEnabled" value="true"/>

                        <property name="name" value="data_region_data_loom"/>

 

                        <!-- initial size -->

                        <property name="initialSize" value="#{10L * 1024 * 1024 * 1024}"/>

                        <!-- max size -->

                        <property name="maxSize" value="#{50L * 1024 * 1024 * 1024}"/>

                       <!-- Enabling RANDOM_2_LRU eviction for this region.  -->

                        <property name="pageEvictionMode" value="RANDOM_2_LRU"/>

                    </bean>

                </property>

 

                <!-- Set the page size -->

                <property name="pageSize" value="#{16L * 1024}"/>

 

                <!-- Changing WAL Mode.-->

                <property name="walMode" value="FSYNC"/>

                <!-- Enable write throttling. -->

                <property name="writeThrottlingEnabled" value="true"/>

 

                <!--  WAL Archiving -->

                <property name="walPath" value="/var/lib/apache-ignite-wal"/>

                <property name="walArchivePath" value="/var/lib/apache-ignite-wal/archive"/>

            </bean>

        </property>

 

        <property name="cacheConfiguration">

            <list>

                <!-- Partitioned cache example configuration (Atomic mode). -->

                <bean class="org.apache.ignite.configuration.CacheConfiguration">

                    <property name="name" value="default"/>

                    <property name="atomicityMode" value="ATOMIC"/>

                    <property name="backups" value="0"/>

                </bean>

            </list>

        </property>

 

        <!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->

        <property name="discoverySpi">

            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

                <property name="ipFinder">

                    <!--

                        Ignite provides several options for automatic discovery that can be used

                        instead os static IP based discovery. For information on all options refer

                        to our documentation: http://apacheignite.readme.io/docs/cluster-config

                    -->

                    <!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->

                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">

                    <!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">-->

                        <property name="addresses">

                            <list>

                                <!--

                                Explicitly specifying address of a local node to let it start and

                                operate normally even if there is no more nodes in the cluster.

                                You can also optionally specify an individual port or port range.

                                -->

                                <value>10.207.86.89</value>

 

                                <!--

                                IP Address and optional port range of a remote node.

                                You can also optionally specify an individual port.

                                -->

                                <value>10.207.86.99:47500..47509</value>

                                <value>10.207.86.51:47500..47509</value>

                            </list>

                        </property>

                    </bean>

                </property>

            </bean>

        </property>

    </bean>

</beans>

 

Thanks,

Conrad

 

 

From: Evgeniy Rudenko <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Monday, October 21, 2019 at 3:10 AM
To: "[hidden email]" <[hidden email]>
Subject: Re: Partition Loss Policy mismatch blocks server restart

 

Hi Conrad

 

All caches in the group should have the same partitionLossPolicy. Cache with different partitionLossPolicy should not be allowed to join the group.

 

Could you tell what version of Ignite you are using. Also could you attach full logs and full xml configurations to check.

 

On Mon, Oct 21, 2019 at 1:16 AM Conrad Mukai (cmukai) <[hidden email]> wrote:

I set up a cluster of server nodes with the following cacheConfiguration:

 

        <property name="cacheConfiguration">

            <list>

                <!-- Partitioned cache example configuration (Atomic mode). -->

                <bean class="org.apache.ignite.configuration.CacheConfiguration">

                    <property name="name" value="default"/>

                    <property name="atomicityMode" value="ATOMIC"/>

                    <property name="backups" value="0"/>

                </bean>

            </list>

        </property>

 

Apparently an application uploaded a client configuration with a cache group and now I cannot restart the cluster with the original configuration. I get the following error:

 

Caused by: class org.apache.ignite.IgniteCheckedException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

 

First question is how can I restart my cluster. The second question is this really due to the client configuration, and if so why is it possible for a client to break the entire service restart?

 

Thanks in advance,

Conrad

 



--

Best regards,
Evgeniy



--
Best regards,
Evgeniy
Conrad Mukai (cmukai) Conrad Mukai (cmukai)
Reply | Threaded
Open this post in threaded view
|

Re: Partition Loss Policy mismatch blocks server restart

It was defined by an application connecting to the cluster with its own configuration. I don’t have the original configuration, but a later version of it is pasted below. This brings me to my main question. Is it OK for client code to reset the configuration like this? If not what are the measures that should be used to prevent this.

 

Thanks,

Conrad

 

 

<?xml version="1.0" encoding="UTF-8"?>

 

<!--

            Licensed to the Apache Software Foundation (ASF) under one or more

  contributor license agreements.  See the NOTICE file distributed with

  this work for additional information regarding copyright ownership.

  The ASF licenses this file to You under the Apache License, Version 2.0

  (the "License"); you may not use this file except in compliance with

  the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software

  distributed under the License is distributed on an "AS IS" BASIS,

  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  See the License for the specific language governing permissions and

  limitations under the License.

-->

 

<!--

              Ignite configuration with all defaults and enabled p2p deployment and enabled events.

-->

<beans xmlns="http://www.springframework.org/schema/beans"

       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

       xmlns:util="http://www.springframework.org/schema/util"

       xsi:schemaLocation="

        http://www.springframework.org/schema/beans

        http://www.springframework.org/schema/beans/spring-beans.xsd

        http://www.springframework.org/schema/util

        http://www.springframework.org/schema/util/spring-util.xsd">

 

 

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <!-- Set to true to enable distributed class loading for examples, default is false. -->

        <property name="peerClassLoadingEnabled" value="false"/>

        <property name="failureDetectionTimeout" value="1234567891011" />

        <property name="clientMode" value="true"/>

 

         <!-- Enable task execution events for examples. -->

        <!--property name="includeEventTypes">

                                      <list>

                Task execution events

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_STARTED"/>

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_FINISHED"/>

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_FAILED"/>

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_TIMEDOUT"/>

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_SESSION_ATTR_SET"/>

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_REDUCED"/>

 

                Cache events

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_PUT"/>

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_READ"/>

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_REMOVED"/>

            </list>

        </property-->

 

        <property name="cacheConfiguration">

            <list>

                <!-- Sensors cache -->

                <bean class="org.apache.ignite.configuration.CacheConfiguration">

                    <property name="name" value="AlphaCaseTelProtobufCache2"/>

                    <property name="backups" value="1"/>

                                    <property name="cacheMode" value="PARTITIONED"/>

                    <property name="partitionLossPolicy" value="READ_WRITE_SAFE"/>

                                    <property name="atomicityMode" value="ATOMIC"/>

                                                                <property name="groupName" value="group_data_loom_2"/>

                                                                <property name="dataRegionName" value="data_region_data_loom"/>

 

                                                                <property name="expiryPolicyFactory">

                                                               <bean class="javax.cache.expiry.CreatedExpiryPolicy" factory-method="factoryOf">

                                               <constructor-arg>

                                                              <bean class="javax.cache.expiry.Duration">

                                                  <constructor-arg value="MINUTES"/>

                                                  <constructor-arg value="10"/>

                                                              </bean>

                                              </constructor-arg>

                                                               </bean>

                                                </property>

 

                    <property name="queryEntities">

                        <list>

                            <bean class="org.apache.ignite.cache.QueryEntity">

                                <!-- Key Type -->

                                <property name="keyType" value="java.lang.String"/>

                                <!-- Value Type -->

                                <property name="valueType" value="com.cisco.telemetry.model.IgniteCasePojo"/>

 

                                <!-- Fields to be used from SQL -->

                               <property name="fields">

                                    <map>

                                        <!--  <entry key="msgGuid" value="java.lang.String"/>

                                                                                                                                    <entry key="case" value="com.cisco.telemetry.model.IgniteCasePojo"/> -->

                                        <entry key="msgGuid" value="java.lang.String"/>

                                        <entry key="proto" value="byte[]"/>

                                        <entry key="kafkaTime" value="java.lang.Long"/>

                                        <entry key="ingestTime" value="java.lang.Long"/>

                                        <entry key="edge" value="java.lang.String"/>

                                    </map>

                                </property>

 

                                <!-- Assign an unique name for the key to access it from SQL easily. -->

                                <property name="keyFieldName" value="msgGuid"/>

                                                                                                                                <property name="valueFieldName" value="proto"/>

 

                                                                                                                               <!-- Changing the table name. -->

                                <property name="tableName" value="CaseTelProtobuf"/>

 

                                <!-- Defining indexed fields.-->

                                <property name="indexes">

                                    <list>

                                        <!-- Single field (aka. column) index -->

                                        <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="msgGuid"/>

                                        </bean>

 

                                        <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="proto"/>

                                        </bean>

 

                                                                                                                                                                <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="kafkaTime"/>

                                        </bean>

 

                                        <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="ingestTime"/>

                                        </bean>

 

                                        <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="edge"/>

                                        </bean>

 

                                    </list>

                                </property>

                            </bean>

                        </list>

                    </property>

                </bean>

 

                <!-- Temperature measurements cache. -->

                <bean class="org.apache.ignite.configuration.CacheConfiguration">

                    <property name="name" value="AlphaUrsProtobufCache2"/>

                    <property name="backups" value="1"/>

<property name="cacheMode" value="PARTITIONED"/>

                    <property name="partitionLossPolicy" value="READ_WRITE_SAFE"/>

 

                                                <property name="atomicityMode" value="ATOMIC"/>

                                                                                <property name="groupName" value="group_data_loom_2"/>

                                                                                <property name="dataRegionName" value="data_region_data_loom"/>

 

                                                                                <property name="expiryPolicyFactory">

                                                               <bean class="javax.cache.expiry.CreatedExpiryPolicy" factory-method="factoryOf">

                                               <constructor-arg>

                                                              <bean class="javax.cache.expiry.Duration">

                                                  <constructor-arg value="MINUTES"/>

                                                  <constructor-arg value="10"/>

                                                              </bean>

                                               </constructor-arg>

                                                               </bean>

                                                </property>

 

                    <property name="queryEntities">

                        <list>

                            <bean class="org.apache.ignite.cache.QueryEntity">

                                <!-- Key Type -->

                                <property name="keyType" value="java.lang.String"/>

                                <!-- Value Type -->

                                <property name="valueType" value="com.cisco.telemetry.model.IgniteUrsPojo"/>

 

                                <!-- Fields to be used from SQL -->

                                             <property name="fields">

                                    <map>

                                        <!--  <entry key="msgGuid" value="java.lang.String"/>

                                        <entry key="urls" value="com.cisco.telemetry.model.IgniteUrsPojo"/>       -->

                                        <entry key="msgGuid" value="java.lang.String"/>

                                        <entry key="proto" value="byte[]"/>

                                        <entry key="kafkaTime" value="java.lang.Long"/>

                                        <entry key="ingestTime" value="java.lang.Long"/>

                                        <entry key="edge" value="java.lang.String"/>

                                    </map>

                                </property>

 

                                <!-- Assign an unique name for the value to access it from SQL easily. -->

                                                                                                                                <property name="keyFieldName" value="msgGuid"/>

                                <property name="valueFieldName" value="proto"/>

 

                                <!-- Changing the table name. -->

                                <property name="tableName" value="UrlProtobuf"/>

 

                                <!-- Defining indexed fields.-->

                                <property name="indexes">

                                    <list>

                                        <!-- Single field (aka. column) index -->

                                        <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="msgGuid"/>

                                        </bean>

 

                                        <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="proto"/>

                                        </bean>

 

                                                                                                                                                                <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="kafkaTime"/>

                                        </bean>

 

                                        <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="ingestTime"/>

                                        </bean>

 

                                        <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="edge"/>

                                        </bean>

                                    </list>

                                </property>

                            </bean>

                        </list>

                    </property>

                </bean>

 

 

 

            </list>

        </property>

 

        <!-- Setting up affinity collocation with Spring XML configuration. -->

        <property name="cacheKeyConfiguration">

            <list>

                <bean class="org.apache.ignite.cache.CacheKeyConfiguration">

                    <property name="typeName" value="java.lang.String"/>

                   <property name="affinityKeyFieldName" value="msgGuid"/>

                </bean>

            </list>

        </property>

 

 

        <!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->

        <property name="discoverySpi">

            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

                                <property name="ipFinder">

                    <!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->

                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">

                        <property name="addresses">

                            <list>

                                <!-- In distributed environment, replace with actual host IP address. -->

                                 <!--<value>127.0.0.1:47500..47509</value>  -->

                                 <value>10.207.86.89:47500..47509</value>

                                 <value>10.207.86.52:47500..47509</value>

                                 <value>10.207.86.37:47500..47509</value>

                                 <value>10.207.86.99:47500..47509</value>

                                 <value>10.207.86.51:47500..47509</value>

                                                                <value>10.207.86.112:47500..47509</value>

                            </list>

                        </property>

                    </bean>

                </property>

            </bean>

        </property>

    </bean>

</beans>

 

 

From: Evgeniy Rudenko <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Tuesday, October 22, 2019 at 2:27 AM
To: "[hidden email]" <[hidden email]>
Subject: Re: Partition Loss Policy mismatch blocks server restart

 

But where do you define AlphaCaseTelProtobufCache? It is missed in the attached config.

 

This is expected behaviour if it is some new cache and you are adding it in the code of server node, because you already have a cache with different PartitionLossPolicy in your running cluster's group. Just change it's loss policy to READ_WRITE_SAFE and you will be able to join server node.

 

On Mon, Oct 21, 2019 at 7:53 PM Conrad Mukai (cmukai) <[hidden email]> wrote:

Here is a log of the failed start up:

 

cp: can't stat '/opt/ignite/apache-ignite/libs/optional/ignite-spring': No such file or directory

cp: can't stat '/opt/ignite/apache-ignite/libs/optional/ignite-indexing': No such file or directory

[21:54:08]    __________  ________________

[21:54:08]   /  _/ ___/ |/ /  _/_  __/ __/

[21:54:08]  _/ // (7 7    // /  / / / _/

[21:54:08] /___/\___/_/|_/___/ /_/ /___/

[21:54:08]

[21:54:08] ver. 2.7.6#20190911-sha1:21f7ca41

[21:54:08] 2019 Copyright(C) Apache Software Foundation

[21:54:08]

[21:54:08] Ignite documentation: http://ignite.apache.org

[21:54:08]

[21:54:08] Quiet mode.

[21:54:08]   ^-- Logging to file '/opt/ignite/apache-ignite/work/log/ignite-7e6a9d33.log'

[21:54:08]   ^-- Logging by 'Log4JLogger [quiet=true, config=/opt/ignite/apache-ignite/config/ignite-log4j.xml]'

[21:54:08]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false or "-v" to ignite.{sh|bat}

[21:54:08]

[21:54:08] OS: Linux 3.10.0-957.1.3.el7.x86_64 amd64

[21:54:08] VM information: OpenJDK Runtime Environment 1.8.0_212-b04 IcedTea OpenJDK 64-Bit Server VM 25.212-b04

[21:54:08] Configured plugins:

[21:54:08]   ^-- None

[21:54:08]

[21:54:08] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]]]

[21:54:08] Message queue limit is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides.

[21:54:08] Security status [authentication=off, tls/ssl=off]

[21:54:08] Automatically adjusted max WAL archive size to 8.0 GiB (to override, use DataStorageConfiguration.setMaxWalArhiveSize)

[2019-10-20 21:54:10,107][ERROR][main][IgniteKernal] Exception during start processors, node will be stopped and close connections

class org.apache.ignite.IgniteCheckedException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

                at org.apache.ignite.internal.processors.cache.GridCacheUtils.validateCacheGroupsAttributesMismatch(GridCacheUtils.java:1052)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.validateCacheGroupConfiguration(ClusterCachesInfo.java:1965)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onStart(ClusterCachesInfo.java:152)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnStart(GridCacheProcessor.java:762)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.onReadyForRead(GridCacheProcessor.java:737)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetastorageReadyForRead(GridCacheDatabaseSharedManager.java:409)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:675)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:4730)

                at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1048)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)

                at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)

                at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1076)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:962)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:861)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:731)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:700)

                at org.apache.ignite.Ignition.start(Ignition.java:348)

                at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)

[2019-10-20 21:54:10,115][ERROR][main][IgniteKernal] Got exception while starting (will rollback startup routine).

class org.apache.ignite.IgniteCheckedException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

                at org.apache.ignite.internal.processors.cache.GridCacheUtils.validateCacheGroupsAttributesMismatch(GridCacheUtils.java:1052)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.validateCacheGroupConfiguration(ClusterCachesInfo.java:1965)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onStart(ClusterCachesInfo.java:152)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnStart(GridCacheProcessor.java:762)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.onReadyForRead(GridCacheProcessor.java:737)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetastorageReadyForRead(GridCacheDatabaseSharedManager.java:409)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:675)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:4730)

                at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1048)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)

                at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)

                at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1076)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:962)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:861)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:731)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:700)

                at org.apache.ignite.Ignition.start(Ignition.java:348)

                at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)

[21:54:10] Ignite node stopped OK [uptime=00:00:03.043]

class org.apache.ignite.IgniteException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

                at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1029)

                at org.apache.ignite.Ignition.start(Ignition.java:351)

                at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)

Caused by: class org.apache.ignite.IgniteCheckedException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

                at org.apache.ignite.internal.processors.cache.GridCacheUtils.validateCacheGroupsAttributesMismatch(GridCacheUtils.java:1052)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.validateCacheGroupConfiguration(ClusterCachesInfo.java:1965)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onStart(ClusterCachesInfo.java:152)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnStart(GridCacheProcessor.java:762)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.onReadyForRead(GridCacheProcessor.java:737)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetastorageReadyForRead(GridCacheDatabaseSharedManager.java:409)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:675)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:4730)

                at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1048)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)

                at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)

                at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1076)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:962)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:861)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:731)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:700)

                at org.apache.ignite.Ignition.start(Ignition.java:348)

                ... 1 more

Failed to start grid: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

 

Here is the configuration file used at start up:

 

<?xml version="1.0" encoding="UTF-8"?>

 

<!--

  Licensed to the Apache Software Foundation (ASF) under one or more

  contributor license agreements.  See the NOTICE file distributed with

  this work for additional information regarding copyright ownership.

  The ASF licenses this file to You under the Apache License, Version 2.0

  (the "License"); you may not use this file except in compliance with

  the License.  You may obtain a copy of the License at

 

       http://www.apache.org/licenses/LICENSE-2.0

 

  Unless required by applicable law or agreed to in writing, software

  distributed under the License is distributed on an "AS IS" BASIS,

  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  See the License for the specific language governing permissions and

  limitations under the License.

-->

 

<!--

    Ignite Spring configuration file to startup Ignite cache.

 

    This file demonstrates how to configure cache using Spring. Provided cache

   will be created on node startup.

 

    Use this configuration file when running HTTP REST examples (see 'examples/rest' folder).

 

    When starting a standalone node, you need to execute the following command:

    {IGNITE_HOME}/bin/ignite.{bat|sh} examples/config/example-cache.xml

 

    When starting Ignite from Java IDE, pass path to this file to Ignition:

    Ignition.start("examples/config/example-cache.xml");

-->

<beans xmlns="http://www.springframework.org/schema/beans"

       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

       xsi:schemaLocation="

        http://www.springframework.org/schema/beans

        http://www.springframework.org/schema/beans/spring-beans.xsd">

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

 

        <property name="dataStorageConfiguration">

            <bean class="org.apache.ignite.configuration.DataStorageConfiguration">

                <property name="defaultDataRegionConfiguration">

                    <bean class="org.apache.ignite.configuration.DataRegionConfiguration">

                        <property name="persistenceEnabled" value="true"/>

                        <property name="name" value="data_region_data_loom"/>

 

                        <!-- initial size -->

                        <property name="initialSize" value="#{10L * 1024 * 1024 * 1024}"/>

                        <!-- max size -->

                        <property name="maxSize" value="#{50L * 1024 * 1024 * 1024}"/>

                       <!-- Enabling RANDOM_2_LRU eviction for this region.  -->

                        <property name="pageEvictionMode" value="RANDOM_2_LRU"/>

                    </bean>

                </property>

 

                <!-- Set the page size -->

                <property name="pageSize" value="#{16L * 1024}"/>

 

                <!-- Changing WAL Mode.-->

                <property name="walMode" value="FSYNC"/>

                <!-- Enable write throttling. -->

                <property name="writeThrottlingEnabled" value="true"/>

 

                <!--  WAL Archiving -->

                <property name="walPath" value="/var/lib/apache-ignite-wal"/>

                <property name="walArchivePath" value="/var/lib/apache-ignite-wal/archive"/>

            </bean>

        </property>

 

        <property name="cacheConfiguration">

            <list>

                <!-- Partitioned cache example configuration (Atomic mode). -->

                <bean class="org.apache.ignite.configuration.CacheConfiguration">

                    <property name="name" value="default"/>

                    <property name="atomicityMode" value="ATOMIC"/>

                    <property name="backups" value="0"/>

                </bean>

            </list>

        </property>

 

        <!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->

        <property name="discoverySpi">

            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

                <property name="ipFinder">

                    <!--

                        Ignite provides several options for automatic discovery that can be used

                        instead os static IP based discovery. For information on all options refer

                        to our documentation: http://apacheignite.readme.io/docs/cluster-config

                    -->

                    <!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->

                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">

                    <!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">-->

                        <property name="addresses">

                            <list>

                                <!--

                                Explicitly specifying address of a local node to let it start and

                                operate normally even if there is no more nodes in the cluster.

                                You can also optionally specify an individual port or port range.

                                -->

                                <value>10.207.86.89</value>

 

                                <!--

                                IP Address and optional port range of a remote node.

                                You can also optionally specify an individual port.

                                -->

                                <value>10.207.86.99:47500..47509</value>

                                <value>10.207.86.51:47500..47509</value>

                            </list>

                        </property>

                    </bean>

                </property>

            </bean>

        </property>

    </bean>

</beans>

 

Thanks,

Conrad

 

 

From: Evgeniy Rudenko <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Monday, October 21, 2019 at 3:10 AM
To: "[hidden email]" <[hidden email]>
Subject: Re: Partition Loss Policy mismatch blocks server restart

 

Hi Conrad

 

All caches in the group should have the same partitionLossPolicy. Cache with different partitionLossPolicy should not be allowed to join the group.

 

Could you tell what version of Ignite you are using. Also could you attach full logs and full xml configurations to check.

 

On Mon, Oct 21, 2019 at 1:16 AM Conrad Mukai (cmukai) <[hidden email]> wrote:

I set up a cluster of server nodes with the following cacheConfiguration:

 

        <property name="cacheConfiguration">

            <list>

                <!-- Partitioned cache example configuration (Atomic mode). -->

                <bean class="org.apache.ignite.configuration.CacheConfiguration">

                    <property name="name" value="default"/>

                    <property name="atomicityMode" value="ATOMIC"/>

                    <property name="backups" value="0"/>

                </bean>

            </list>

        </property>

 

Apparently an application uploaded a client configuration with a cache group and now I cannot restart the cluster with the original configuration. I get the following error:

 

Caused by: class org.apache.ignite.IgniteCheckedException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

 

First question is how can I restart my cluster. The second question is this really due to the client configuration, and if so why is it possible for a client to break the entire service restart?

 

Thanks in advance,

Conrad

 



--

Best regards,
Evgeniy



--

Best regards,
Evgeniy

Evgeniy Rudenko Evgeniy Rudenko
Reply | Threaded
Open this post in threaded view
|

Re: Partition Loss Policy mismatch blocks server restart

It is not some kind of reset, you just adding new cache and this is allowed for the client. Rules for new caches are the same for client and server nodes. Ignite just checks what is already in cluster. So in your case it should be the following behaviour:
1) first cache starts in group_data_loom group with READ_WRITE_SAFE loss policy, it doesn't matter from where it came, client or server
2) you are trying to start anther cache with IGNORE loss policy in the same group
3) you are declined because it is not allowed to have 2 different policies in the same group

On Tue, Oct 22, 2019 at 8:03 PM Conrad Mukai (cmukai) <[hidden email]> wrote:

It was defined by an application connecting to the cluster with its own configuration. I don’t have the original configuration, but a later version of it is pasted below. This brings me to my main question. Is it OK for client code to reset the configuration like this? If not what are the measures that should be used to prevent this.

 

Thanks,

Conrad

 

 

<?xml version="1.0" encoding="UTF-8"?>

 

<!--

            Licensed to the Apache Software Foundation (ASF) under one or more

  contributor license agreements.  See the NOTICE file distributed with

  this work for additional information regarding copyright ownership.

  The ASF licenses this file to You under the Apache License, Version 2.0

  (the "License"); you may not use this file except in compliance with

  the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software

  distributed under the License is distributed on an "AS IS" BASIS,

  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  See the License for the specific language governing permissions and

  limitations under the License.

-->

 

<!--

              Ignite configuration with all defaults and enabled p2p deployment and enabled events.

-->

<beans xmlns="http://www.springframework.org/schema/beans"

       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

       xmlns:util="http://www.springframework.org/schema/util"

       xsi:schemaLocation="

        http://www.springframework.org/schema/beans

        http://www.springframework.org/schema/beans/spring-beans.xsd

        http://www.springframework.org/schema/util

        http://www.springframework.org/schema/util/spring-util.xsd">

 

 

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <!-- Set to true to enable distributed class loading for examples, default is false. -->

        <property name="peerClassLoadingEnabled" value="false"/>

        <property name="failureDetectionTimeout" value="1234567891011" />

        <property name="clientMode" value="true"/>

 

         <!-- Enable task execution events for examples. -->

        <!--property name="includeEventTypes">

                                      <list>

                Task execution events

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_STARTED"/>

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_FINISHED"/>

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_FAILED"/>

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_TIMEDOUT"/>

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_SESSION_ATTR_SET"/>

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_REDUCED"/>

 

                Cache events

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_PUT"/>

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_READ"/>

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_REMOVED"/>

            </list>

        </property-->

 

        <property name="cacheConfiguration">

            <list>

                <!-- Sensors cache -->

                <bean class="org.apache.ignite.configuration.CacheConfiguration">

                    <property name="name" value="AlphaCaseTelProtobufCache2"/>

                    <property name="backups" value="1"/>

                                    <property name="cacheMode" value="PARTITIONED"/>

                    <property name="partitionLossPolicy" value="READ_WRITE_SAFE"/>

                                    <property name="atomicityMode" value="ATOMIC"/>

                                                                <property name="groupName" value="group_data_loom_2"/>

                                                                <property name="dataRegionName" value="data_region_data_loom"/>

 

                                                                <property name="expiryPolicyFactory">

                                                               <bean class="javax.cache.expiry.CreatedExpiryPolicy" factory-method="factoryOf">

                                               <constructor-arg>

                                                              <bean class="javax.cache.expiry.Duration">

                                                  <constructor-arg value="MINUTES"/>

                                                  <constructor-arg value="10"/>

                                                              </bean>

                                              </constructor-arg>

                                                               </bean>

                                                </property>

 

                    <property name="queryEntities">

                        <list>

                            <bean class="org.apache.ignite.cache.QueryEntity">

                                <!-- Key Type -->

                                <property name="keyType" value="java.lang.String"/>

                                <!-- Value Type -->

                                <property name="valueType" value="com.cisco.telemetry.model.IgniteCasePojo"/>

 

                                <!-- Fields to be used from SQL -->

                               <property name="fields">

                                    <map>

                                        <!--  <entry key="msgGuid" value="java.lang.String"/>

                                                                                                                                    <entry key="case" value="com.cisco.telemetry.model.IgniteCasePojo"/> -->

                                        <entry key="msgGuid" value="java.lang.String"/>

                                        <entry key="proto" value="byte[]"/>

                                        <entry key="kafkaTime" value="java.lang.Long"/>

                                        <entry key="ingestTime" value="java.lang.Long"/>

                                        <entry key="edge" value="java.lang.String"/>

                                    </map>

                                </property>

 

                                <!-- Assign an unique name for the key to access it from SQL easily. -->

                                <property name="keyFieldName" value="msgGuid"/>

                                                                                                                                <property name="valueFieldName" value="proto"/>

 

                                                                                                                               <!-- Changing the table name. -->

                                <property name="tableName" value="CaseTelProtobuf"/>

 

                                <!-- Defining indexed fields.-->

                                <property name="indexes">

                                    <list>

                                        <!-- Single field (aka. column) index -->

                                        <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="msgGuid"/>

                                        </bean>

 

                                        <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="proto"/>

                                        </bean>

 

                                                                                                                                                                <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="kafkaTime"/>

                                        </bean>

 

                                        <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="ingestTime"/>

                                        </bean>

 

                                        <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="edge"/>

                                        </bean>

 

                                    </list>

                                </property>

                            </bean>

                        </list>

                    </property>

                </bean>

 

                <!-- Temperature measurements cache. -->

                <bean class="org.apache.ignite.configuration.CacheConfiguration">

                    <property name="name" value="AlphaUrsProtobufCache2"/>

                    <property name="backups" value="1"/>

<property name="cacheMode" value="PARTITIONED"/>

                    <property name="partitionLossPolicy" value="READ_WRITE_SAFE"/>

 

                                                <property name="atomicityMode" value="ATOMIC"/>

                                                                                <property name="groupName" value="group_data_loom_2"/>

                                                                                <property name="dataRegionName" value="data_region_data_loom"/>

 

                                                                                <property name="expiryPolicyFactory">

                                                               <bean class="javax.cache.expiry.CreatedExpiryPolicy" factory-method="factoryOf">

                                               <constructor-arg>

                                                              <bean class="javax.cache.expiry.Duration">

                                                  <constructor-arg value="MINUTES"/>

                                                  <constructor-arg value="10"/>

                                                              </bean>

                                               </constructor-arg>

                                                               </bean>

                                                </property>

 

                    <property name="queryEntities">

                        <list>

                            <bean class="org.apache.ignite.cache.QueryEntity">

                                <!-- Key Type -->

                                <property name="keyType" value="java.lang.String"/>

                                <!-- Value Type -->

                                <property name="valueType" value="com.cisco.telemetry.model.IgniteUrsPojo"/>

 

                                <!-- Fields to be used from SQL -->

                                             <property name="fields">

                                    <map>

                                        <!--  <entry key="msgGuid" value="java.lang.String"/>

                                        <entry key="urls" value="com.cisco.telemetry.model.IgniteUrsPojo"/>       -->

                                        <entry key="msgGuid" value="java.lang.String"/>

                                        <entry key="proto" value="byte[]"/>

                                        <entry key="kafkaTime" value="java.lang.Long"/>

                                        <entry key="ingestTime" value="java.lang.Long"/>

                                        <entry key="edge" value="java.lang.String"/>

                                    </map>

                                </property>

 

                                <!-- Assign an unique name for the value to access it from SQL easily. -->

                                                                                                                                <property name="keyFieldName" value="msgGuid"/>

                                <property name="valueFieldName" value="proto"/>

 

                                <!-- Changing the table name. -->

                                <property name="tableName" value="UrlProtobuf"/>

 

                                <!-- Defining indexed fields.-->

                                <property name="indexes">

                                    <list>

                                        <!-- Single field (aka. column) index -->

                                        <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="msgGuid"/>

                                        </bean>

 

                                        <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="proto"/>

                                        </bean>

 

                                                                                                                                                                <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="kafkaTime"/>

                                        </bean>

 

                                        <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="ingestTime"/>

                                        </bean>

 

                                        <bean class="org.apache.ignite.cache.QueryIndex">

                                            <constructor-arg value="edge"/>

                                        </bean>

                                    </list>

                                </property>

                            </bean>

                        </list>

                    </property>

                </bean>

 

 

 

            </list>

        </property>

 

        <!-- Setting up affinity collocation with Spring XML configuration. -->

        <property name="cacheKeyConfiguration">

            <list>

                <bean class="org.apache.ignite.cache.CacheKeyConfiguration">

                    <property name="typeName" value="java.lang.String"/>

                   <property name="affinityKeyFieldName" value="msgGuid"/>

                </bean>

            </list>

        </property>

 

 

        <!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->

        <property name="discoverySpi">

            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

                                <property name="ipFinder">

                    <!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->

                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">

                        <property name="addresses">

                            <list>

                                <!-- In distributed environment, replace with actual host IP address. -->

                                 <!--<value>127.0.0.1:47500..47509</value>  -->

                                 <value>10.207.86.89:47500..47509</value>

                                 <value>10.207.86.52:47500..47509</value>

                                 <value>10.207.86.37:47500..47509</value>

                                 <value>10.207.86.99:47500..47509</value>

                                 <value>10.207.86.51:47500..47509</value>

                                                                <value>10.207.86.112:47500..47509</value>

                            </list>

                        </property>

                    </bean>

                </property>

            </bean>

        </property>

    </bean>

</beans>

 

 

From: Evgeniy Rudenko <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Tuesday, October 22, 2019 at 2:27 AM
To: "[hidden email]" <[hidden email]>
Subject: Re: Partition Loss Policy mismatch blocks server restart

 

But where do you define AlphaCaseTelProtobufCache? It is missed in the attached config.

 

This is expected behaviour if it is some new cache and you are adding it in the code of server node, because you already have a cache with different PartitionLossPolicy in your running cluster's group. Just change it's loss policy to READ_WRITE_SAFE and you will be able to join server node.

 

On Mon, Oct 21, 2019 at 7:53 PM Conrad Mukai (cmukai) <[hidden email]> wrote:

Here is a log of the failed start up:

 

cp: can't stat '/opt/ignite/apache-ignite/libs/optional/ignite-spring': No such file or directory

cp: can't stat '/opt/ignite/apache-ignite/libs/optional/ignite-indexing': No such file or directory

[21:54:08]    __________  ________________

[21:54:08]   /  _/ ___/ |/ /  _/_  __/ __/

[21:54:08]  _/ // (7 7    // /  / / / _/

[21:54:08] /___/\___/_/|_/___/ /_/ /___/

[21:54:08]

[21:54:08] ver. 2.7.6#20190911-sha1:21f7ca41

[21:54:08] 2019 Copyright(C) Apache Software Foundation

[21:54:08]

[21:54:08] Ignite documentation: http://ignite.apache.org

[21:54:08]

[21:54:08] Quiet mode.

[21:54:08]   ^-- Logging to file '/opt/ignite/apache-ignite/work/log/ignite-7e6a9d33.log'

[21:54:08]   ^-- Logging by 'Log4JLogger [quiet=true, config=/opt/ignite/apache-ignite/config/ignite-log4j.xml]'

[21:54:08]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false or "-v" to ignite.{sh|bat}

[21:54:08]

[21:54:08] OS: Linux 3.10.0-957.1.3.el7.x86_64 amd64

[21:54:08] VM information: OpenJDK Runtime Environment 1.8.0_212-b04 IcedTea OpenJDK 64-Bit Server VM 25.212-b04

[21:54:08] Configured plugins:

[21:54:08]   ^-- None

[21:54:08]

[21:54:08] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]]]

[21:54:08] Message queue limit is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides.

[21:54:08] Security status [authentication=off, tls/ssl=off]

[21:54:08] Automatically adjusted max WAL archive size to 8.0 GiB (to override, use DataStorageConfiguration.setMaxWalArhiveSize)

[2019-10-20 21:54:10,107][ERROR][main][IgniteKernal] Exception during start processors, node will be stopped and close connections

class org.apache.ignite.IgniteCheckedException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

                at org.apache.ignite.internal.processors.cache.GridCacheUtils.validateCacheGroupsAttributesMismatch(GridCacheUtils.java:1052)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.validateCacheGroupConfiguration(ClusterCachesInfo.java:1965)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onStart(ClusterCachesInfo.java:152)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnStart(GridCacheProcessor.java:762)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.onReadyForRead(GridCacheProcessor.java:737)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetastorageReadyForRead(GridCacheDatabaseSharedManager.java:409)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:675)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:4730)

                at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1048)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)

                at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)

                at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1076)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:962)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:861)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:731)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:700)

                at org.apache.ignite.Ignition.start(Ignition.java:348)

                at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)

[2019-10-20 21:54:10,115][ERROR][main][IgniteKernal] Got exception while starting (will rollback startup routine).

class org.apache.ignite.IgniteCheckedException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

                at org.apache.ignite.internal.processors.cache.GridCacheUtils.validateCacheGroupsAttributesMismatch(GridCacheUtils.java:1052)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.validateCacheGroupConfiguration(ClusterCachesInfo.java:1965)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onStart(ClusterCachesInfo.java:152)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnStart(GridCacheProcessor.java:762)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.onReadyForRead(GridCacheProcessor.java:737)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetastorageReadyForRead(GridCacheDatabaseSharedManager.java:409)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:675)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:4730)

                at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1048)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)

                at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)

                at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1076)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:962)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:861)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:731)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:700)

                at org.apache.ignite.Ignition.start(Ignition.java:348)

                at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)

[21:54:10] Ignite node stopped OK [uptime=00:00:03.043]

class org.apache.ignite.IgniteException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

                at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1029)

                at org.apache.ignite.Ignition.start(Ignition.java:351)

                at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)

Caused by: class org.apache.ignite.IgniteCheckedException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

                at org.apache.ignite.internal.processors.cache.GridCacheUtils.validateCacheGroupsAttributesMismatch(GridCacheUtils.java:1052)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.validateCacheGroupConfiguration(ClusterCachesInfo.java:1965)

                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onStart(ClusterCachesInfo.java:152)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnStart(GridCacheProcessor.java:762)

                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.onReadyForRead(GridCacheProcessor.java:737)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetastorageReadyForRead(GridCacheDatabaseSharedManager.java:409)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:675)

                at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:4730)

                at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1048)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)

                at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)

                at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)

                at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1076)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:962)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:861)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:731)

                at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:700)

                at org.apache.ignite.Ignition.start(Ignition.java:348)

                ... 1 more

Failed to start grid: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

 

Here is the configuration file used at start up:

 

<?xml version="1.0" encoding="UTF-8"?>

 

<!--

  Licensed to the Apache Software Foundation (ASF) under one or more

  contributor license agreements.  See the NOTICE file distributed with

  this work for additional information regarding copyright ownership.

  The ASF licenses this file to You under the Apache License, Version 2.0

  (the "License"); you may not use this file except in compliance with

  the License.  You may obtain a copy of the License at

 

       http://www.apache.org/licenses/LICENSE-2.0

 

  Unless required by applicable law or agreed to in writing, software

  distributed under the License is distributed on an "AS IS" BASIS,

  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  See the License for the specific language governing permissions and

  limitations under the License.

-->

 

<!--

    Ignite Spring configuration file to startup Ignite cache.

 

    This file demonstrates how to configure cache using Spring. Provided cache

   will be created on node startup.

 

    Use this configuration file when running HTTP REST examples (see 'examples/rest' folder).

 

    When starting a standalone node, you need to execute the following command:

    {IGNITE_HOME}/bin/ignite.{bat|sh} examples/config/example-cache.xml

 

    When starting Ignite from Java IDE, pass path to this file to Ignition:

    Ignition.start("examples/config/example-cache.xml");

-->

<beans xmlns="http://www.springframework.org/schema/beans"

       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

       xsi:schemaLocation="

        http://www.springframework.org/schema/beans

        http://www.springframework.org/schema/beans/spring-beans.xsd">

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

 

        <property name="dataStorageConfiguration">

            <bean class="org.apache.ignite.configuration.DataStorageConfiguration">

                <property name="defaultDataRegionConfiguration">

                    <bean class="org.apache.ignite.configuration.DataRegionConfiguration">

                        <property name="persistenceEnabled" value="true"/>

                        <property name="name" value="data_region_data_loom"/>

 

                        <!-- initial size -->

                        <property name="initialSize" value="#{10L * 1024 * 1024 * 1024}"/>

                        <!-- max size -->

                        <property name="maxSize" value="#{50L * 1024 * 1024 * 1024}"/>

                       <!-- Enabling RANDOM_2_LRU eviction for this region.  -->

                        <property name="pageEvictionMode" value="RANDOM_2_LRU"/>

                    </bean>

                </property>

 

                <!-- Set the page size -->

                <property name="pageSize" value="#{16L * 1024}"/>

 

                <!-- Changing WAL Mode.-->

                <property name="walMode" value="FSYNC"/>

                <!-- Enable write throttling. -->

                <property name="writeThrottlingEnabled" value="true"/>

 

                <!--  WAL Archiving -->

                <property name="walPath" value="/var/lib/apache-ignite-wal"/>

                <property name="walArchivePath" value="/var/lib/apache-ignite-wal/archive"/>

            </bean>

        </property>

 

        <property name="cacheConfiguration">

            <list>

                <!-- Partitioned cache example configuration (Atomic mode). -->

                <bean class="org.apache.ignite.configuration.CacheConfiguration">

                    <property name="name" value="default"/>

                    <property name="atomicityMode" value="ATOMIC"/>

                    <property name="backups" value="0"/>

                </bean>

            </list>

        </property>

 

        <!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->

        <property name="discoverySpi">

            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

                <property name="ipFinder">

                    <!--

                        Ignite provides several options for automatic discovery that can be used

                        instead os static IP based discovery. For information on all options refer

                        to our documentation: http://apacheignite.readme.io/docs/cluster-config

                    -->

                    <!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->

                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">

                    <!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">-->

                        <property name="addresses">

                            <list>

                                <!--

                                Explicitly specifying address of a local node to let it start and

                                operate normally even if there is no more nodes in the cluster.

                                You can also optionally specify an individual port or port range.

                                -->

                                <value>10.207.86.89</value>

 

                                <!--

                                IP Address and optional port range of a remote node.

                                You can also optionally specify an individual port.

                                -->

                                <value>10.207.86.99:47500..47509</value>

                                <value>10.207.86.51:47500..47509</value>

                            </list>

                        </property>

                    </bean>

                </property>

            </bean>

        </property>

    </bean>

</beans>

 

Thanks,

Conrad

 

 

From: Evgeniy Rudenko <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Monday, October 21, 2019 at 3:10 AM
To: "[hidden email]" <[hidden email]>
Subject: Re: Partition Loss Policy mismatch blocks server restart

 

Hi Conrad

 

All caches in the group should have the same partitionLossPolicy. Cache with different partitionLossPolicy should not be allowed to join the group.

 

Could you tell what version of Ignite you are using. Also could you attach full logs and full xml configurations to check.

 

On Mon, Oct 21, 2019 at 1:16 AM Conrad Mukai (cmukai) <[hidden email]> wrote:

I set up a cluster of server nodes with the following cacheConfiguration:

 

        <property name="cacheConfiguration">

            <list>

                <!-- Partitioned cache example configuration (Atomic mode). -->

                <bean class="org.apache.ignite.configuration.CacheConfiguration">

                    <property name="name" value="default"/>

                    <property name="atomicityMode" value="ATOMIC"/>

                    <property name="backups" value="0"/>

                </bean>

            </list>

        </property>

 

Apparently an application uploaded a client configuration with a cache group and now I cannot restart the cluster with the original configuration. I get the following error:

 

Caused by: class org.apache.ignite.IgniteCheckedException: Partition Loss Policy mismatch for caches related to the same group [groupName=group_data_loom, existingCache=AlphaCaseTelProtobufCache1, existingPartitionLossPolicy=READ_WRITE_SAFE, startingCache=AlphaCaseTelProtobufCache, startingPartitionLossPolicy=IGNORE]

 

First question is how can I restart my cluster. The second question is this really due to the client configuration, and if so why is it possible for a client to break the entire service restart?

 

Thanks in advance,

Conrad

 



--

Best regards,
Evgeniy



--

Best regards,
Evgeniy



--
Best regards,
Evgeniy