Size of Data(Keys) are doubled when using CACHE in REPLICATED mode

classic Classic list List threaded Threaded
8 messages Options
Ali Bagewadi Ali Bagewadi
Reply | Threaded
Open this post in threaded view
|

Size of Data(Keys) are doubled when using CACHE in REPLICATED mode

Hello,
1-I have two servers one on local and other on remote host.
Both nodes are added to Baseline Topology.
2-I have started the cache in replicated mode on both the server nodes.
The data is replicated on both nodes properly however when both nodes are
running the size of data in node shown is doubled.
e.g If we add one key to local node and one key to remote node
expectations: Size of data in node should be 2 when both servers are running
actual: Size of data in node shows 4.

Please suggest if my understanding is correct for REPLICATED mode.

Below are the config files:

Config file for local node:

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/util
        http://www.springframework.org/schema/util/spring-util.xsd">
    <bean id="grid.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">
       
        <property name="peerClassLoadingEnabled" value="true"/>
     
     
       <property name="dataStorageConfiguration">
           <bean
class="org.apache.ignite.configuration.DataStorageConfiguration">
               <property name="defaultDataRegionConfiguration">
                   <bean
class="org.apache.ignite.configuration.DataRegionConfiguration">
                           <property name="persistenceEnabled" value="true"/>
                           <property name="initialSize" value="#{100L * 1024
* 1024}"/>
                           <property name="maxSize" value="#{400L * 1024 *
1024}"/>
                   </bean>
               </property>
           <property name="pageSize" value="#{4 * 1024}"/>
           <property name="storagePath"
value="/media/linux_hdd/keymanagementdata"/>
           <property name="walPath"
value="/media/linux_hdd/keymanagementdata/db/wal"/>
           <property name="walArchivePath"
value="/media/linux_hdd/keymanagementdata/db/wal/archive"/>
           </bean>
       </property>

        <property name="cacheConfiguration">
            <bean
class="org.apache.ignite.configuration.CacheConfiguration">
               
                <property name="name" value="DSU_DataBase"/>
               
                <property name="cacheMode" value="REPLICATED"/>
               
            </bean>
        </property>


       
        <property name="discoverySpi">
            <bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                   
                   
                   
                    <bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                        <property name="addresses">
                            <list>
                               
                                <value>127.0.0.1</value>
                                <value>192.168.111.112:7912</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
</beans>


Config file for remote node:


<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/util
        http://www.springframework.org/schema/util/spring-util.xsd">
    <bean id="grid.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">
       
        <property name="peerClassLoadingEnabled" value="true"/>
     
     
       <property name="dataStorageConfiguration">
           <bean
class="org.apache.ignite.configuration.DataStorageConfiguration">
               <property name="defaultDataRegionConfiguration">
                   <bean
class="org.apache.ignite.configuration.DataRegionConfiguration">
                           <property name="persistenceEnabled" value="true"/>
                           <property name="initialSize" value="#{100L * 1024
* 1024}"/>
                           <property name="maxSize" value="#{400L * 1024 *
1024}"/>
                   </bean>
               </property>
           <property name="pageSize" value="#{4 * 1024}"/>
           <property name="storagePath"
value="/media/linux_hdd/keymanagementdata"/>
           <property name="walPath"
value="/media/linux_hdd/keymanagementdata/db/wal"/>
           <property name="walArchivePath"
value="/media/linux_hdd/keymanagementdata/db/wal/archive"/>
           </bean>
       </property>

        <property name="cacheConfiguration">
            <bean
class="org.apache.ignite.configuration.CacheConfiguration">
               
                <property name="name" value="DSU_DataBase"/>
               
                <property name="cacheMode" value="REPLICATED"/>
               
            </bean>
        </property>


       
        <property name="discoverySpi">
            <bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                   
                   
                   
                    <bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                        <property name="addresses">
                            <list>
                               
                                <value>127.0.0.1</value>
                                <value>192.168.111.111:7912</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
</beans>






--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Size of Data(Keys) are doubled when using CACHE in REPLICATED mode

Hello!

How do you calculate the "size of the data"?

Regards,
--
Ilya Kasnacheev


пт, 14 авг. 2020 г. в 09:06, Ali Bagewadi <[hidden email]>:
Hello,
1-I have two servers one on local and other on remote host.
Both nodes are added to Baseline Topology.
2-I have started the cache in replicated mode on both the server nodes.
The data is replicated on both nodes properly however when both nodes are
running the size of data in node shown is doubled.
e.g If we add one key to local node and one key to remote node
expectations: Size of data in node should be 2 when both servers are running
actual: Size of data in node shows 4.

Please suggest if my understanding is correct for REPLICATED mode.

Below are the config files:

Config file for local node:

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/util
        http://www.springframework.org/schema/util/spring-util.xsd">
    <bean id="grid.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="peerClassLoadingEnabled" value="true"/>


       <property name="dataStorageConfiguration">
           <bean
class="org.apache.ignite.configuration.DataStorageConfiguration">
               <property name="defaultDataRegionConfiguration">
                   <bean
class="org.apache.ignite.configuration.DataRegionConfiguration">
                           <property name="persistenceEnabled" value="true"/>
                           <property name="initialSize" value="#{100L * 1024
* 1024}"/>
                           <property name="maxSize" value="#{400L * 1024 *
1024}"/>
                   </bean>
               </property>
           <property name="pageSize" value="#{4 * 1024}"/>
           <property name="storagePath"
value="/media/linux_hdd/keymanagementdata"/>
           <property name="walPath"
value="/media/linux_hdd/keymanagementdata/db/wal"/>
           <property name="walArchivePath"
value="/media/linux_hdd/keymanagementdata/db/wal/archive"/>
           </bean>
       </property>

        <property name="cacheConfiguration">
            <bean
class="org.apache.ignite.configuration.CacheConfiguration">

                <property name="name" value="DSU_DataBase"/>

                <property name="cacheMode" value="REPLICATED"/>

            </bean>
        </property>



        <property name="discoverySpi">
            <bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">



                    <bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                        <property name="addresses">
                            <list>

                                <value>127.0.0.1</value>
                                <value>192.168.111.112:7912</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
</beans>


Config file for remote node:


<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/util
        http://www.springframework.org/schema/util/spring-util.xsd">
    <bean id="grid.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="peerClassLoadingEnabled" value="true"/>


       <property name="dataStorageConfiguration">
           <bean
class="org.apache.ignite.configuration.DataStorageConfiguration">
               <property name="defaultDataRegionConfiguration">
                   <bean
class="org.apache.ignite.configuration.DataRegionConfiguration">
                           <property name="persistenceEnabled" value="true"/>
                           <property name="initialSize" value="#{100L * 1024
* 1024}"/>
                           <property name="maxSize" value="#{400L * 1024 *
1024}"/>
                   </bean>
               </property>
           <property name="pageSize" value="#{4 * 1024}"/>
           <property name="storagePath"
value="/media/linux_hdd/keymanagementdata"/>
           <property name="walPath"
value="/media/linux_hdd/keymanagementdata/db/wal"/>
           <property name="walArchivePath"
value="/media/linux_hdd/keymanagementdata/db/wal/archive"/>
           </bean>
       </property>

        <property name="cacheConfiguration">
            <bean
class="org.apache.ignite.configuration.CacheConfiguration">

                <property name="name" value="DSU_DataBase"/>

                <property name="cacheMode" value="REPLICATED"/>

            </bean>
        </property>



        <property name="discoverySpi">
            <bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">



                    <bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                        <property name="addresses">
                            <list>

                                <value>127.0.0.1</value>
                                <value>192.168.111.111:7912</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
</beans>






--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Ali Bagewadi Ali Bagewadi
Reply | Threaded
Open this post in threaded view
|

Re: Size of Data(Keys) are doubled when using CACHE in REPLICATED mode

Hello,

We get the size with below code.

    IgniteConfiguration cfg;
    cfg.springCfgPath = std::string("/home/dsudev/DataBaseConfig.xml");

    /* Start a node to access DataBase */
    Ignite node = Ignition::Start(cfg);
    node.SetActive(true);

    /* Get cache instance */
    mCache = node.GetOrCreateCache<tKey, DBStorage>(CACHE_NAME);

   /*Get size*/
   mcache.size(); ---->This gives us the size or no of records.


   Thank you,
   Ali.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Size of Data(Keys) are doubled when using CACHE in REPLICATED mode

Hello!

I do not recommend issuing setActive(true) after starting a node, this is an antipattern which can lead to problems.

I recommend only activating the cluster when it is ready, or allowing it to auto-activate itself.

Regards,
--
Ilya Kasnacheev


вт, 18 авг. 2020 г. в 19:56, Ali Bagewadi <[hidden email]>:
Hello,

We get the size with below code.

    IgniteConfiguration cfg;
    cfg.springCfgPath = std::string("/home/dsudev/DataBaseConfig.xml");

    /* Start a node to access DataBase */
    Ignite node = Ignition::Start(cfg);
    node.SetActive(true);

    /* Get cache instance */
    mCache = node.GetOrCreateCache<tKey, DBStorage>(CACHE_NAME);

   /*Get size*/
   mcache.size(); ---->This gives us the size or no of records.


   Thank you,
   Ali.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
rakshita04 rakshita04
Reply | Threaded
Open this post in threaded view
|

Re: Size of Data(Keys) are doubled when using CACHE in REPLICATED mode

What is auto-activate feature?
How does it work?
And for how long should we wait before calling setActive(true)?
Is there a way we can come to know whether cluster is activated or not?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Size of Data(Keys) are doubled when using CACHE in REPLICATED mode

Hello!

If you had a baseline topology before, and cluster was restarted, and then all nodes come back, they will be auto-activated.
By explicitly calling setActive(true) on incomplete cluster you interfere with that process.

You should call setActive(true) as soon as all server nodes (which you expect to have at the moment) have joined the cluster.

Regards,
--
Ilya Kasnacheev


чт, 20 авг. 2020 г. в 14:45, rakshita04 <[hidden email]>:
What is auto-activate feature?
How does it work?
And for how long should we wait before calling setActive(true)?
Is there a way we can come to know whether cluster is activated or not?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
rakshita04 rakshita04
Reply | Threaded
Open this post in threaded view
|

Re: Size of Data(Keys) are doubled when using CACHE in REPLICATED mode

Is there any API in C++ to know whether "all server nodes (which you expect
to have at the moment) have joined the cluster"??
name of the function would be great help.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
akorensh akorensh
Reply | Threaded
Open this post in threaded view
|

Re: Size of Data(Keys) are doubled when using CACHE in REPLICATED mode

Hi,
  When all baseline nodes have joined the cluster it will self-activate.
  After self-activation Ignite::isActive() will return true
https://ignite.apache.org/releases/latest/cppdoc/classignite_1_1Ignite.html#a5455dd6d8f8453073fbd32c60fe0c3af

When isActive() returns true it means, by defintion, that all baseline nodes
are online.


from the doc:
  When the cluster is activated for the first time, baseline topology is
automatically established from the current set of server nodes. Once this is
done, information about nodes that constitute the baseline topology is
persisted to disk. Then, even if you shutdown and restart the cluster, the
cluster will be activated automatically once all the nodes set in the
baseline topology are up and running.


https://apacheignite.readme.io/docs/baseline-topology#activating-the-cluster
Thanks, Alex



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/