not able to change keyspace name when Using 3rd Party Persistence (Cassandra) Together with Ignite Native Persistence

classic Classic list List threaded Threaded
2 messages Options
xmw45688 xmw45688
Reply | Threaded
Open this post in threaded view
|

not able to change keyspace name when Using 3rd Party Persistence (Cassandra) Together with Ignite Native Persistence

I'm able to use Cassandra Together with Ignite Persistence (see the example
below) usingIgnite 2.8.1.  However I could not change to another keyspace
which has the same tables as the keyspace initially configurfed in the
ignite config file.  The data changes are still stored in the old keyspace
instead of the new keyspace that are in ignite config file.  It's seem to me
that the keyspace are cached in the Ignite native persistence layer.  

How do I change the keyspaces of Cassandra?  

The use case is that we use Ignite as cache initially with the data stored
in Cassandra.  With the data increase, caching all data is impractical.  So
we plan to flush the data into the disk when the memory reaches the limits.
We still want to store the data into Cassandra at the same time.  How do we
load the the existing data from Cassandra Store to Native Persistence store
when switching from Ignite Cache + Cassandra Store to Ignite Cache +Ignite
persistence + Cassandra Store?
Here is what I did -
1) duplicated the existing tables (no data) into the another keyspace in
Cassandra
2) read the data from the original keyspace with Cassndra driver. This
loaded the data into
  2.1) ignite cache and native persistence
  2.2) store the data into the new Cassandra keyspace.  it's undesirable to
the store the original data back to the original keyspace.
3) change the cassandra connection setting to switch to the original
keyspace because Ignite Cache and Native Persistence layer have all data
from the Cassandra.
  3.1) switching to the original cassandra keyspace did not take any effect.
The data changes were still stored into the new keyspace.  

Config files
1. cassandra-connection-settings.xml
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd">

    <bean id="loadBalancingPolicy"
class="com.datastax.driver.core.policies.TokenAwarePolicy">
        <constructor-arg
type="com.datastax.driver.core.policies.LoadBalancingPolicy">
            <bean
class="com.datastax.driver.core.policies.RoundRobinPolicy"/>
        </constructor-arg>
    </bean>
        <bean id="reconnectionPolicy"
           
class="com.datastax.driver.core.policies.ExponentialReconnectionPolicy">
            <constructor-arg name="baseDelayMs" value="100" />
            <constructor-arg name="maxDelayMs" value="6000000" />
        </bean>

        <bean id="retryPolicy"
            class="com.datastax.driver.core.policies.DefaultRetryPolicy">
        </bean>
       
               
    <bean
class="org.apache.ignite.cache.store.cassandra.datasource.DataSource"
name="cassandra">
        <property name="contactPoints" value="127.0.0.1:9042"/>
        <property name="user" value="cassandra"/>
        <property name="password" value="cassandra"/>
        <property name="readConsistency" value="ONE"/>
        <property name="writeConsistency" value="ONE"/>
        <property name="loadBalancingPolicy" ref="loadBalancingPolicy"/>
        <property name="reconnectionPolicy"  ref="reconnectionPolicy"/>
        <property name="retryPolicy"         ref="retryPolicy"/>
    </bean>

</beans>

2. dim_store_key_persistence.xm
<persistence keyspace="ignitetest" table="dim_store">
    <keyPersistence class="com.procurant.model.IdKey" strategy="POJO">
                <partitionKey>
            <field name="id" column="id" />
        </partitionKey>    
    </keyPersistence>
    <valuePersistence class="com.procurant.model.DimStore" strategy="POJO" >  
    <field name="id" column="id" />
    <field name="name" column="name" />
    <field name="zip" column="zip" />
    <field name="addr" column="addr" />
    </valuePersistence>
</persistence>

3. ignite config
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">

       
   
   
        <import resource="classpath:cassandra-connection-settings.xml" />
        <import resource="classpath:ignite-discoverySpi-settings.xml" />

   
    <bean id="dim_store_persistence_settings"
class="org.apache.ignite.cache.store.cassandra.persistence.KeyValuePersistenceSettings">
        <constructor-arg type="org.springframework.core.io.Resource"
value="classpath:dim_store_key_persistence.xml" />
    </bean>

   
    <bean class="org.apache.ignite.configuration.IgniteConfiguration">
               
                <property name="igniteInstanceName" value="MyCluster"/>
            <property name="peerClassLoadingEnabled" value="true"/>


                <property name="workDirectory"
value="/c:/apache/data/home/work-cassandra"/>
                       
        <property name="dataStorageConfiguration">
            <bean
class="org.apache.ignite.configuration.DataStorageConfiguration">
                       
                <property name="defaultDataRegionConfiguration">
                    <bean
class="org.apache.ignite.configuration.DataRegionConfiguration">
                        <property name="name" value="Default_Region"/>
                        <property name="initialSize" value="#{10L * 1024 *
1024}"/>
                       
                        <property name="maxSize" value="#{4L * 1024 * 1024 *
1024}"/>
                       
           
                        <property name="persistenceEnabled" value="true"/>
                    </bean>
                </property>
                                           
                <property name="storagePath"
value="c:/apache/data/simple-cassandra/storage"/>
                <property name="walFlushFrequency" value="10000"/>
                <property name="walMode" value="LOG_ONLY"/>
                <property name="walPath"      
value="c:/apache/data/simple-cassandra/wal"/>
                <property name="walArchivePath"
value="c:/apache/data/simple-cassandra/wal/archive"/>                                                          
            </bean>
        </property>
       
        <property name="cacheConfiguration">
            <list>


                <bean
class="org.apache.ignite.configuration.CacheConfiguration">
                    <property name="name" value="DimStore"/>
                    <property name="cacheMode" value="REPLICATED"/>
                    <property name="atomicityMode" value="TRANSACTIONAL"/>                  
                    <property name="readThrough" value="true"/>
                    <property name="writeThrough" value="true"/>                    
                    <property name="sqlSchema" value="partition"/>
                    <property name="indexedTypes">
                        <list>
                            <value
type="java.lang.Class">com.procurant.model.IdKey</value>
                            <value
type="java.lang.Class">com.procurant.model.DimStore</value>
                        </list>
                    </property>
                   
                    <property name="cacheStoreFactory">
                        <bean
class="org.apache.ignite.cache.store.cassandra.CassandraCacheStoreFactory">
                            <property name="dataSource" ref="cassandra"/>
                            <property name="persistenceSettingsBean"
value="dim_store_persistence_settings"></property>
                        </bean>
                    </property>
               
                </bean>
               
            </list>
        </property>
       
        <property name="discoverySpi" >
            <ref bean="tcpDiscoverySpi"/>
                </property>
               

           
    </bean>
       
</beans>




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: not able to change keyspace name when Using 3rd Party Persistence (Cassandra) Together with Ignite Native Persistence

Hello!

I'm afraid that you may need to drop and recreate these caches to change their configuration, including Cache Store config.

Regards,
--
Ilya Kasnacheev


пн, 28 сент. 2020 г. в 00:27, xmw45688 <[hidden email]>:
I'm able to use Cassandra Together with Ignite Persistence (see the example
below) usingIgnite 2.8.1.  However I could not change to another keyspace
which has the same tables as the keyspace initially configurfed in the
ignite config file.  The data changes are still stored in the old keyspace
instead of the new keyspace that are in ignite config file.  It's seem to me
that the keyspace are cached in the Ignite native persistence layer. 

How do I change the keyspaces of Cassandra? 

The use case is that we use Ignite as cache initially with the data stored
in Cassandra.  With the data increase, caching all data is impractical.  So
we plan to flush the data into the disk when the memory reaches the limits.
We still want to store the data into Cassandra at the same time.  How do we
load the the existing data from Cassandra Store to Native Persistence store
when switching from Ignite Cache + Cassandra Store to Ignite Cache +Ignite
persistence + Cassandra Store?
Here is what I did -
1) duplicated the existing tables (no data) into the another keyspace in
Cassandra
2) read the data from the original keyspace with Cassndra driver. This
loaded the data into
  2.1) ignite cache and native persistence
  2.2) store the data into the new Cassandra keyspace.  it's undesirable to
the store the original data back to the original keyspace.
3) change the cassandra connection setting to switch to the original
keyspace because Ignite Cache and Native Persistence layer have all data
from the Cassandra.
  3.1) switching to the original cassandra keyspace did not take any effect.
The data changes were still stored into the new keyspace.   

Config files
1. cassandra-connection-settings.xml
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd">

    <bean id="loadBalancingPolicy"
class="com.datastax.driver.core.policies.TokenAwarePolicy">
        <constructor-arg
type="com.datastax.driver.core.policies.LoadBalancingPolicy">
            <bean
class="com.datastax.driver.core.policies.RoundRobinPolicy"/>
        </constructor-arg>
    </bean>
        <bean id="reconnectionPolicy"

class="com.datastax.driver.core.policies.ExponentialReconnectionPolicy">
            <constructor-arg name="baseDelayMs" value="100" />
            <constructor-arg name="maxDelayMs" value="6000000" />
        </bean>

        <bean id="retryPolicy"
            class="com.datastax.driver.core.policies.DefaultRetryPolicy">
        </bean>


    <bean
class="org.apache.ignite.cache.store.cassandra.datasource.DataSource"
name="cassandra">
        <property name="contactPoints" value="127.0.0.1:9042"/>
        <property name="user" value="cassandra"/>
        <property name="password" value="cassandra"/>
        <property name="readConsistency" value="ONE"/>
        <property name="writeConsistency" value="ONE"/>
        <property name="loadBalancingPolicy" ref="loadBalancingPolicy"/>
        <property name="reconnectionPolicy"  ref="reconnectionPolicy"/>
        <property name="retryPolicy"         ref="retryPolicy"/>
    </bean>

</beans>

2. dim_store_key_persistence.xm
<persistence keyspace="ignitetest" table="dim_store">
    <keyPersistence class="com.procurant.model.IdKey" strategy="POJO">
                <partitionKey>
            <field name="id" column="id" />
        </partitionKey>   
    </keyPersistence>
    <valuePersistence class="com.procurant.model.DimStore" strategy="POJO" > 
        <field name="id" column="id" />
        <field name="name" column="name" />
        <field name="zip" column="zip" />
        <field name="addr" column="addr" />
    </valuePersistence>
</persistence>

3. ignite config
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">




        <import resource="classpath:cassandra-connection-settings.xml" />       
        <import resource="classpath:ignite-discoverySpi-settings.xml" />


    <bean id="dim_store_persistence_settings"
class="org.apache.ignite.cache.store.cassandra.persistence.KeyValuePersistenceSettings">
        <constructor-arg type="org.springframework.core.io.Resource"
value="classpath:dim_store_key_persistence.xml" />
    </bean>


    <bean class="org.apache.ignite.configuration.IgniteConfiguration">

                <property name="igniteInstanceName" value="MyCluster"/>
            <property name="peerClassLoadingEnabled" value="true"/>


                <property name="workDirectory"
value="/c:/apache/data/home/work-cassandra"/>

        <property name="dataStorageConfiguration">
            <bean
class="org.apache.ignite.configuration.DataStorageConfiguration">

                <property name="defaultDataRegionConfiguration">
                    <bean
class="org.apache.ignite.configuration.DataRegionConfiguration">
                        <property name="name" value="Default_Region"/>
                        <property name="initialSize" value="#{10L * 1024 *
1024}"/>

                        <property name="maxSize" value="#{4L * 1024 * 1024 *
1024}"/>


                        <property name="persistenceEnabled" value="true"/>
                    </bean>
                </property>

                <property name="storagePath"
value="c:/apache/data/simple-cassandra/storage"/>
                <property name="walFlushFrequency" value="10000"/>
                <property name="walMode" value="LOG_ONLY"/>
                <property name="walPath"       
value="c:/apache/data/simple-cassandra/wal"/>
                <property name="walArchivePath"
value="c:/apache/data/simple-cassandra/wal/archive"/>                                                         
            </bean>
        </property>

        <property name="cacheConfiguration">
            <list>


                <bean
class="org.apache.ignite.configuration.CacheConfiguration">
                    <property name="name" value="DimStore"/>
                    <property name="cacheMode" value="REPLICATED"/>
                    <property name="atomicityMode" value="TRANSACTIONAL"/>                   
                    <property name="readThrough" value="true"/>
                    <property name="writeThrough" value="true"/>                   
                    <property name="sqlSchema" value="partition"/>
                    <property name="indexedTypes">
                        <list>
                            <value
type="java.lang.Class">com.procurant.model.IdKey</value>
                            <value
type="java.lang.Class">com.procurant.model.DimStore</value>
                        </list>
                    </property>

                    <property name="cacheStoreFactory">
                        <bean
class="org.apache.ignite.cache.store.cassandra.CassandraCacheStoreFactory">
                            <property name="dataSource" ref="cassandra"/>
                            <property name="persistenceSettingsBean"
value="dim_store_persistence_settings"></property>
                        </bean>
                    </property>

                </bean>

            </list>
        </property>

        <property name="discoverySpi" >
            <ref bean="tcpDiscoverySpi"/>
                </property>



    </bean>

</beans>




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/