Apache Ignite Users

Ignite native persistence

classic Classic list List threaded Threaded
♦
♦
 9 messages Options
niamin niamin
Reply | Threaded
Open this post in threaded view
♦
♦
|

Ignite native persistence

What is the right way to enable native persistence? As per Ignite
documentation, I added the following property to enable native persistence:
<property name="dataStorageConfiguration">
                    <bean
class="org.apache.ignite.configuration.DataStorageConfiguration">
                      <property name="defaultDataRegionConfiguration">
                        <bean
class="org.apache.ignite.configuration.DataRegionConfiguration">
                          <property name="persistenceEnabled" value="true"/>
                        </bean>
                      </property>
                    </bean>
                  </property>

I also added ignite.start(true) to my loader. However, I keep getting the
following WARN level message:

2019-11-25 12:52:26.709  WARN 16599 --- [           main]
o.a.i.i.p.c.store.CacheOsStoreManager    : <FOO> Calling Cache.loadCache()
method will have no effect, CacheConfiguration.getStore() is not defined for
cache: FOO


The solution works as an in-memory cache/solution though.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
dmagda dmagda
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Ignite native persistence

Hi there,

Looks like you are trying to preload data from the persistence to memory with Cache.loadCache method. Presently, this operation is not supported for the native persistence and it’s optional - just start issuing your queries after the cluster restart and Ignite will get all the data needed for you from disk. Eventually all the data will be loaded in RAM while your app is running and querying the cluster.

Does it answer your question?

On Monday, November 25, 2019, niamin <[hidden email]> wrote:
What is the right way to enable native persistence? As per Ignite
documentation, I added the following property to enable native persistence:
<property name="dataStorageConfiguration">
                    <bean
class="org.apache.ignite.configuration.DataStorageConfiguration">
                      <property name="defaultDataRegionConfiguration">
                        <bean
class="org.apache.ignite.configuration.DataRegionConfiguration">
                          <property name="persistenceEnabled" value="true"/>
                        </bean>
                      </property>
                    </bean>
                  </property>

I also added ignite.start(true) to my loader. However, I keep getting the
following WARN level message:

2019-11-25 12:52:26.709  WARN 16599 --- [           main]
o.a.i.i.p.c.store.CacheOsStoreManager    : <FOO> Calling Cache.loadCache()
method will have no effect, CacheConfiguration.getStore() is not defined for
cache: FOO


The solution works as an in-memory cache/solution though.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/


--
-
Denis
niamin niamin
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Ignite native persistence

Thanaks Denis. However, how do or should I preload the data to disk first? Or
as Ignite fetches data from the 3rd part datastore it is expected to
distribute between RAM and disk? Is there an example project that implements
this strategy? On my POC, I have SpringBoot project that includes the Ignite
Server and I was using SpringDataRepoository to configure the Queries which
were executed by Ignite server.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
niamin niamin
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Ignite native persistence

In reply to this post by niamin
I found what I was doing wrong. I can now load/pre-fetch from 3rd party store
to cache. It looks like the data is present in in-memory cache but also
stored in the persistent store. However, when I run a native query
configured on my SpringData Repository I get an error that the SQL table
mentioned in the query is not found. The same query/api was working when
persistent store was not enabled.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
GB GB
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Ignite native persistence

Please send exception/error message you are getting and the query you are trying to execute.


On Tue, Nov 26, 2019, 8:04 PM niamin <[hidden email]> wrote:
I found what I was doing wrong. I can now load/pre-fetch from 3rd party store
to cache. It looks like the data is present in in-memory cache but also
stored in the persistent store. However, when I run a native query
configured on my SpringData Repository I get an error that the SQL table
mentioned in the query is not found. The same query/api was working when
persistent store was not enabled.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
niamin niamin
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Ignite native persistence

ignite_sql_error.ignite_sql_error
<http://apache-ignite-users.70518.x6.nabble.com/file/t2637/ignite_sql_error.ignite_sql_error>  

ARInvoiceRepository.java
<http://apache-ignite-users.70518.x6.nabble.com/file/t2637/ARInvoiceRepository.java>  

Attached is the file containing statcktrace and the repository file. The
test case is trying to executed the
getInvoicesForOIStatement method.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
dmagda dmagda
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Ignite native persistence

You might need to set the cache name as a schema for the queries. For instance, the query needs to be transformed to "Select ... {YOUR_CACHE_NAME}.ARInvoice FROM ..".

Also, do you really need to have both 3rd party and native persistence enabled for the cluster. If the 3rd party is only used as an initial data source during the loading phase then you can use can create a simple custom loader that pulls records from the 3rd party and loads them into Ignite via IgniteDataStreamer:


-
Denis


On Tue, Nov 26, 2019 at 11:44 AM niamin <[hidden email]> wrote:
ignite_sql_error.ignite_sql_error
<http://apache-ignite-users.70518.x6.nabble.com/file/t2637/ignite_sql_error.ignite_sql_error> 

ARInvoiceRepository.java
<http://apache-ignite-users.70518.x6.nabble.com/file/t2637/ARInvoiceRepository.java> 

Attached is the file containing statcktrace and the repository file. The
test case is trying to executed the
getInvoicesForOIStatement method.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
niamin niamin
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Ignite native persistence

Tried with appending cache name as schema with no success. Attach is a link
to my project:
https://drive.google.com/open?id=1F53um8TeUK45U3SOW0_Vlj04S8DWyjKI

Please take a look. I suspect it is something in my code.

Thanks,
Naushad



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Ignite native persistence

Hello!

When running your reproducer, I don't see any "method will have no effect" messages, instead I can see in debugger that loadCache() is triggered and then com.gpc.rpm.IgniteCacheLoader.datagrid.store.spring.CacheSpringARInvoiceStore2#loadCache is called.

Can you please describe the problem in more detail?

Regards,
--
Ilya Kasnacheev


пн, 2 дек. 2019 г. в 23:46, niamin <[hidden email]>:
Tried with appending cache name as schema with no success. Attach is a link
to my project:
https://drive.google.com/open?id=1F53um8TeUK45U3SOW0_Vlj04S8DWyjKI

Please take a look. I suspect it is something in my code.

Thanks,
Naushad



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Free forum by Nabble Edit this page