Can Ignite transaction manage cached and non-cached data writes?

classic Classic list List threaded Threaded
6 messages Options
bijunathg bijunathg
Reply | Threaded
Open this post in threaded view
|

Can Ignite transaction manage cached and non-cached data writes?

Hi,
Our application wants to do SQL queries and writes on some cached data
(partitioned) and at the same time update some other non-cached data in the
same transactional context. We do not want to cache everything for
optimizing the cache memory footprint.

The data-store could be any RDBMS store. Both the cached data and non-cached
data are stored on the same DB schema (instance).

We can enable the Write-through mode for the cached data so that Ignite will
directly write to the store.
 Could any of you please advice the best practices to manage such a
transaction? How will we write the non-cached data in the same transactional
context? Does Ignite provide any provision to write the non-cached data also
to the underlying store within the same Ignite transaction?

Thanks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
aealexsandrov aealexsandrov
Reply | Threaded
Open this post in threaded view
|

Re: Can Ignite transaction manage cached and non-cached data writes?

Hi,

By default Ignite will handle only updates that were done using Ignite
cache API.

I guess that you have already read next article:

https://www.gridgain.com/resources/blog/apache-ignite-transactions-architecture-transaction-handling-level-3rd-party

BR,
Andrei

9/3/2019 2:03 PM, bijunathg пишет:

> Hi,
> Our application wants to do SQL queries and writes on some cached data
> (partitioned) and at the same time update some other non-cached data in the
> same transactional context. We do not want to cache everything for
> optimizing the cache memory footprint.
>
> The data-store could be any RDBMS store. Both the cached data and non-cached
> data are stored on the same DB schema (instance).
>
> We can enable the Write-through mode for the cached data so that Ignite will
> directly write to the store.
>   Could any of you please advice the best practices to manage such a
> transaction? How will we write the non-cached data in the same transactional
> context? Does Ignite provide any provision to write the non-cached data also
> to the underlying store within the same Ignite transaction?
>
> Thanks
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Uwe Geercken Uwe Geercken
Reply | Threaded
Open this post in threaded view
|

Aw: Re: Can Ignite transaction manage cached and non-cached data writes?

 
Hello,
 
my first post here for Apache Ignite...
 
I have a question and looking for some advice. I have data from a relational database which I will load into Ignite. Each table from the source system has a UID (number), to uniquely identify each record.
 
If I understand it correctly, then the key in a cache in Ignite must be unique. So the source data might have duplications in the UID across the tables which is ok in the RDBMS but not in Ignite.
 
Can you give me some advice of how this issue is handled best? My idea is to create a unique prefix for each source table and thus make the source UID also unique across the cache.
 
Or do you have a different recommendation? Due to being new to Ignite, it could also well be that I miss something.
 
Rgds,
 
Uwe
GB GB
Reply | Threaded
Open this post in threaded view
|

Re: Re: Can Ignite transaction manage cached and non-cached data writes?

Hello Uwe,

Why don't you create separate cache corresponding to each table in rdbms? 

Best Regards,
Gaurav

On Tue, Sep 3, 2019, 8:31 PM Uwe Geercken <[hidden email]> wrote:
 
Hello,
 
my first post here for Apache Ignite...
 
I have a question and looking for some advice. I have data from a relational database which I will load into Ignite. Each table from the source system has a UID (number), to uniquely identify each record.
 
If I understand it correctly, then the key in a cache in Ignite must be unique. So the source data might have duplications in the UID across the tables which is ok in the RDBMS but not in Ignite.
 
Can you give me some advice of how this issue is handled best? My idea is to create a unique prefix for each source table and thus make the source UID also unique across the cache.
 
Or do you have a different recommendation? Due to being new to Ignite, it could also well be that I miss something.
 
Rgds,
 
Uwe
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Can Ignite transaction manage cached and non-cached data writes?

In reply to this post by bijunathg
Hello!

I don't think you can put any data to Cache Store without actually putting it in a cache.

Regards,
--
Ilya Kasnacheev


вт, 3 сент. 2019 г. в 14:03, bijunathg <[hidden email]>:
Hi,
Our application wants to do SQL queries and writes on some cached data
(partitioned) and at the same time update some other non-cached data in the
same transactional context. We do not want to cache everything for
optimizing the cache memory footprint.

The data-store could be any RDBMS store. Both the cached data and non-cached
data are stored on the same DB schema (instance).

We can enable the Write-through mode for the cached data so that Ignite will
directly write to the store.
 Could any of you please advice the best practices to manage such a
transaction? How will we write the non-cached data in the same transactional
context? Does Ignite provide any provision to write the non-cached data also
to the underlying store within the same Ignite transaction?

Thanks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
bijunathg bijunathg
Reply | Threaded
Open this post in threaded view
|

Re: Can Ignite transaction manage cached and non-cached data writes?

In reply to this post by aealexsandrov
Hi,
Thanks for the response.
We found an option of passing an "application provided Hibernate session" to
Ignite and reuse the same for all the cache stores involved in the
transaction. This way Ignite will use the same Hibernate session for all the
cache updates within the transaction. This is by following the below
approach:
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/store/CacheStoreSessionListener.html

Since application is providing the Hibernate session to Ignite, application
can also add more elements (non-cached entries) to it before calling
Ignite.commit(). This seems to work fine. But we would like to know if this
approach has any concern w.r.t reliability/consistency in edge cases.

Thanks,
Bijunath



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/