Ignite transaction recovery on third-party persistence

classic Classic list List threaded Threaded
3 messages Options
bijunathg bijunathg
Reply | Threaded
Open this post in threaded view
|

Ignite transaction recovery on third-party persistence

Hi,

We have strict transactional requirements to query and update some cached
data. Our queries are mostly SQL and so we fully load data at application
startup. Our cached data is huge so we use a partitioned cache for
scalability.

Also for some BI applications, we need to write the data into an RDBMS
persistent store and make it available in a consistent manner. So we are
planning to use Ignite's write-through mode which updates both the DB and
cache atomically within an Ignite transaction.

But while exploring the Ignite documentation on third-party persistence and
transaction management
(https://www.gridgain.com/resources/blog/apache-ignite-transactions-architecture-transaction-handling-level-3rd-party),
looks like transaction failure & recovery scenarios (like DB write
successful, but cache commit failed) work only if Native persistence is
enabled. Since we need SQL queries, the read-through approach may not work,
as per our understanding.

Should we enable both Ignite native persistence for High availability and
third-party persistence for our BI requirements? Will this ensure that the
SQL query results are always consistent?

Thanks,



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Ignite transaction recovery on third-party persistence

Hello!

Transaction failure and recovery scenarios have no relation with Native Persistence. Transactional safety is guaranteed irrespective of Native persistence setting.

However, since Cache Store does not support two phase commit, there may be divergence between Native Persistence and Cache Store, by definition.

Regards,
--
Ilya Kasnacheev


вт, 3 сент. 2019 г. в 14:36, bijunathg <[hidden email]>:
Hi,

We have strict transactional requirements to query and update some cached
data. Our queries are mostly SQL and so we fully load data at application
startup. Our cached data is huge so we use a partitioned cache for
scalability.

Also for some BI applications, we need to write the data into an RDBMS
persistent store and make it available in a consistent manner. So we are
planning to use Ignite's write-through mode which updates both the DB and
cache atomically within an Ignite transaction.

But while exploring the Ignite documentation on third-party persistence and
transaction management
(https://www.gridgain.com/resources/blog/apache-ignite-transactions-architecture-transaction-handling-level-3rd-party),
looks like transaction failure & recovery scenarios (like DB write
successful, but cache commit failed) work only if Native persistence is
enabled. Since we need SQL queries, the read-through approach may not work,
as per our understanding.

Should we enable both Ignite native persistence for High availability and
third-party persistence for our BI requirements? Will this ensure that the
SQL query results are always consistent?

Thanks,



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
dmagda dmagda
Reply | Threaded
Open this post in threaded view
|

Re: Ignite transaction recovery on third-party persistence

Minor clarification. The updates will be fully consistent in case if a transaction commits to Ignite Memory + RBDMS or Ignite Memory + Native Persistence. There are no guarantees if the transaction updates both RDBMS or Native Persistence.

-
Denis


On Thu, Sep 5, 2019 at 7:03 AM Ilya Kasnacheev <[hidden email]> wrote:
Hello!

Transaction failure and recovery scenarios have no relation with Native Persistence. Transactional safety is guaranteed irrespective of Native persistence setting.

However, since Cache Store does not support two phase commit, there may be divergence between Native Persistence and Cache Store, by definition.

Regards,
--
Ilya Kasnacheev


вт, 3 сент. 2019 г. в 14:36, bijunathg <[hidden email]>:
Hi,

We have strict transactional requirements to query and update some cached
data. Our queries are mostly SQL and so we fully load data at application
startup. Our cached data is huge so we use a partitioned cache for
scalability.

Also for some BI applications, we need to write the data into an RDBMS
persistent store and make it available in a consistent manner. So we are
planning to use Ignite's write-through mode which updates both the DB and
cache atomically within an Ignite transaction.

But while exploring the Ignite documentation on third-party persistence and
transaction management
(https://www.gridgain.com/resources/blog/apache-ignite-transactions-architecture-transaction-handling-level-3rd-party),
looks like transaction failure & recovery scenarios (like DB write
successful, but cache commit failed) work only if Native persistence is
enabled. Since we need SQL queries, the read-through approach may not work,
as per our understanding.

Should we enable both Ignite native persistence for High availability and
third-party persistence for our BI requirements? Will this ensure that the
SQL query results are always consistent?

Thanks,



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/