3rd Party Persistence and two phase commit

classic Classic list List threaded Threaded
17 messages Options
Andrey Nestrogaev Andrey Nestrogaev
Reply | Threaded
Open this post in threaded view
|

3rd Party Persistence and two phase commit

Hi all!

We have a cache "Accounts".
We use 3rd Party Persistence with Write-Through mode

In the cache there are two accounts.
Account width id = 1 is located on node A and account with id = 2 is located
on node B.
In a single transaction we want to transfer the amount of $100 from account
1 to account 2.

How Ignite in the described case supports ACID transactions?
What happens if the CacheStore on node A successfully executes the commit in
the overrided method sessionEnd
and when the sessionEnd method is executed on node B, an error occurs and
the data is not commit in the database.

In this case, we get inconsistent data in the database, right?

Thanks!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Denis Mekhanikov Denis Mekhanikov
Reply | Threaded
Open this post in threaded view
|

Re: 3rd Party Persistence and two phase commit

Hi Andrey!

Actually, commit to the 3rd party database will be performed from the node, that initiated the transaction.
So, there will be only one transaction on the backing database.
Otherwise it is impossible to guarantee data consistency, as you noticed.

Denis

ср, 10 янв. 2018 г. в 18:58, Andrey Nestrogaev <[hidden email]>:
Hi all!

We have a cache "Accounts".
We use 3rd Party Persistence with Write-Through mode

In the cache there are two accounts.
Account width id = 1 is located on node A and account with id = 2 is located
on node B.
In a single transaction we want to transfer the amount of $100 from account
1 to account 2.

How Ignite in the described case supports ACID transactions?
What happens if the CacheStore on node A successfully executes the commit in
the overrided method sessionEnd
and when the sessionEnd method is executed on node B, an error occurs and
the data is not commit in the database.

In this case, we get inconsistent data in the database, right?

Thanks!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Denis Magda-2 Denis Magda-2
Reply | Threaded
Open this post in threaded view
|

Re: 3rd Party Persistence and two phase commit

Covered here:

Denis

On Jan 10, 2018, at 8:27 AM, Denis Mekhanikov <[hidden email]> wrote:

Hi Andrey!

Actually, commit to the 3rd party database will be performed from the node, that initiated the transaction.
So, there will be only one transaction on the backing database.
Otherwise it is impossible to guarantee data consistency, as you noticed.

Denis

ср, 10 янв. 2018 г. в 18:58, Andrey Nestrogaev <[hidden email]>:
Hi all!

We have a cache "Accounts".
We use 3rd Party Persistence with Write-Through mode

In the cache there are two accounts.
Account width id = 1 is located on node A and account with id = 2 is located
on node B.
In a single transaction we want to transfer the amount of $100 from account
1 to account 2.

How Ignite in the described case supports ACID transactions?
What happens if the CacheStore on node A successfully executes the commit in
the overrided method sessionEnd
and when the sessionEnd method is executed on node B, an error occurs and
the data is not commit in the database.

In this case, we get inconsistent data in the database, right?

Thanks!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Andrey Nestrogaev Andrey Nestrogaev
Reply | Threaded
Open this post in threaded view
|

Re: 3rd Party Persistence and two phase commit

In reply to this post by Denis Mekhanikov
Hi Denis, thanks for clarifying!

Did I understand you correctly that in any cases all interactions with 3rd
party database will occur only on the initial node and the sessionEnd
method will be called only once on the initial node?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Andrey Nestrogaev Andrey Nestrogaev
Reply | Threaded
Open this post in threaded view
|

Re: 3rd Party Persistence and two phase commit

In reply to this post by Denis Magda-2
Hi Denis,

Yes, I have already read the article you mentioned.

But it shows an example where the primary data being changed is located on
the same node, at least I understand it so.
In my original understanding it was that each node creates its own
connection to the 3rd database. But perhaps this works only in the case of
Read-Through, and with distributed transactions, another approach works.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ALEKSEY KUZNETSOV ALEKSEY KUZNETSOV
Reply | Threaded
Open this post in threaded view
|

Re: 3rd Party Persistence and two phase commit


Hi, Andrey!

Actually, data could be persisted not on tx initiating node, but on primary(I.e. we have partitioned cache and local cache) .

Additionally, data would be persisted on backup node if you enable the corresponding flag.

> 11 янв. 2018 г., в 10:12, Andrey Nestrogaev <[hidden email]> написал(а):
>
> Hi Denis,
>
> Yes, I have already read the article you mentioned.
>
> But it shows an example where the primary data being changed is located on
> the same node, at least I understand it so.
> In my original understanding it was that each node creates its own
> connection to the 3rd database. But perhaps this works only in the case of
> Read-Through, and with distributed transactions, another approach works.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Andrey Nestrogaev Andrey Nestrogaev
Reply | Threaded
Open this post in threaded view
|

Re: 3rd Party Persistence and two phase commit

Hi Aleksey, thanks for info,

"/Actually, data could be persisted not on tx initiating node, but on
primary(I.e. we have partitioned cache and local cache)/"
Ok, but no matter where the data is persisted, there will always be only 1
database connection within the transaction, no matter how many nodes are
involved in the transaction.
Right?

"/Additionally, data would be persisted on backup node if you enable the
corresponding flag./"
Would be persisted on backup node with using the same database connection as
for primary node?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ALEKSEY KUZNETSOV ALEKSEY KUZNETSOV
Reply | Threaded
Open this post in threaded view
|

Re: 3rd Party Persistence and two phase commit

Local store means store, that resides only on one node. No other nodes see it.

If you don't have local stores in cluster(only distributed ones), then it will be the only db connection within transaction opened. 
But If you have local stores, then nodes could open their own connections to local stores(i.e. in replicated cache nodes could open connections to local stores, if any).

Only local stores could be filled with data from backup nodes, therefore new connection must be opened(cannot reuse old one from primary node).


чт, 11 янв. 2018 г. в 13:48, Andrey Nestrogaev <[hidden email]>:
Hi Aleksey, thanks for info,

"/Actually, data could be persisted not on tx initiating node, but on
primary(I.e. we have partitioned cache and local cache)/"
Ok, but no matter where the data is persisted, there will always be only 1
database connection within the transaction, no matter how many nodes are
involved in the transaction.
Right?

"/Additionally, data would be persisted on backup node if you enable the
corresponding flag./"
Would be persisted on backup node with using the same database connection as
for primary node?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/


--

Best Regards,

Kuznetsov Aleksey

Denis Magda-2 Denis Magda-2
Reply | Threaded
Open this post in threaded view
|

Re: 3rd Party Persistence and two phase commit

Guys, let me step in and explain how it works with a 3rd party database like an RDBMS.

1. Write-through mode (all the changes are persisted right away).

Transaction coordinator commits a transaction at the RDBMS level first and only *then* commits it at the cluster level. This is actually what that blog is about. So, the transaction coordinator (your application) maintains a direct connection to the database.

If the transaction failed at the database level it won’t be committed on the cluster side at all.

2. Write-behind mode (the changes are persisted asynchronously).

Transaction coordinator commits a transaction at the cluster level and primary nodes will commit the changes asynchronously depending on the workload and settings of the write-behind store impl that writes to your database.

Hope this helps.

Denis

On Jan 11, 2018, at 3:35 AM, ALEKSEY KUZNETSOV <[hidden email]> wrote:

Local store means store, that resides only on one node. No other nodes see it.

If you don't have local stores in cluster(only distributed ones), then it will be the only db connection within transaction opened. 
But If you have local stores, then nodes could open their own connections to local stores(i.e. in replicated cache nodes could open connections to local stores, if any).

Only local stores could be filled with data from backup nodes, therefore new connection must be opened(cannot reuse old one from primary node).


чт, 11 янв. 2018 г. в 13:48, Andrey Nestrogaev <[hidden email]>:
Hi Aleksey, thanks for info,

"/Actually, data could be persisted not on tx initiating node, but on
primary(I.e. we have partitioned cache and local cache)/"
Ok, but no matter where the data is persisted, there will always be only 1
database connection within the transaction, no matter how many nodes are
involved in the transaction.
Right?

"/Additionally, data would be persisted on backup node if you enable the
corresponding flag./"
Would be persisted on backup node with using the same database connection as
for primary node?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/


--

Best Regards,

Kuznetsov Aleksey


vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|

Re: 3rd Party Persistence and two phase commit

Aleksey,

Are you talking about use case when a transaction spawns a distributed cache
AND a local cache? If so, this sounds like a very weird use case. Do we even
allow this?

-Val



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ALEKSEY KUZNETSOV ALEKSEY KUZNETSOV
Reply | Threaded
Open this post in threaded view
|

Re: 3rd Party Persistence and two phase commit

Hi!

I just wanted to show an example, when Write-through mode is enabled and data is persisted by primary node - not transaction coordinator.

Assume, we have transactional cache in cluster, near cache is disabled, node A is primary for key1, node B is neither primary nor backup for key1, and some backup nodes. And every node has its local db.
If we start transaction on node B, then data would be persisted by primary node and perhaps by backup nodes, transaction coordinator would not open db connections.

ср, 17 янв. 2018 г. в 3:59, vkulichenko <[hidden email]>:
Aleksey,

Are you talking about use case when a transaction spawns a distributed cache
AND a local cache? If so, this sounds like a very weird use case. Do we even
allow this?

-Val



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/


--

Best Regards,

Kuznetsov Aleksey

Andrey Nestrogaev Andrey Nestrogaev
Reply | Threaded
Open this post in threaded view
|

Re: 3rd Party Persistence and two phase commit

Hi,

Aleksey, what is "local db"?
I did not find anything on this in the documentation.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ALEKSEY KUZNETSOV ALEKSEY KUZNETSOV
Reply | Threaded
Open this post in threaded view
|

Re: 3rd Party Persistence and two phase commit

I meant local store = local db.

ср, 17 янв. 2018 г. в 13:52, Andrey Nestrogaev <[hidden email]>:
Hi,

Aleksey, what is "local db"?
I did not find anything on this in the documentation.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/


--

Best Regards,

Kuznetsov Aleksey

Andrey Nestrogaev Andrey Nestrogaev
Reply | Threaded
Open this post in threaded view
|

Re: 3rd Party Persistence and two phase commit

Local store = Apache Ignite Native Persistence?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Denis Mekhanikov Denis Mekhanikov
Reply | Threaded
Open this post in threaded view
|

Re: 3rd Party Persistence and two phase commit

Aleksey,

You described an invalid case. Cache stores on all nodes for a single cache should point to the same database. Otherwise data won't be the same in different databases.
Different nodes are not supposed to have personal local DBs.

If you want nodes to store data on disk in distributed manner, consider using Ignite native persistence: https://apacheignite.readme.io/docs/distributed-persistent-store

Denis

ср, 17 янв. 2018 г. в 14:17, Andrey Nestrogaev <[hidden email]>:
Local store = Apache Ignite Native Persistence?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ALEKSEY KUZNETSOV ALEKSEY KUZNETSOV
Reply | Threaded
Open this post in threaded view
|

Re: 3rd Party Persistence and two phase commit

Denis, no, its valid case.

you can set certain StoreFactory in cache configuration so that it create *different* instances of database(local) on different nodes. Then, my case will be applied.



ср, 17 янв. 2018 г. в 14:45, Denis Mekhanikov <[hidden email]>:
Aleksey,

You described an invalid case. Cache stores on all nodes for a single cache should point to the same database. Otherwise data won't be the same in different databases.
Different nodes are not supposed to have personal local DBs.

If you want nodes to store data on disk in distributed manner, consider using Ignite native persistence: https://apacheignite.readme.io/docs/distributed-persistent-store

Denis

ср, 17 янв. 2018 г. в 14:17, Andrey Nestrogaev <[hidden email]>:
Local store = Apache Ignite Native Persistence?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/


--

Best Regards,

Kuznetsov Aleksey

Denis Mekhanikov Denis Mekhanikov
Reply | Threaded
Open this post in threaded view
|

Re: 3rd Party Persistence and two phase commit

Alexey,

Of cause you can, but you will get an undefined behaviour in this case :)

It is assumed, that all nodes are using the same database. Otherwise consistency is not guaranteed.

Maybe it is worth to clarify this point in the documentation explicitly.

Denis

ср, 17 янв. 2018 г. в 15:10, ALEKSEY KUZNETSOV <[hidden email]>:
Denis, no, its valid case.

you can set certain StoreFactory in cache configuration so that it create *different* instances of database(local) on different nodes. Then, my case will be applied.



ср, 17 янв. 2018 г. в 14:45, Denis Mekhanikov <[hidden email]>:
Aleksey,

You described an invalid case. Cache stores on all nodes for a single cache should point to the same database. Otherwise data won't be the same in different databases.
Different nodes are not supposed to have personal local DBs.

If you want nodes to store data on disk in distributed manner, consider using Ignite native persistence: https://apacheignite.readme.io/docs/distributed-persistent-store

Denis

ср, 17 янв. 2018 г. в 14:17, Andrey Nestrogaev <[hidden email]>:
Local store = Apache Ignite Native Persistence?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

--

Best Regards,

Kuznetsov Aleksey