Affinity key for ignite tables

classic Classic list List threaded Threaded
3 messages Options
favas favas
Reply | Threaded
Open this post in threaded view
|

Affinity key for ignite tables

HI,

 

I have read through the collocating data in ignite node and it is possible to give affinity key in ignite tables using “AFFINITY_KEY” keywords.

All the examples I have gone through was mention with simple two tables and it has one to one relationship.

 

I still confused on how the affinity key column is relating the value in one table with other table if I have different tables relating with same key.

Consider one sample data structure below.

 

 

If I have all these tables created on ignite, how can I collocate the data using affinity key.

 

Regards,

Favas 

 

akurbanov akurbanov
Reply | Threaded
Open this post in threaded view
|

Re: Affinity key for ignite tables

Hello,

Ignite does map the entries to nodes in the following way: entry key > affinity key > partition > node

You can control the first part of this mapping by defining an affinity key for the type. All the cache entries with same affinity key will be stored at the same node (affinity key = CustID, then for each N (int, long?) all customers, customers docs and sale entries with CustID = N will be stored on the same regions for everything on the right of affinity key (partition and node)).

If you have 2+ caches that have the same affinity key, let say in your example it will be ProdID or CustID, they will be on the same node. 

This is obviously not easy for this case: if you will have ProdID as affinity key, you can collocate your products and sales of these products will be residing on the same node. If you will use CustID as affinity key, you will end up having Customer, CustomerDocs data and Sales located on the same node for each customer. However, the rest of the tables won't be collocated in these cases. 

There is another condition that should be also satisfied (in ideal world): keys >> partitions >> nodes, and it would be great if the difference is in orders of magnitude. And I would say in that case, if some of the caches are not massive, they could be switched to be replicated across all the nodes.

So I would imagine that there is a limited set of products. I would make this table replicated and have collocated all the other tables for this case using CustID as affinity key if there is a large amount of customers and/or amount of customers is not fixed. A more real-world example would require knowledge of real data to make a better decision.

Regards,
Anton

чт, 3 окт. 2019 г. в 12:26, Muhammed Favas <[hidden email]>:

HI,

 

I have read through the collocating data in ignite node and it is possible to give affinity key in ignite tables using “AFFINITY_KEY” keywords.

All the examples I have gone through was mention with simple two tables and it has one to one relationship.

 

I still confused on how the affinity key column is relating the value in one table with other table if I have different tables relating with same key.

Consider one sample data structure below.

 

 

If I have all these tables created on ignite, how can I collocate the data using affinity key.

 

Regards,

Favas 

 

dmagda dmagda
Reply | Threaded
Open this post in threaded view
|

Re: Affinity key for ignite tables

Only one-to-one relation is possible for affinity collocation. Your Sales can be either collocated with Product or Customer. More details are here:

-
Denis


On Thu, Oct 3, 2019 at 7:31 AM Anton Kurbanov <[hidden email]> wrote:
Hello,

Ignite does map the entries to nodes in the following way: entry key > affinity key > partition > node

You can control the first part of this mapping by defining an affinity key for the type. All the cache entries with same affinity key will be stored at the same node (affinity key = CustID, then for each N (int, long?) all customers, customers docs and sale entries with CustID = N will be stored on the same regions for everything on the right of affinity key (partition and node)).

If you have 2+ caches that have the same affinity key, let say in your example it will be ProdID or CustID, they will be on the same node. 

This is obviously not easy for this case: if you will have ProdID as affinity key, you can collocate your products and sales of these products will be residing on the same node. If you will use CustID as affinity key, you will end up having Customer, CustomerDocs data and Sales located on the same node for each customer. However, the rest of the tables won't be collocated in these cases. 

There is another condition that should be also satisfied (in ideal world): keys >> partitions >> nodes, and it would be great if the difference is in orders of magnitude. And I would say in that case, if some of the caches are not massive, they could be switched to be replicated across all the nodes.

So I would imagine that there is a limited set of products. I would make this table replicated and have collocated all the other tables for this case using CustID as affinity key if there is a large amount of customers and/or amount of customers is not fixed. A more real-world example would require knowledge of real data to make a better decision.

Regards,
Anton

чт, 3 окт. 2019 г. в 12:26, Muhammed Favas <[hidden email]>:

HI,

 

I have read through the collocating data in ignite node and it is possible to give affinity key in ignite tables using “AFFINITY_KEY” keywords.

All the examples I have gone through was mention with simple two tables and it has one to one relationship.

 

I still confused on how the affinity key column is relating the value in one table with other table if I have different tables relating with same key.

Consider one sample data structure below.

 

 

If I have all these tables created on ignite, how can I collocate the data using affinity key.

 

Regards,

Favas