Affinity collocation and node rebalancing

classic Classic list List threaded Threaded
6 messages Options
lhendra lhendra
Reply | Threaded
Open this post in threaded view
|

Affinity collocation and node rebalancing

Hi,

I'm totally new to Ignite and have been reading lots of documentation.
I'm trying to find info on how node rebalancing is handled in relation to
affinity collocation.
There are 2 use cases that I want to confirm:
(1) Create a new table with an affinity key to an existing table.
(2) Add new columns to existing tables.
In both cases, let's say we start loading data that causes nodes to fill up,
and we throw in a new node. Does Ignite automatically perform node
rebalancing WHILE preserving the affinity constraints in any/both cases?

Thanks,
Lily



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
dmagda dmagda
Reply | Threaded
Open this post in threaded view
|

Re: Affinity collocation and node rebalancing

Hello Lily,

Ignite ensures data consistency all the times taking care of data rebalancing scenarios as well. For instance, if there are Table1 and Table2 and they are collocated (Table2.affinityKey links it to Table1.someKey) then during rebalancing all the collocated records of both tables will be moved to the new node.

If to go in details, Ignite rebalances partitions. Once your cluster topology changes, the affinity function recalculates partitions distributions to make it even cluster-wide. As a result, some of the partitions will stay on existing nodes while the others will be moved to the new one. Collocated data is always mapped/stored to/in the partition that has the same number as the partition of the parental record  - for instance, if Table1.someKey is mapped to partition 5 of Table1 then all the records of Table2 that Table2.affinityKey==Table1.someKey will be stored in the partition 5 of Table2. If partition 5 needs to be rebalanced, then it will be rebalanced for all the tables you have so that the new node owns all the partitions 5 of all the tables. The only requirement here is that the total partition number and affinity function must be the same for all the tables (which is true by default). 

-
Denis


On Tue, Jul 30, 2019 at 4:28 PM lhendra <[hidden email]> wrote:
Hi,

I'm totally new to Ignite and have been reading lots of documentation.
I'm trying to find info on how node rebalancing is handled in relation to
affinity collocation.
There are 2 use cases that I want to confirm:
(1) Create a new table with an affinity key to an existing table.
(2) Add new columns to existing tables.
In both cases, let's say we start loading data that causes nodes to fill up,
and we throw in a new node. Does Ignite automatically perform node
rebalancing WHILE preserving the affinity constraints in any/both cases?

Thanks,
Lily



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
lhendra lhendra
Reply | Threaded
Open this post in threaded view
|

Re: Affinity collocation and node rebalancing

Denis, thank you for the detailed explanation

One follow-up question, if you don't mind.
Is there a limit to how many tables you can chain in the affinity
collocation? We might have a use case where we want have 30 tables chained
together in affinity collocation (not too many columns each). Providing that
we'll have enough memory, is this something doable in Ignite?

Thanks,
Lily



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
dmagda dmagda
Reply | Threaded
Open this post in threaded view
|

Re: Affinity collocation and node rebalancing

It doesn't seem like an issue to me. The only one thing you should keep in mind is related to the potential size of partitions.

Let's say you have Cities and Residents tables, and every resident is collocated with its City ID. So, for New York you'll certainly have more Residents than for Boston; thus,  the partition that stores all the Residents of NY will be larger than the Boston's one. It can lead to the situation when one node consumes more memory space only because it stores more partitions of the larger size. In reality, and that's considered in Ignite design, it's assumed that there will be many cities like NY and Boston and all of them will be evenly distributed across different partitions and, thus, all the nodes will occupy somehow equal memory space.

For those scenarios, when the even distribution doesn't work out (because there are just two cities of the size of NY and you have 2 nodes that consume too much space) you can come up with an alternate collocation scheme when for Residents living in NY you use a complex affinity key like (city_id, district_id) and with that help the district_id will also be involved in the data distribution.

-
Denis


On Thu, Aug 1, 2019 at 10:09 AM lhendra <[hidden email]> wrote:
Denis, thank you for the detailed explanation

One follow-up question, if you don't mind.
Is there a limit to how many tables you can chain in the affinity
collocation? We might have a use case where we want have 30 tables chained
together in affinity collocation (not too many columns each). Providing that
we'll have enough memory, is this something doable in Ignite?

Thanks,
Lily



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
lhendra lhendra
Reply | Threaded
Open this post in threaded view
|

Re: Affinity collocation and node rebalancing

Denis, again this is really great - definitely something that I will take
into consideration during the schema design.
Really appreciate your thoughtful replies and explanations!

Lily



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
dmagda dmagda
Reply | Threaded
Open this post in threaded view
|

Re: Affinity collocation and node rebalancing

Thanks for raising the questions! We'll update the documentation accordingly clarifying some tricky areas and scenarios.

-
Denis


On Thu, Aug 1, 2019 at 1:33 PM lhendra <[hidden email]> wrote:
Denis, again this is really great - definitely something that I will take
into consideration during the schema design.
Really appreciate your thoughtful replies and explanations!

Lily



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/