I have problems with balancing cache partitioning among nodes

classic Classic list List threaded Threaded
4 messages Options
AJoshua AJoshua
Reply | Threaded
Open this post in threaded view
|

I have problems with balancing cache partitioning among nodes

so here goes to the senario, we can assume from the examples of gridgain tutorial.
We have a person cache and I want to have only 8 partitions and all persons should be distributed among these partitions. we can imagine, I have 8 companies in which each person belongs to one of those but I have no company cache, only a key ’compKey' of type long. I have assigned @AffinityKeyMapped to compKey.

The problem is if I have two node so the data is partitioned unbalanced i.e.
Node 1 keeps all compKey {2.4} and its people while Node 2 keeps compKey {1,3,5,6,7,8} in most case in this form.
so if we write 1,000 people, exactly the same number of each org Key i.e. 125 of each, so people are balanced between node1 and 2 {250 respectively 750}
vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|

Re: I have problems with balancing cache partitioning among nodes

Hi,

This is possible with this little number of affinity keys, because built-in affinity functions give better results on the growing data. Affinity function is pluggable, so you can implement AffinityFunction interface and provide it in the CacheConfiguration.

But at first I would make sure that you really need a partitioned cache. Generally, it's needed for large data sets or growing data sets that you want to scale out when needed. Sounds like both are not your cases, so probably you can just use replicated cache?

See this page for more information about different cache modes: https://apacheignite.readme.io/docs/cache-modes

-Val
AJoshua AJoshua
Reply | Threaded
Open this post in threaded view
|

Re: I have problems with balancing cache partitioning among nodes

This post was updated on .
my goal is to test the performance between gigaspaces and gridgain in gigaspaces i have implemented partitioning in the same way as the gridgain.
On gigaspaces grid service container that corresponds to the Node on griddgain and each GSC is assigned with a number of partitions. the goal is that the test should be hundreds of thousands data test on .should I increase the number companykey? or implementing AffinityFunction Interface


-A
vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|

Re: I have problems with balancing cache partitioning among nodes

If you increase the number of companies, you should get much better distribution. I would try this out and see how it changes the results.

If you have a very specific use case where the number of companies is very limited, but you still want to use partitioned cache, you can implement the affinity function. Let me know if you have additional questions about this.

-Val