Re:

classic Classic list List threaded Threaded
10 messages Options
a.dolmatov a.dolmatov
Reply | Threaded
Open this post in threaded view
|

Re:

Did you try to implement custom affinity function to control distribution?


On Fri, Oct 25, 2019, 05:50 ARSENAL 1886 <[hidden email]> wrote:

Normally, Apache Ignite distribute data equally to nodes
How can I control this distribution?
I want to distribute more data to some nodes, and less data to other nodes
I also want to choose node to store cache
Thank you!!

 

Sent from Mail for Windows 10

 

yblazart yblazart
Reply | Threaded
Open this post in threaded view
|

Re:

Hello, of my tries, with 40Gb of input data, data is not equally partitioned, documentation say that you can have 5 to 10% of differences. I saw 30%

Le dim. 27 oct. 2019 à 09:56, Andrey Dolmatov <[hidden email]> a écrit :
Did you try to implement custom affinity function to control distribution?


On Fri, Oct 25, 2019, 05:50 ARSENAL 1886 <[hidden email]> wrote:

Normally, Apache Ignite distribute data equally to nodes
How can I control this distribution?
I want to distribute more data to some nodes, and less data to other nodes
I also want to choose node to store cache
Thank you!!

 

Sent from Mail for Windows 10

 

BorisBelozerov BorisBelozerov
Reply | Threaded
Open this post in threaded view
|

Re:

In reply to this post by a.dolmatov
I don't know how to try it! Can you help me?
dmagda dmagda
Reply | Threaded
Open this post in threaded view
|

Re:

What's the primary reason for such custom distribution? As it was suggested, you can always implement your own AffinityFunction but it has to be battle-tested really well, otherwise, if anything is missed during the implementation it can lead to hard to reproduce and debug issues in production.

-
Denis


On Sun, Oct 27, 2019 at 11:35 PM Trần Đình Oánh <[hidden email]> wrote:
I don't know how to try it! Can you help me?
BorisBelozerov BorisBelozerov
Reply | Threaded
Open this post in threaded view
|

Re: Re:

I want to distribute more data to some nodes because they have more memory to
save data
Can you give me some example code for it? Thanks!!!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
yblazart yblazart
Reply | Threaded
Open this post in threaded view
|

Re: Re:

Another way could be to give some attributes values to this node, then create caches only on these node, no?

Le jeu. 31 oct. 2019 à 08:58, BorisBelozerov <[hidden email]> a écrit :
I want to distribute more data to some nodes because they have more memory to
save data
Can you give me some example code for it? Thanks!!!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
BorisBelozerov BorisBelozerov
Reply | Threaded
Open this post in threaded view
|

Re: Re:

I don't understand, can you explain for me?? Thank you!!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Mikael Mikael
Reply | Threaded
Open this post in threaded view
|

Re:

Hi!

You can do almost anything messing around with affinity keys and the
AffinityKeyMapper API, but if you can I would suggest you use two
different caches one is used on the "low" memory nodes and the other on
the "lots of memory" nodes, it would be easier to work with, it's easy
to mess up the affinity key mapping but it can be done.

You can find some ideas here:

http://apache-ignite-users.70518.x6.nabble.com/How-to-use-Affinity-Function-to-Map-a-set-of-Keys-to-a-particular-node-in-a-cluster-td21995.html

Mikael


Den 2019-10-31 kl. 09:27, skrev BorisBelozerov:
> I don't understand, can you explain for me?? Thank you!!
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
BorisBelozerov BorisBelozerov
Reply | Threaded
Open this post in threaded view
|

Re: Re:

How can I choose node? By IP, MAC, or other criteria??
Thank you!!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Mikael Mikael
Reply | Threaded
Open this post in threaded view
|

Re:

That is up to you, you can use anything you want, the most obvious
choice would be the built in attributes feature in Ignite configuration:

         <!-- Attributes -->
           <property name="userAttributes">
             <map>
               <entry key="MyAttribute" value="MyValue"/>
             </map>
           </property>

You will find a lot of information under the "Affinity Function" and
"Affinity Key Mapper" sections here:

https://apacheignite.readme.io/docs/affinity-collocation

Did you consider the idea of using multiple caches ? that would be much
easier to implement if it is a possible solution for you ?

Mikael


Den 2019-11-01 kl. 03:01, skrev BorisBelozerov:
> How can I choose node? By IP, MAC, or other criteria??
> Thank you!!
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>