How does Apache Ignite distribute???

classic Classic list List threaded Threaded
9 messages Options
BorisBelozerov BorisBelozerov
Reply | Threaded
Open this post in threaded view
|

How does Apache Ignite distribute???

Hello, I want to ask a question:
Apache Ignite distribute data equal each nodes, and use Affinity
How does Apache Ignite distribute and how does Affinity work??
Thank you!!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Mikael Mikael
Reply | Threaded
Open this post in threaded view
|

Re: How does Apache Ignite distribute???

Hi!

You have a good description at:

https://apacheignite.readme.io/docs/affinity-collocation

Long story short, it use hash codes on the keys to control distribution
of the entries, affinity keys allow you to control what is used to
calculate the hash codes, you can for example have a string and an
integer as a key where only the integer is used as the affinity key, so
all keys with the same integer value (and different strings) will end up
in the same node.

Mikael

Den 2019-10-31 kl. 08:42, skrev BorisBelozerov:

> Hello, I want to ask a question:
> Apache Ignite distribute data equal each nodes, and use Affinity
> How does Apache Ignite distribute and how does Affinity work??
> Thank you!!
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
BorisBelozerov BorisBelozerov
Reply | Threaded
Open this post in threaded view
|

Re: How does Apache Ignite distribute???

Can I customize affinity by the way I want to?
How can I do it? Thank you!!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
stephendarlington stephendarlington
Reply | Threaded
Open this post in threaded view
|

Re: How does Apache Ignite distribute???

What do you want to customise? How does the current behaviour not meet your requirements?

> On 31 Oct 2019, at 08:02, BorisBelozerov <[hidden email]> wrote:
>
> Can I customize affinity by the way I want to?
> How can I do it? Thank you!!
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/


BorisBelozerov BorisBelozerov
Reply | Threaded
Open this post in threaded view
|

Re: How does Apache Ignite distribute???

I want to give more data for some chosen nodes (because these computer have
more memory to save data), but I don't know how to do it.
How can I choose node, and how can I distribute more data for chosen nodes?
Thank you!!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
stephendarlington stephendarlington
Reply | Threaded
Open this post in threaded view
|

Re: How does Apache Ignite distribute???

The direct answer to your question is: implement your own org.apache.ignite.cache.affinity.AffinityFunction.

But that’s hard to do correctly. You’d probably be better running, say, two copies of Ignite on the machines with more memory.

Regards,
Stephen

> On 31 Oct 2019, at 15:25, BorisBelozerov <[hidden email]> wrote:
>
> I want to give more data for some chosen nodes (because these computer have
> more memory to save data), but I don't know how to do it.
> How can I choose node, and how can I distribute more data for chosen nodes?
> Thank you!!
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/


yblazart yblazart
Reply | Threaded
Open this post in threaded view
|

Re: How does Apache Ignite distribute???

Or simply give an attribute with a specific value to the nodes with memory and only create caches on those nodes.

Envoyé de mon iPhone

> Le 31 oct. 2019 à 17:19, Stephen Darlington <[hidden email]> a écrit :
>
> The direct answer to your question is: implement your own org.apache.ignite.cache.affinity.AffinityFunction.
>
> But that’s hard to do correctly. You’d probably be better running, say, two copies of Ignite on the machines with more memory.
>
> Regards,
> Stephen
>
>> On 31 Oct 2019, at 15:25, BorisBelozerov <[hidden email]> wrote:
>>
>> I want to give more data for some chosen nodes (because these computer have
>> more memory to save data), but I don't know how to do it.
>> How can I choose node, and how can I distribute more data for chosen nodes?
>> Thank you!!
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
>
BorisBelozerov BorisBelozerov
Reply | Threaded
Open this post in threaded view
|

Re: How does Apache Ignite distribute???

In reply to this post by stephendarlington
Can you give me some code to implement it? Thank you!!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Stanislav Lukyanov Stanislav Lukyanov
Reply | Threaded
Open this post in threaded view
|

Re: How does Apache Ignite distribute???

I believe that the correct answer to your question - don't do that.

The strength of distributed systems is that you have a number of identical pieces which you can scale out virtually with no limits.

If your cluster is heterogenous - i.e. all the nodes are different in size, amount of data and power - you may easily run into issues with uneven load distribution and eventually into performance issues.
Even more, you can run into cluster management issues, e.g. if one big node leaves the cluster then the remaining small nodes will redistribute the data and may not have enough space to do that.

If you really-really want something like that, and you think that your specific use case is not going to suffer from uneven load or cluster management issues, then go with the suggestion from Stephen - run multiple nodes on big machines.

Stan

On Fri, Nov 1, 2019 at 4:36 AM BorisBelozerov <[hidden email]> wrote:
Can you give me some code to implement it? Thank you!!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/