How do I provide AffinityFunction.BackupFilter when mainly using C++ bindings

classic Classic list List threaded Threaded
9 messages Options
tolga tolga
Reply | Threaded
Open this post in threaded view
|

How do I provide AffinityFunction.BackupFilter when mainly using C++ bindings

Hello,

I am in a similar situation with the following unanswered SO question: http://stackoverflow.com/questions/43386765/how-do-i-provide-affinityfunction-backupfilter-when-mainly-using-c-bindings

How can I achieve this? Quoting the question:

I am using the C++ bindings to have redundancy in my application. Alongside the main C++ node, I am running a vanilla Java node via ignite.sh as a backup on another node. I would like appoint this vanilla Java node to always stay a backup node and never a primary node as long as there is a C++ node running. Also, I need the C++ nodes to always stay as primary nodes. A little data loss is acceptable with the PRIMARY_SYNC synchronization.

My research led me to AffinityFunction.BackupFilter property to filter C++ nodes as primary. It seems that there is also some functions to give attributes to nodes. So I guess I can set a specific attribute on C++ nodes and filter them to always stay as primary nodes.

However, C++ bindings apparently neither provide a way to set backup filter nor allow setting attributes on the launched node. I have noticed some modules get plugged through ignite-dir/libs but there is no tutorial about that approach to add AffinityFunction. How can I achieve what I need? I need to plug a custom affinity function while using C++ as the main and a way to distinguish the C++ nodes.

vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|

Re: How do I provide AffinityFunction.BackupFilter when mainly using C++ bindings

Hi,

There is no way to do this unless you implement your own affinity function. Out of the box you can control whether a particular node can be used as a partition backup having primary node for this partition already assigned.

What is the purpose of such deployment? What are you trying to achieve with this?

-Val
tolga tolga
Reply | Threaded
Open this post in threaded view
|

Re: How do I provide AffinityFunction.BackupFilter when mainly using C++ bindings

I am trying to create an ACTIVE/PASSIVE redundant environment, passive node being the vanilla .sh node. I would like to have control on what node is appointed as primary.

According to what you are saying, for a moment I have to have only one node (that being C++ node) online for it to be primary node at switchover. I would like to avoid that because if a crash happens at that time, the data will be lost.

Is there a tutorial on writing a plugin in ./libs folder that defines a custom affinity function and configuring it? There is also the matter of identifying the C++ node.

On Thu, Apr 13, 2017, 12:32 PM vkulichenko <[hidden email]> wrote:
Hi,

There is no way to do this unless you implement your own affinity function.
Out of the box you can control whether a particular node can be used as a
partition backup having primary node for this partition already assigned.

What is the purpose of such deployment? What are you trying to achieve with
this?

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-do-I-provide-AffinityFunction-BackupFilter-when-mainly-using-C-bindings-tp11930p11938.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
tolga tolga
Reply | Threaded
Open this post in threaded view
|

Re: How do I provide AffinityFunction.BackupFilter when mainly using C++ bindings

I have managed to put a AffinityFunction implementation in ./libs and load/use it. But I still could not figure out how to detect my C++ node. 

I see C++ API allows you to name my node via Ignition::Start(const IgniteConfiguration &cfg, const char *name) but ClusterNode does not have a method to get the node's name. It allows you to get the attributes but C++ does not provide anything to set the attribute on the node. What would you recommend to distinguish my C++ node?

On Thu, Apr 13, 2017 at 4:04 PM tolga <[hidden email]> wrote:
I have managed to put a AffinityFunction implementation in ./libs and
load/use it. But I still could not figure out how to detect my C++ node.

I see C++ API allows you to name my node via /Ignition::Start(const
IgniteConfiguration &cfg, const char *name)/ but  ClusterNode
<https://www.gridgain.com/sdk/pe/latest/javadoc/org/apache/ignite/cluster/ClusterNode.html>
does not have a method to get the node's name. It allows you to get the
attributes but C++ does not provide anything to set the attribute on the
node. What would you recommend to distinguish my C++ node?



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-do-I-provide-AffinityFunction-BackupFilter-when-mainly-using-C-bindings-tp11930p11945.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Igor Sapego Igor Sapego
Reply | Threaded
Open this post in threaded view
|

Re: How do I provide AffinityFunction.BackupFilter when mainly using C++ bindings

Well, there is a way to get C++ nodes. You can use the following code for this:

IgniteCluster#forAttribute("org.apache.ignite.platform", "cpp");

That returns you a cluster group of C++ nodes if that's what you need.

Best Regards,
Igor

On Thu, Apr 13, 2017 at 4:05 PM, Tolga HOŞGÖR <[hidden email]> wrote:
I have managed to put a AffinityFunction implementation in ./libs and load/use it. But I still could not figure out how to detect my C++ node. 

I see C++ API allows you to name my node via Ignition::Start(const IgniteConfiguration &cfg, const char *name) but ClusterNode does not have a method to get the node's name. It allows you to get the attributes but C++ does not provide anything to set the attribute on the node. What would you recommend to distinguish my C++ node?

On Thu, Apr 13, 2017 at 4:04 PM tolga <[hidden email]> wrote:
I have managed to put a AffinityFunction implementation in ./libs and
load/use it. But I still could not figure out how to detect my C++ node.

I see C++ API allows you to name my node via /Ignition::Start(const
IgniteConfiguration &cfg, const char *name)/ but  ClusterNode
<https://www.gridgain.com/sdk/pe/latest/javadoc/org/apache/ignite/cluster/ClusterNode.html>
does not have a method to get the node's name. It allows you to get the
attributes but C++ does not provide anything to set the attribute on the
node. What would you recommend to distinguish my C++ node?



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-do-I-provide-AffinityFunction-BackupFilter-when-mainly-using-C-bindings-tp11930p11945.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

tolga tolga
Reply | Threaded
Open this post in threaded view
|

Re: How do I provide AffinityFunction.BackupFilter when mainly using C++ bindings

This will do it. I have found out ClusterNode::attribute also includes environment variables so I can even further identify my nodes.

I also want to make sure if I get setAffinityBackupFilter right. Let's say I have two nodes and I return false for one of them and true for the other. I assume the one with false return will be assigned as PRIMARY and the one I return true will be assigned as BACKUP. Is it correct or the false one is not eligible to hold data at all and a primary is chosen among backups?

On Thu, Apr 13, 2017 at 4:23 PM Igor Sapego <[hidden email]> wrote:
Well, there is a way to get C++ nodes. You can use the following code for this:

IgniteCluster#forAttribute("org.apache.ignite.platform", "cpp");

That returns you a cluster group of C++ nodes if that's what you need.

Best Regards,
Igor

On Thu, Apr 13, 2017 at 4:05 PM, Tolga HOŞGÖR <[hidden email]> wrote:
I have managed to put a AffinityFunction implementation in ./libs and load/use it. But I still could not figure out how to detect my C++ node. 

I see C++ API allows you to name my node via Ignition::Start(const IgniteConfiguration &cfg, const char *name) but ClusterNode does not have a method to get the node's name. It allows you to get the attributes but C++ does not provide anything to set the attribute on the node. What would you recommend to distinguish my C++ node?

On Thu, Apr 13, 2017 at 4:04 PM tolga <[hidden email]> wrote:
I have managed to put a AffinityFunction implementation in ./libs and
load/use it. But I still could not figure out how to detect my C++ node.

I see C++ API allows you to name my node via /Ignition::Start(const
IgniteConfiguration &cfg, const char *name)/ but  ClusterNode
<https://www.gridgain.com/sdk/pe/latest/javadoc/org/apache/ignite/cluster/ClusterNode.html>
does not have a method to get the node's name. It allows you to get the
attributes but C++ does not provide anything to set the attribute on the
node. What would you recommend to distinguish my C++ node?



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-do-I-provide-AffinityFunction-BackupFilter-when-mainly-using-C-bindings-tp11930p11945.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|

Re: How do I provide AffinityFunction.BackupFilter when mainly using C++ bindings

In reply to this post by tolga
Why not create two equal C++ nodes and use both of them? You will give you data redundancy and load balancing, without any custom affinity functions.

Your model seems to be worse and also requires more effort. I would recommend to use abilities provided by Ignite out of the box.

-Val
tolga tolga
Reply | Threaded
Open this post in threaded view
|

Re: How do I provide AffinityFunction.BackupFilter when mainly using C++ bindings

It is not applicable because it will require too much effort to convert the rest of the system for ACTIVE/ACTIVE set-up.

On Thu, Apr 13, 2017 at 5:10 PM vkulichenko <[hidden email]> wrote:
Why not create two equal C++ nodes and use both of them? You will give you
data redundancy and load balancing, without any custom affinity functions.

Your model seems to be worse and also requires more effort. I would
recommend to use abilities provided by Ignite out of the box.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-do-I-provide-AffinityFunction-BackupFilter-when-mainly-using-C-bindings-tp11930p11953.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|

Re: How do I provide AffinityFunction.BackupFilter when mainly using C++ bindings

You can use APIs only on one of them, or create a cluster of two server nodes, embed a client in the application and use APIs there. In any case, I don't see why you need custom affinity function or even backup filter.

And your understanding of backup filter is not correct. Primary node for a partition is assigned first and backup filter does not control it in any way. The filter is then invoked when backups are assigned, providing corresponding primary that is already chosen. You can tell whether you want to use a node as a backup for that primary or not. This is usually used to make sure primary and backup are in different server racks, for example. It's not really applicable to your case.

-Val