Remote Filter Execution

classic Classic list List threaded Threaded
13 messages Options
VeenaMithare VeenaMithare
Reply | Threaded
Open this post in threaded view
|

Remote Filter Execution

Hello,

We have a 3 server cluster .  The Caches on this cluster are configured as
PARTITIONED, with 1 BACKUP.

We also have a client executing a simple continuous query awaiting updates
on record where NAME=AA.

1. When this record is updated, I see the remote filter being executed only
on one node( say : Server1 ) . Should this not be executed on atleast 2
nodes since the Cache is configured with 1 Backup .

From the documentation :
https://apacheignite.readme.io/docs/continuous-queries#remote-filter

This filter is executed on primary and backup nodes for a given key and
evaluates whether an update should be propagated as an event to the query's
local listener.

2. If Server1 i.e.  the server executing this remote filter is restarted (
say for some new deployment or any other reason ), I see that the remote
filter is not deployed on this again.

And now, changes in any other record on this table ( i.e. records other than
NAME=AA ) are also passed to the client - since there is no remote filter to
filter this record.

Please let me know what I am missing.






--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Remote Filter Execution

Hello!

This is weird, I recommend checking logs for exceptions, maybe the filter cannot run on two other server nodes.

Regards,
--
Ilya Kasnacheev


пт, 10 июл. 2020 г. в 10:25, VeenaMithare <[hidden email]>:
Hello,

We have a 3 server cluster .  The Caches on this cluster are configured as
PARTITIONED, with 1 BACKUP.

We also have a client executing a simple continuous query awaiting updates
on record where NAME=AA.

1. When this record is updated, I see the remote filter being executed only
on one node( say : Server1 ) . Should this not be executed on atleast 2
nodes since the Cache is configured with 1 Backup .

From the documentation :
https://apacheignite.readme.io/docs/continuous-queries#remote-filter

This filter is executed on primary and backup nodes for a given key and
evaluates whether an update should be propagated as an event to the query's
local listener.

2. If Server1 i.e.  the server executing this remote filter is restarted (
say for some new deployment or any other reason ), I see that the remote
filter is not deployed on this again.

And now, changes in any other record on this table ( i.e. records other than
NAME=AA ) are also passed to the client - since there is no remote filter to
filter this record.

Please let me know what I am missing.






--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
VeenaMithare VeenaMithare
Reply | Threaded
Open this post in threaded view
|

Re: Remote Filter Execution

HI Ilya,

Please find the attached logs.

You will notice that this log is only on server 1  - "projectname LISTENS:
Entered Remote Filter ."

You can also see that the filter has been created on all the three nodes. (
LOG : projectname LISTENS: Filter created  )

server1RemoteFilterExecuted.txt
<http://apache-ignite-users.70518.x6.nabble.com/file/t2757/server1RemoteFilterExecuted.txt>  
server2.txt
<http://apache-ignite-users.70518.x6.nabble.com/file/t2757/server2.txt>  
server3.txt
<http://apache-ignite-users.70518.x6.nabble.com/file/t2757/server3.txt>  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Remote Filter Execution

Hello!

I see the following lines on all 3 nodes:

2020-07-09 16:41:02,957 [disco-notifier-worker-#101] DEBUG com.COMPANYNAME.prophet.configstore.common.remote.ConfigStoreTableRemoteFilterFactory [] - projectname LISTENS: Filter creation started with andPredicateMap:{NAME=(EQUAL,CENTRE)},orPredicateMap:null, Remotefilterfactory:com.COMPANYNAME.prophet.configstore.common.remote.ConfigStoreTableRemoteFilterFactory@14bf552f, Service:client Instance:null, HostName:null 2020-07-09 16:41:02,958 [disco-notifier-worker-#101] DEBUG com.COMPANYNAME.prophet.configstore.common.remote.ConfigStoreTableRemoteFilterFactory [] - projectname LISTENS: Filter created with andPredicateMap{NAME=(EQUAL,CENTRE)},orPredicateMapnull, Remotefilterfactory:com.COMPANYNAME.prophet.configstore.common.remote.ConfigStoreTableRemoteFilterFactory@14bf552f,

Is it relevant?


Regards,
--
Ilya Kasnacheev


пт, 10 июл. 2020 г. в 16:02, VeenaMithare <[hidden email]>:
HI Ilya,

Please find the attached logs.

You will notice that this log is only on server 1  - "projectname LISTENS:
Entered Remote Filter ."

You can also see that the filter has been created on all the three nodes. (
LOG : projectname LISTENS: Filter created  )

server1RemoteFilterExecuted.txt
<http://apache-ignite-users.70518.x6.nabble.com/file/t2757/server1RemoteFilterExecuted.txt
server2.txt
<http://apache-ignite-users.70518.x6.nabble.com/file/t2757/server2.txt
server3.txt
<http://apache-ignite-users.70518.x6.nabble.com/file/t2757/server3.txt



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
VeenaMithare VeenaMithare
Reply | Threaded
Open this post in threaded view
|

Re: Remote Filter Execution

Hi Ilya,

Yes, So filter gets created but not executed on more than one node. 'Entered
Remote Filter' happens only on one node. 'Filter created' log is printed in
the 'create()' method of the Factory<CacheEntryEventFilter>.
'Entered Remote Filter' is printed in the evaluate method of the
CacheEntryEventFilter.

That is what I was saying in my earlier post -

You will notice that this log is only on server 1  - "projectname LISTENS:
Entered Remote Filter ."

You can also see that the filter has been created on all the three nodes. (
LOG : projectname LISTENS: Filter created  )

regards,
Veena.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Remote Filter Execution

Hello!

Maybe there's something with your filter implementation which prevents it from functioning.

Can you throw together a small reproducer project to show this behavior?

Regards,
--
Ilya Kasnacheev


пт, 10 июл. 2020 г. в 19:35, VeenaMithare <[hidden email]>:
Hi Ilya,

Yes, So filter gets created but not executed on more than one node. 'Entered
Remote Filter' happens only on one node. 'Filter created' log is printed in
the 'create()' method of the Factory<CacheEntryEventFilter>.
'Entered Remote Filter' is printed in the evaluate method of the
CacheEntryEventFilter.

That is what I was saying in my earlier post -

You will notice that this log is only on server 1  - "projectname LISTENS:
Entered Remote Filter ."

You can also see that the filter has been created on all the three nodes. (
LOG : projectname LISTENS: Filter created  )

regards,
Veena.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
VeenaMithare VeenaMithare
Reply | Threaded
Open this post in threaded view
|

Re: Remote Filter Execution

HI Ilya,

I will put together something and post it here.

regards,
Veena.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
VeenaMithare VeenaMithare
Reply | Threaded
Open this post in threaded view
|

Re: Remote Filter Execution

Hi Ilya,

Please find the reproducer project with a readme.txt.

I have put in comments on how to reproduce both the issues mentioned in my
original mail.

regards,
Veena. RemoteFilterIssueProject.zip
<http://apache-ignite-users.70518.x6.nabble.com/file/t2757/RemoteFilterIssueProject.zip>  
Readme.txt
<http://apache-ignite-users.70518.x6.nabble.com/file/t2757/Readme.txt>  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Remote Filter Execution

Hello!

I have just checked this behavior on 2.8.1, and I don't see any of these issues popping up. All 3 server nodes run remote listener (judging by log), and after restarting of nodes I can still see updates on 'id1' appearing in client npde's log.

I didn't check it on 2.7.6. Can you confirm or deny that it works on 2.8.1 for you? Are you sure that all nodes are in the baseline?

Regards,
--
Ilya Kasnacheev


ср, 15 июл. 2020 г. в 13:55, VeenaMithare <[hidden email]>:
Hi Ilya,

Please find the reproducer project with a readme.txt.

I have put in comments on how to reproduce both the issues mentioned in my
original mail.

regards,
Veena. RemoteFilterIssueProject.zip
<http://apache-ignite-users.70518.x6.nabble.com/file/t2757/RemoteFilterIssueProject.zip
Readme.txt
<http://apache-ignite-users.70518.x6.nabble.com/file/t2757/Readme.txt



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
VeenaMithare VeenaMithare
Reply | Threaded
Open this post in threaded view
|

Re: Remote Filter Execution

This post was updated on .
CONTENTS DELETED
The author has deleted this message.
VeenaMithare VeenaMithare
Reply | Threaded
Open this post in threaded view
|

Re: Remote Filter Execution

Hi Ilya,

I can confirm that both issues do not occur in 2.8.1. ( The remote filter
evaluate executes only on 2 nodes as expected ..) Also the remote filter
gets deployed whenever there is a server restart.

The other thing for me to check is if we can use 2.8.1 ( We had issues on
2.8.0  jdbc thin client library , because of which we couldnt use it )

Thank you so much for the help ,

regards,
Veena.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
VeenaMithare VeenaMithare
Reply | Threaded
Open this post in threaded view
|

Re: Remote Filter Execution

I can also confirm that both these issues happen consistently on 2.7.6. All
the nodes are in baseline when both these issues happen.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Remote Filter Execution

Hello!

There's nothing we can do about 2.7.x right now, I suggest upgrading to 2.8.1.

Regards,
--
Ilya Kasnacheev


пт, 17 июл. 2020 г. в 10:06, VeenaMithare <[hidden email]>:
I can also confirm that both these issues happen consistently on 2.7.6. All
the nodes are in baseline when both these issues happen.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/