High availability of local listeners for ContinuousQuery or Events

classic Classic list List threaded Threaded
4 messages Options
38797715 38797715
Reply | Threaded
Open this post in threaded view
|

High availability of local listeners for ContinuousQuery or Events

Hi community,

For local listeners registered for ContinuousQuery and Events, is there
a corresponding high availability mechanism design? That is, if the node
registering the local listener fails, what state will the cluster be?

If do not register a local listener, but write the business code in the
remote filter and return false, is this a good practice?


ibelyakov ibelyakov
Reply | Threaded
Open this post in threaded view
|

Re: High availability of local listeners for ContinuousQuery or Events

Hi,

In case the node, which registered a continuous query fails, the continuous query will be undeployed from the cluster. The cluster state won't be changed.

It's not a good practice to write the business code in a remote filter. Could you please clarify more details regarding your use case?

Igor

On Thu, Oct 29, 2020 at 4:46 PM 38797715 <[hidden email]> wrote:
Hi community,

For local listeners registered for ContinuousQuery and Events, is there
a corresponding high availability mechanism design? That is, if the node
registering the local listener fails, what state will the cluster be?

If do not register a local listener, but write the business code in the
remote filter and return false, is this a good practice?


38797715 38797715
Reply | Threaded
Open this post in threaded view
|

Re: High availability of local listeners for ContinuousQuery or Events

Hi Igor,

We hope that if the local listener node fails, we can have a mechanism similar to fail over. Otherwise, if the local listener node fails and restarts, the events during the failure will be lost.

在 2020/10/30 上午12:04, Igor Belyakov 写道:
Hi,

In case the node, which registered a continuous query fails, the continuous query will be undeployed from the cluster. The cluster state won't be changed.

It's not a good practice to write the business code in a remote filter. Could you please clarify more details regarding your use case?

Igor

On Thu, Oct 29, 2020 at 4:46 PM 38797715 <[hidden email]> wrote:
Hi community,

For local listeners registered for ContinuousQuery and Events, is there
a corresponding high availability mechanism design? That is, if the node
registering the local listener fails, what state will the cluster be?

If do not register a local listener, but write the business code in the
remote filter and return false, is this a good practice?


Stanislav Lukyanov Stanislav Lukyanov
Reply | Threaded
Open this post in threaded view
|

Re: High availability of local listeners for ContinuousQuery or Events

Hi,

It's true that currently you need to implement something for Continuous Queries failover in your application code.
Continuous Queries have setInitialQuery API to help with that.

How it's supposed to work is: you define an initial query (e.g. an SQL query) which fetches the data that's already in the cache when the CQ is being registered.
In case of failover, your initial query should return the data your CQ listener has missed (or might have missed) while it was down.
For example, the following design would work
- The cache has a "last update time" field and an SQL index over that field
- The initial query of the CQ queries all data updated in the last 30 minutes; it will be fast because of the index
- After you've started the CQ, you process the initial query results; if your CQ listener was down for less than 30 minutes, it will receive all updates it has missed (plus some that it has already seen - it needs to filter them out or just be idempotent)

I'm sure that an out-of-the-box HA mechanism for CQ listeners will be added in the future versions of the platform.

Stan

On 30 Oct 2020, at 11:01, 38797715 <[hidden email]> wrote:

Hi Igor,

We hope that if the local listener node fails, we can have a mechanism similar to fail over. Otherwise, if the local listener node fails and restarts, the events during the failure will be lost.

在 2020/10/30 上午12:04, Igor Belyakov 写道:
Hi,

In case the node, which registered a continuous query fails, the continuous query will be undeployed from the cluster. The cluster state won't be changed.

It's not a good practice to write the business code in a remote filter. Could you please clarify more details regarding your use case?

Igor

On Thu, Oct 29, 2020 at 4:46 PM 38797715 <[hidden email]> wrote:
Hi community,

For local listeners registered for ContinuousQuery and Events, is there
a corresponding high availability mechanism design? That is, if the node
registering the local listener fails, what state will the cluster be?

If do not register a local listener, but write the business code in the
remote filter and return false, is this a good practice?