Apache Ignite support for continuous queries over sliding windows

classic Classic list List threaded Threaded
4 messages Options
stefano.rebora stefano.rebora
Reply | Threaded
Open this post in threaded view
|

Apache Ignite support for continuous queries over sliding windows

Hi,

I'm investigating the use of Apache Ignite for real-time stream processing.
In particular, I'm interested in applying continuous queries for computing
the
average of some values over a sliding window.

I've some questions:

1) Is Apache Ignite a suitable technology for this kind of scenario?

2) Does Apache Ignite support continuous queries over sliding windows?
In Ignite doc for v2.7 sliding windows are not mentioned, but in some
previous versions sliding windows are configured as Ignite cache eviction
policies.

3) If 2) is yes, how to implement the continuous query for computing the
mean value?
Is there a suggest pattern to follow? It seems to me that continuous queries
support
only remote filter.

Thanks in advance.

Stefano



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Nikolay Izhikov Nikolay Izhikov
Reply | Threaded
Open this post in threaded view
|

Re: Apache Ignite support for continuous queries over sliding windows

Hello, Stefano.

> Does Apache Ignite support continuous queries over sliding windows?

Seems, you can easily implement this on top of regular ContinuousQuery.
You can combine several event batches based on your logic and apply it to your internal consumer.

пт, 19 апр. 2019 г. в 11:08, stefano.rebora <[hidden email]>:
Hi,

I'm investigating the use of Apache Ignite for real-time stream processing.
In particular, I'm interested in applying continuous queries for computing
the
average of some values over a sliding window.

I've some questions:

1) Is Apache Ignite a suitable technology for this kind of scenario?

2) Does Apache Ignite support continuous queries over sliding windows?
In Ignite doc for v2.7 sliding windows are not mentioned, but in some
previous versions sliding windows are configured as Ignite cache eviction
policies.

3) If 2) is yes, how to implement the continuous query for computing the
mean value?
Is there a suggest pattern to follow? It seems to me that continuous queries
support
only remote filter.

Thanks in advance.

Stefano



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
stefano.rebora stefano.rebora
Reply | Threaded
Open this post in threaded view
|

Re: Apache Ignite support for continuous queries over sliding windows

Thank you for the answer.

What exactly do you mean by "combining several event batches based on your
logic"? Can you give me an example, please? Can Ignite cache eviction
policies be useful?

Does any one else have any suggestions about questions 1) and 3) ?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Apache Ignite support for continuous queries over sliding windows

Hello!

Yes, you can do it via continuous query. Apache Ignite will send every entry to your continuous query but you will have to implement windowing yourself.

I.e. remember the last N entries in a local data structure, discard entries that are no longer needed.

Regards,
--
Ilya Kasnacheev


вт, 23 апр. 2019 г. в 18:03, stefano.rebora <[hidden email]>:
Thank you for the answer.

What exactly do you mean by "combining several event batches based on your
logic"? Can you give me an example, please? Can Ignite cache eviction
policies be useful?

Does any one else have any suggestions about questions 1) and 3) ?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/