Continuous SqlListQuery?

classic Classic list List threaded Threaded
4 messages Options
Mirko Raner Mirko Raner
Reply | Threaded
Open this post in threaded view
|

Continuous SqlListQuery?

I noticed that continuous queries can only be configured for query types that implement Query<Cache.Entry<K, V>>, i.e. ScanQuery, TextQuery, and SqlQuery.
We have the problem that we need to extract only one or two fields out of a cache that has rather wide rows, so SqlQuery would return lots of unnecessary columns for us, but SqlListQuery fits the bill.
Also, the architecture that we built on top of Ignite is built exclusively with continuous queries in mind (in other words, a non-continuous query does not fit nicely into our current model).
I'm relatively new to Ignite, so I'm wondering if this could be solved by creating my own type of wrapper query that adapts SqlListQuery to Query<Cache.Entry<K, V>>, but I'm not sure how that would interact with Ignite's query distribution and serialization.
Can someone confirm that this kind of approach could work, or if not, suggest an alternative route?
vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|

Re: Continuous SqlListQuery?

Hi Mirko,

Continuous query is predicate-based only (i.e., client receives notifications for updates in cache filtered by provided predicate). The 'initialQuery' you're talking about gives you the initial snapshot - the state of the cache at the moment of execution.

Are you executing queries on application startup? If this is the case, you won't need initial query at all.

Does this make sense for you?

-Val
Mirko Raner Mirko Raner
Reply | Threaded
Open this post in threaded view
|

Re: Continuous SqlListQuery?

Thanks for the prompt reply, Val.
In short, I need a continuous SqlListQuery, but it looks like that is not supported by Ignite at the moment.
We have a cache that consists of lots of rows and columns and we need to extract a list of unique values for only a handful of fields. We would like to use SqlListQuery because we're not interested in the entire rows (which consist of 50+ fields in our case). We are using continuous queries to stream updates to the client side (the continuous query is triggered by the initial client request, not at start-up).
I already looked at some of the internal Ignite code, and it seems that type conformance to Query<Cache.Entry<K, V>> is only a superficial problem. Providing a custom implementation of that interface will not solve my issue. The real problem is that there is code in Ignite that performs instanceof checks for specific query implementations (e.g., in IgniteCacheProxy), which means that Ignite only supports query implementations that it knows.
So, my initial query wrapper idea is off the table. Do you have another suggestion how we can execute continuous SQL queries that do not return complete cache rows but only a few specific fields?
vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|

Re: Continuous SqlListQuery?

Mirko,

You're right, this is currently not supported for continuous queries, so you can listen only for full value objects.

This will be fixed when we add transformers to continuous queries, I'm going to do this in one of the next releases. Here is the ticket that you can track: https://issues.apache.org/jira/browse/IGNITE-425

-Val