Closure method on a Cache Query

classic Classic list List threaded Threaded
6 messages Options
fluffy fluffy
Reply | Threaded
Open this post in threaded view
|

Closure method on a Cache Query

Using GridGain, I used to be able to associate a GridClosure method with a GridCacheQuery. You could simply pass this Closure method into the GridCacheQuery.execute() function and it would perform a function on each cache element that matched the SQL query.

This basically consolidated a number of tasks:

- Instead of receiving entire objects from a query, a much smaller result value was sent back to the node that initiated the query
- Allowed for the specification of some functionality within each Cache Element rather than being a fairly dull data store
- Allowed for distributed processing of Cache Element information on the node that each valid element existed before being aggregated/merged on the calling node

I do not see this functionality as having been ported to the Ignite release. At least not directly. Is there a way to do this in the current Ignite version?

I was looking at either a ScanQuery or using the AffinityRun methods, but the later don't seem to allow me to perform an SQL query first to limit the Cache Elements....
alexey.goncharuk alexey.goncharuk
Reply | Threaded
Open this post in threaded view
|

Re: Closure method on a Cache Query

I agree that there is a lack of functionality here, I will create a ticket to address that. However, closure broadcast should do the trick:

You can broadcast an IgniteCallable to all cache nodes and run a local SQL query (see setLocal method on the Query object) to limit your result set only to the entries local for the node. After transformation/aggregation of the entries you will return the results which will be transferred to the caller.
alexey.goncharuk alexey.goncharuk
Reply | Threaded
Open this post in threaded view
|

Re: Closure method on a Cache Query

In reply to this post by fluffy
However, if you only need a remote transformer, not a remote reducer, custom SQL function will also work. You can access keys and values in SQL fields queries using built-in '_key' and '_val' names.

To use a custom SQL function, you need to declare a class which will contain your custom logic
public class SqlFunctions {
    @QuerySqlFunction
    public static int pow(int x, int n) {
        return (int)Math.pow(x, n);
    }
}
Next, you need to register this class in cache configuration using setSqlFunctionClasses method.

After this you can access this function like any other built-in SQL function, like this: "select pow(_val, 3) from Integer where _val > 5"
dsetrakyan dsetrakyan
Reply | Threaded
Open this post in threaded view
|

Re: Closure method on a Cache Query

In reply to this post by fluffy
Also, if you need to return only a portion of the object (just a few fields), you can use SqlFieldsQuery and select only specific fields:

select a, b, c from Person...

D.

On Tue, Jun 9, 2015 at 1:20 PM, fluffy <[hidden email]> wrote:
Using GridGain, I used to be able to associate a GridClosure method with a
GridCacheQuery. You could simply pass this Closure method into the
GridCacheQuery.execute() function and it would perform a function on each
cache element that matched the SQL query.

This basically consolidated a number of tasks:

- Instead of receiving entire objects from a query, a much smaller result
value was sent back to the node that initiated the query
- Allowed for the specification of some functionality within each Cache
Element rather than being a fairly dull data store
- Allowed for distributed processing of Cache Element information on the
node that each valid element existed before being aggregated/merged on the
calling node

I do not see this functionality as having been ported to the Ignite release.
At least not directly. Is there a way to do this in the current Ignite
version?

I was looking at either a ScanQuery or using the AffinityRun methods, but
the later don't seem to allow me to perform an SQL query first to limit the
Cache Elements....



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Closure-method-on-a-Cache-Query-tp456.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

yakov yakov
Reply | Threaded
Open this post in threaded view
|

Re: Closure method on a Cache Query

I am crossposting this to dev list as well.

So, is there a way to introduce custom aggregate function? I assume no.

But even if we have such ability, writing and plugging such custom function seem to  be much harder to do than direct reducer/transformer from java code. However, this is not very common usecase in SQL, I think.

I think it would be very nice to support custom aggregate functions. As far as java-reducers - this is not very intuitive and I would skip adding them back if custom aggregate is possible.

Sergi, Alex, can you please share your opinions?

--Yakov

2015-06-09 22:21 GMT+03:00 Dmitriy Setrakyan <[hidden email]>:
Also, if you need to return only a portion of the object (just a few fields), you can use SqlFieldsQuery and select only specific fields:

select a, b, c from Person...

D.

On Tue, Jun 9, 2015 at 1:20 PM, fluffy <[hidden email]> wrote:
Using GridGain, I used to be able to associate a GridClosure method with a
GridCacheQuery. You could simply pass this Closure method into the
GridCacheQuery.execute() function and it would perform a function on each
cache element that matched the SQL query.

This basically consolidated a number of tasks:

- Instead of receiving entire objects from a query, a much smaller result
value was sent back to the node that initiated the query
- Allowed for the specification of some functionality within each Cache
Element rather than being a fairly dull data store
- Allowed for distributed processing of Cache Element information on the
node that each valid element existed before being aggregated/merged on the
calling node

I do not see this functionality as having been ported to the Ignite release.
At least not directly. Is there a way to do this in the current Ignite
version?

I was looking at either a ScanQuery or using the AffinityRun methods, but
the later don't seem to allow me to perform an SQL query first to limit the
Cache Elements....



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Closure-method-on-a-Cache-Query-tp456.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.


fluffy fluffy
Reply | Threaded
Open this post in threaded view
|

Re: Closure method on a Cache Query

In reply to this post by alexey.goncharuk
Thanks Alexey. This SQLFunctions capability pretty much replicates what I had with GridGain. It's nice in that it is now a 'library' capability set that can be used. But it is also a bit more cumbersome to setup since all cache object values now need to be passed to an external routine rather than the object doing the math on itself.

My main interest is simply a transformation, not a reduction...

Thanks again