How to batch SQL queries?

classic Classic list List threaded Threaded
6 messages Options
javadevmtl javadevmtl
Reply | Threaded
Open this post in threaded view
|

How to batch SQL queries?

Is there a way to batch a bunch of SQL queries together and execute them in parallel?

I was thinking of using IgniteCallable and executing those all in parallel, but not sure how. Is it the right way?


Thanks
Sergi Vladykin Sergi Vladykin
Reply | Threaded
Open this post in threaded view
|

Re: How to batch SQL queries?

What do you mean by batch? In JDBC world you usually batch updates but not queries.

Sergi

2015-08-13 23:18 GMT+03:00 javadevmtl <[hidden email]>:
Is there a way to batch a bunch of SQL queries together and execute them in
parallel?

I was thinking of using IgniteCallable and executing those all in parallel,
but not sure how. Is it the right way?


Thanks



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-batch-SQL-queries-tp952.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

javadevmtl javadevmtl
Reply | Threaded
Open this post in threaded view
|

Re: How to batch SQL queries?

Oh sorry, let me rephrase it. I would like to run 5 sql queries in parallel and then present the result to "client" application rather then run them 1 by 1

So I wandering hot to do it with compute() and closures...
Sergi Vladykin Sergi Vladykin
Reply | Threaded
Open this post in threaded view
|

Re: How to batch SQL queries?

The easiest way I see is to setup your own thread pool for these parallel queries and run queries inside of a Callable.
Note that when cursor is returned from a query the query was not actually executed, it will be executed when you
will call iterator() or getAll() methods. Thus it makes sense to return getAll() from Callable but not Cursor.

Sergi



2015-08-14 19:54 GMT+03:00 javadevmtl <[hidden email]>:
Oh sorry, let me rephrase it. I would like to run 5 sql queries in parallel
and then present the result to "client" application rather then run them 1
by 1

So I wandering hot to do it with compute() and closures...



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-batch-SQL-queries-tp952p983.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

javadevmtl javadevmtl
Reply | Threaded
Open this post in threaded view
|

Re: How to batch SQL queries?

So this wont work?

Collection<IgniteCallable<String>> calls = new ArrayList<>();

calls.add(() -> {
        String sql = "SELECT SOMFIELD FROM SOMETHING WHERE SOMEOTHERFIELD = ?";
                                                               
        QueryCursor<List<?>> cursor = cache.query(new SqlFieldsQuery(sql).setArgs(someValue));
                                                               
        return "" + cursor.getAll();
});
                                               
                                               
compute.call(calls);
                                               
ComputeTaskFuture<Collection<String>> fut = compute.future();


Sergi Vladykin Sergi Vladykin
Reply | Threaded
Open this post in threaded view
|

Re: How to batch SQL queries?

This should work, but this will distribute queries to remote nodes before execution. I don't know details of your application but usually this way you will introduce a performance penalty (until you are running complex query with expensive reduce part, which you want to distribute to other nodes in cluster).

Sergi

2015-08-17 21:21 GMT+03:00 javadevmtl <[hidden email]>:
So this wont work?

Collection<IgniteCallable&lt;String>> calls = new ArrayList<>();

calls.add(() -> {
        String sql = "SELECT SOMFIELD FROM SOMETHING WHERE SOMEOTHERFIELD = ?";

        QueryCursor<List&lt;?>> cursor = cache.query(new
SqlFieldsQuery(sql).setArgs(someValue));

        return "" + cursor.getAll();
});


compute.call(calls);

ComputeTaskFuture<Collection&lt;String>> fut = compute.future();






--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-batch-SQL-queries-tp952p1001.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.