Query execution in ignite

classic Classic list List threaded Threaded
5 messages Options
prasadbhalerao1983 prasadbhalerao1983
Reply | Threaded
Open this post in threaded view
|

Query execution in ignite

Hi,

When SQL is executed, does ignite always scan only primary partitions of all available nodes in cluster irrespective of cache mode partitioned or replicated?



Thanks ,
Prasad
akorensh akorensh
Reply | Threaded
Open this post in threaded view
|

RE: Query execution in ignite

Hi,

 

   If a query is executed against a fully REPLICATED data then Ignite will send it to a single cluster node and run it over the local data there.

 

 

 

 if a query is executed over a PARTITIONED data, then the execution flow will be the following:

The query will be parsed and split into multiple map queries and a single reduce query.

·         All the map queries are executed on all the nodes where required data resides.

·         All the nodes provide result sets of local execution to the query initiator (reducer) that, in turn, will accomplish the reduce phase by properly merging provided result sets.

 

 

 

 More information here: https://apacheignite-sql.readme.io/docs/how-ignite-sql-works

Thanks, Alex

 

From: Prasad Bhalerao <[hidden email]>
Sent: Friday, October 25, 2019 1:31 AM
To: [hidden email]
Subject: Query execution in ignite

 

Hi,

 

When SQL is executed, does ignite always scan only primary partitions of all available nodes in cluster irrespective of cache mode partitioned or replicated?

 

 

 

Thanks ,

Prasad

prasadbhalerao1983 prasadbhalerao1983
Reply | Threaded
Open this post in threaded view
|

Re: Query execution in ignite


Question is specifically about primary and secondary partitions.

So in case of replicated cache ignite scans primary and secondary partitions of any one of the node of the cluster to to fetch the data.

But is it the case with partitioned cache.
I mean in case of partitioned cache when SQL is executed, does ignite scan primary as well as secondary partitions of each node in the cluster or it just scans primary partitions of all the nodes in the cluster as the query is being executed on all nodes?

On Fri 25 Oct, 2019, 10:46 PM <[hidden email] wrote:

Hi,

 

   If a query is executed against a fully REPLICATED data then Ignite will send it to a single cluster node and run it over the local data there.

 

 

 

 if a query is executed over a PARTITIONED data, then the execution flow will be the following:

The query will be parsed and split into multiple map queries and a single reduce query.

·         All the map queries are executed on all the nodes where required data resides.

·         All the nodes provide result sets of local execution to the query initiator (reducer) that, in turn, will accomplish the reduce phase by properly merging provided result sets.

 

 

 

 More information here: https://apacheignite-sql.readme.io/docs/how-ignite-sql-works

Thanks, Alex

 

From: Prasad Bhalerao <[hidden email]>
Sent: Friday, October 25, 2019 1:31 AM
To: [hidden email]
Subject: Query execution in ignite

 

Hi,

 

When SQL is executed, does ignite always scan only primary partitions of all available nodes in cluster irrespective of cache mode partitioned or replicated?

 

 

 

Thanks ,

Prasad

akorensh akorensh
Reply | Threaded
Open this post in threaded view
|

RE: Query execution in ignite

In PARTITIONED mode SQL queries are executed over each node’s primary partitions only.

 

Here is more information about distributed joins: https://apacheignite-sql.readme.io/docs/distributed-joins

 

 

 

From: Prasad Bhalerao <[hidden email]>
Sent: Saturday, October 26, 2019 12:25 AM
To: [hidden email]
Subject: Re: Query execution in ignite

 

 

Question is specifically about primary and secondary partitions.

 

So in case of replicated cache ignite scans primary and secondary partitions of any one of the node of the cluster to to fetch the data.

 

But is it the case with partitioned cache.

I mean in case of partitioned cache when SQL is executed, does ignite scan primary as well as secondary partitions of each node in the cluster or it just scans primary partitions of all the nodes in the cluster as the query is being executed on all nodes?

 

On Fri 25 Oct, 2019, 10:46 PM <[hidden email] wrote:

Hi,

 

   If a query is executed against a fully REPLICATED data then Ignite will send it to a single cluster node and run it over the local data there.

 

 

 

 if a query is executed over a PARTITIONED data, then the execution flow will be the following:

The query will be parsed and split into multiple map queries and a single reduce query.

·         All the map queries are executed on all the nodes where required data resides.

·         All the nodes provide result sets of local execution to the query initiator (reducer) that, in turn, will accomplish the reduce phase by properly merging provided result sets.

 

 

 

 More information here: https://apacheignite-sql.readme.io/docs/how-ignite-sql-works

Thanks, Alex

 

From: Prasad Bhalerao <[hidden email]>
Sent: Friday, October 25, 2019 1:31 AM
To: [hidden email]
Subject: Query execution in ignite

 

Hi,

 

When SQL is executed, does ignite always scan only primary partitions of all available nodes in cluster irrespective of cache mode partitioned or replicated?

 

 

 

Thanks ,

Prasad

Stanislav Lukyanov Stanislav Lukyanov
Reply | Threaded
Open this post in threaded view
|

Re: Query execution in ignite

This is not exactly correct.
When you do an SQL query with only PARTITIONED tables, or with a mix of PARTITIONED and REPLICATED, the data will be taken from the primary partions of PARTITIONED tables and *all* partitions of REPLICATED tables.
When you do an SQL query with only REPLICATED tables, the data will be taken from all partitions (as they're all always available).

E.g. take this query
SELECT * FROM A JOIN B WHERE A.FIELD_A = B.FIELD_B

This query will work correctly if the data is collocated, i.e. in the following cases:
- A and B are both REPLICATED
- A is PARTIONED and B is REPLICATED
- A is REPLICATED and B is PARTITIONED
- A and B are both PARTITIONED with FIELD_A and FIELD_B both being their affinity keys

If A and B are both PARTITIONED but FIELD_A or FIELD_B are not affinity keys then the data is not correctly collocated and the query will return incorrect results.

Stan

Stan

On Mon, Oct 28, 2019 at 9:34 PM <[hidden email]> wrote:

In PARTITIONED mode SQL queries are executed over each node’s primary partitions only.

 

Here is more information about distributed joins: https://apacheignite-sql.readme.io/docs/distributed-joins

 

 

 

From: Prasad Bhalerao <[hidden email]>
Sent: Saturday, October 26, 2019 12:25 AM
To: [hidden email]
Subject: Re: Query execution in ignite

 

 

Question is specifically about primary and secondary partitions.

 

So in case of replicated cache ignite scans primary and secondary partitions of any one of the node of the cluster to to fetch the data.

 

But is it the case with partitioned cache.

I mean in case of partitioned cache when SQL is executed, does ignite scan primary as well as secondary partitions of each node in the cluster or it just scans primary partitions of all the nodes in the cluster as the query is being executed on all nodes?

 

On Fri 25 Oct, 2019, 10:46 PM <[hidden email] wrote:

Hi,

 

   If a query is executed against a fully REPLICATED data then Ignite will send it to a single cluster node and run it over the local data there.

 

 

 

 if a query is executed over a PARTITIONED data, then the execution flow will be the following:

The query will be parsed and split into multiple map queries and a single reduce query.

·         All the map queries are executed on all the nodes where required data resides.

·         All the nodes provide result sets of local execution to the query initiator (reducer) that, in turn, will accomplish the reduce phase by properly merging provided result sets.

 

 

 

 More information here: https://apacheignite-sql.readme.io/docs/how-ignite-sql-works

Thanks, Alex

 

From: Prasad Bhalerao <[hidden email]>
Sent: Friday, October 25, 2019 1:31 AM
To: [hidden email]
Subject: Query execution in ignite

 

Hi,

 

When SQL is executed, does ignite always scan only primary partitions of all available nodes in cluster irrespective of cache mode partitioned or replicated?

 

 

 

Thanks ,

Prasad