Role of H2 datbase in Apache Ignite

classic Classic list List threaded Threaded
6 messages Options
Malashree Malashree
Reply | Threaded
Open this post in threaded view
|

Role of H2 datbase in Apache Ignite

Role of H2 datbase in Apache Ignite and how it helps



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Mikhail Mikhail
Reply | Threaded
Open this post in threaded view
|

Re: Role of H2 datbase in Apache Ignite

Hi,

Could you please formulate your question? Because right not your message
looks like a request for google.
I think the following article has answer for your question:
https://apacheignite-sql.readme.io/docs/how-ignite-sql-works

Thanks,
Mike.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
eugene miretsky eugene miretsky
Reply | Threaded
Open this post in threaded view
|

Re: Role of H2 datbase in Apache Ignite

Hello, 

I have been struggling with this question myself for a while now too.
I think the documents are very ambiguous on how exactly H2 is being used.

The document that you linked say 
"Apache Ignite leverages from H2's SQL query parser and optimizer as well as the execution planner. Lastly, H2 executes a query locally on a particular node and passes a local result to a distributed Ignite SQL engine for further processing."

And 
"However, the data, as well as the indexes, are always stored in the Ignite that executes queries in a distributed and fault-tolerant manner which is not supported by H2."

To me, this leaves a lot of ambiguity on how H2 is leveraged on a single Ignite node.  (I get that the Reduce stage, as well as distributed transactions, are handled by Ignite, but how about the 'map' stage on a single node). 

How is a query executed on a single node? 
Example query: Select count(customer_id) from user where (age > 20) group by customer_id

What steps are taken?
  1. execution plan: H2 creates an execution plan
  2. data retrieval:  Since data is stored off-heap, it has to be brought into heap.  Does H2 have anything to do with this step, or is it only Ignite? When are indexes used for that? 
  3. Query execution: Once the data is on heap, what executes the Query (the group_by, aggregations, filters that were not handled by indexes, etc.)? H2 or Ignite? 



On Fri, Sep 21, 2018 at 9:27 AM Mikhail <[hidden email]> wrote:
Hi,

Could you please formulate your question? Because right not your message
looks like a request for google.
I think the following article has answer for your question:
https://apacheignite-sql.readme.io/docs/how-ignite-sql-works

Thanks,
Mike.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Stanislav Lukyanov Stanislav Lukyanov
Reply | Threaded
Open this post in threaded view
|

RE: Role of H2 datbase in Apache Ignite

In short, Ignite replaces H2’s storage level with its own.

For example, Ignite implements H2’s Index interface with its own off-heap data structures underneath.

When Ignite executes an SQL query, it will ask H2 to process it, then H2 will callback to Ignite’s implementations

of H2’s interfaces (such as Index) to actually retrieve the data.

I guess the on-heap processing is mostly H2, although there is a lot of work done by Ignite to make the distributed

map-reduce work like creating temporary tables for intermediate results.

 

Stan

 

From: [hidden email]
Sent: 9 октября 2018 г. 21:52
To: [hidden email]
Subject: Re: Role of H2 datbase in Apache Ignite

 

Hello, 

 

I have been struggling with this question myself for a while now too.

I think the documents are very ambiguous on how exactly H2 is being used.

 

The document that you linked say 

"Apache Ignite leverages from H2's SQL query parser and optimizer as well as the execution planner. Lastly, H2 executes a query locally on a particular node and passes a local result to a distributed Ignite SQL engine for further processing."

 

And 

"However, the data, as well as the indexes, are always stored in the Ignite that executes queries in a distributed and fault-tolerant manner which is not supported by H2."

 

To me, this leaves a lot of ambiguity on how H2 is leveraged on a single Ignite node.  (I get that the Reduce stage, as well as distributed transactions, are handled by Ignite, but how about the 'map' stage on a single node). 

 

How is a query executed on a single node? 

Example query: Select count(customer_id) from user where (age > 20) group by customer_id

 

What steps are taken?

  1. execution plan: H2 creates an execution plan
  2. data retrieval:  Since data is stored off-heap, it has to be brought into heap.  Does H2 have anything to do with this step, or is it only Ignite? When are indexes used for that? 
  3. Query execution: Once the data is on heap, what executes the Query (the group_by, aggregations, filters that were not handled by indexes, etc.)? H2 or Ignite? 

 

 

 

On Fri, Sep 21, 2018 at 9:27 AM Mikhail <[hidden email]> wrote:

Hi,

Could you please formulate your question? Because right not your message
looks like a request for google.
I think the following article has answer for your question:
https://apacheignite-sql.readme.io/docs/how-ignite-sql-works

Thanks,
Mike.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

 

eugene miretsky eugene miretsky
Reply | Threaded
Open this post in threaded view
|

Re: Role of H2 datbase in Apache Ignite

Thanks! 

So does it mean that CacheConfiguration.queryParallelism is really an H2 settings? 


On Tue, Oct 9, 2018 at 4:27 PM Stanislav Lukyanov <[hidden email]> wrote:

In short, Ignite replaces H2’s storage level with its own.

For example, Ignite implements H2’s Index interface with its own off-heap data structures underneath.

When Ignite executes an SQL query, it will ask H2 to process it, then H2 will callback to Ignite’s implementations

of H2’s interfaces (such as Index) to actually retrieve the data.

I guess the on-heap processing is mostly H2, although there is a lot of work done by Ignite to make the distributed

map-reduce work like creating temporary tables for intermediate results.

 

Stan

 

From: [hidden email]
Sent: 9 октября 2018 г. 21:52
To: [hidden email]
Subject: Re: Role of H2 datbase in Apache Ignite

 

Hello, 

 

I have been struggling with this question myself for a while now too.

I think the documents are very ambiguous on how exactly H2 is being used.

 

The document that you linked say 

"Apache Ignite leverages from H2's SQL query parser and optimizer as well as the execution planner. Lastly, H2 executes a query locally on a particular node and passes a local result to a distributed Ignite SQL engine for further processing."

 

And 

"However, the data, as well as the indexes, are always stored in the Ignite that executes queries in a distributed and fault-tolerant manner which is not supported by H2."

 

To me, this leaves a lot of ambiguity on how H2 is leveraged on a single Ignite node.  (I get that the Reduce stage, as well as distributed transactions, are handled by Ignite, but how about the 'map' stage on a single node). 

 

How is a query executed on a single node? 

Example query: Select count(customer_id) from user where (age > 20) group by customer_id

 

What steps are taken?

  1. execution plan: H2 creates an execution plan
  2. data retrieval:  Since data is stored off-heap, it has to be brought into heap.  Does H2 have anything to do with this step, or is it only Ignite? When are indexes used for that? 
  3. Query execution: Once the data is on heap, what executes the Query (the group_by, aggregations, filters that were not handled by indexes, etc.)? H2 or Ignite? 

 

 

 

On Fri, Sep 21, 2018 at 9:27 AM Mikhail <[hidden email]> wrote:

Hi,

Could you please formulate your question? Because right not your message
looks like a request for google.
I think the following article has answer for your question:
https://apacheignite-sql.readme.io/docs/how-ignite-sql-works

Thanks,
Mike.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

 

Stanislav Lukyanov Stanislav Lukyanov
Reply | Threaded
Open this post in threaded view
|

RE: Role of H2 datbase in Apache Ignite

No, that one actually impacts how indexes are built, which is Ignite’s domain.

 

queryParallellism is the number of index trees that are created by Ignite.

In terms of Ignite-H2 communication, queryParallelism=N means that Ignite will send N parallel queries to H2,

each using a different index tree, then merge them.

 

Stan

 

From: [hidden email]
Sent: 11 октября 2018 г. 20:58
To: [hidden email]
Subject: Re: Role of H2 datbase in Apache Ignite

 

Thanks! 

 

So does it mean that CacheConfiguration.queryParallelism is really an H2 settings? 

 

 

On Tue, Oct 9, 2018 at 4:27 PM Stanislav Lukyanov <[hidden email]> wrote:

In short, Ignite replaces H2’s storage level with its own.

For example, Ignite implements H2’s Index interface with its own off-heap data structures underneath.

When Ignite executes an SQL query, it will ask H2 to process it, then H2 will callback to Ignite’s implementations

of H2’s interfaces (such as Index) to actually retrieve the data.

I guess the on-heap processing is mostly H2, although there is a lot of work done by Ignite to make the distributed

map-reduce work like creating temporary tables for intermediate results.

 

Stan

 

From: [hidden email]
Sent: 9 октября 2018 г. 21:52
To: [hidden email]
Subject: Re: Role of H2 datbase in Apache Ignite

 

Hello, 

 

I have been struggling with this question myself for a while now too.

I think the documents are very ambiguous on how exactly H2 is being used.

 

The document that you linked say 

"Apache Ignite leverages from H2's SQL query parser and optimizer as well as the execution planner. Lastly, H2 executes a query locally on a particular node and passes a local result to a distributed Ignite SQL engine for further processing."

 

And 

"However, the data, as well as the indexes, are always stored in the Ignite that executes queries in a distributed and fault-tolerant manner which is not supported by H2."

 

To me, this leaves a lot of ambiguity on how H2 is leveraged on a single Ignite node.  (I get that the Reduce stage, as well as distributed transactions, are handled by Ignite, but how about the 'map' stage on a single node). 

 

How is a query executed on a single node? 

Example query: Select count(customer_id) from user where (age > 20) group by customer_id

 

What steps are taken?

  1. execution plan: H2 creates an execution plan
  2. data retrieval:  Since data is stored off-heap, it has to be brought into heap.  Does H2 have anything to do with this step, or is it only Ignite? When are indexes used for that? 
  3. Query execution: Once the data is on heap, what executes the Query (the group_by, aggregations, filters that were not handled by indexes, etc.)? H2 or Ignite? 

 

 

 

On Fri, Sep 21, 2018 at 9:27 AM Mikhail <[hidden email]> wrote:

Hi,

Could you please formulate your question? Because right not your message
looks like a request for google.
I think the following article has answer for your question:
https://apacheignite-sql.readme.io/docs/how-ignite-sql-works

Thanks,
Mike.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/