Query about running SQL with durable memory

classic Classic list List threaded Threaded
5 messages Options
iostream iostream
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Query about running SQL with durable memory

This post was updated on .
Hi,

I want to understand how SQL queries work when persistence store is enabled in v2.1.

1. Suppose I have 10 Person entries in the disk, out of which only 5 are in-memory. Now if I run a SQL query which is expected to count the number of entries in Person cache, will the query run only on the disk or RAM or will it run on both? How will the query processor resolve the 5 duplicates on the disk and the RAM?

2. If the query will run on both disk and RAM, how will the query performance be impacted?

3. Suppose I set my cache size limit to 10 entries, will the remaining 5 entries be copied from disk to RAM when I run the SQL count query (as stated in point 1)?

Thanks!

afedotov afedotov
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Query about running SQL with durable memory

Hi,

The count should be 15.
If a page is not found in RAM, then the durable memory will request it from the Persistent Store.
Please check https://apacheignite.readme.io/v2.1/docs/distributed-persistent-store#section-usage

Are you observing a different behavior?


Kind regards,
Alex.

On Sun, Aug 6, 2017 at 8:17 AM, iostream [via Apache Ignite Users] <[hidden email]> wrote:
Hi,

I want to understand how SQL queries work when persistence store is enabled in v2.1.

Suppose I have 10 Person entries in the disk, out of which only 5 are in-memory. Now if I run a SQL query which is expected to count the number of entries in Person cache, will the query run only on the disk or RAM or will it run on both?

If the query will run on both the disk and RAM, will the count be 10 or 15 (10 on disk + 5 in RAM)? Does the SQL processor know which entries are present in-memory to resolve duplicates?

Thanks!




If you reply to this email, your message will be added to the discussion below:
http://apache-ignite-users.70518.x6.nabble.com/Query-about-running-SQL-with-durable-memory-tp16015.html
To start a new topic under Apache Ignite Users, email [hidden email]
To unsubscribe from Apache Ignite Users, click here.
NAML

dsetrakyan dsetrakyan
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Query about running SQL with durable memory

In reply to this post by iostream


On Sat, Aug 5, 2017 at 10:17 PM, iostream <[hidden email]> wrote:
Suppose I have 10 Person entries in the disk, out of which only 5 are
in-memory. Now if I run a SQL query which is expected to count the number of
entries in Person cache, will the query run only on the disk or RAM or will
it run on both?

The SQL query will simply run over the total data set, which is 10 persons, but it will obviously process the data that is cached in-memory faster. You can also speed up SQL queries by indexing the data.
 

If the query will run on both the disk and RAM, will the count be 10 or 15
(10 on disk + 5 in RAM)? Does the SQL processor know which entries are
present in-memory to resolve duplicates?


If I understand your example correctly, then the super-set of data is 10 Persons (right?). In this case, the total count returned by Ignite will be 10.
 

--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Query-about-running-SQL-with-durable-memory-tp16015.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Dmitri Bronnikov Dmitri Bronnikov
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Query about running SQL with durable memory

If memory cache is empty and all 10 entries are on disk, will Ignite de-serialize all of them? Can de-serialization be avoided if the query has a filter, e.g. SELECT COUNT(*) FROM EMP WHERE SALARY > ?
vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Query about running SQL with durable memory

Dmitry,

When you execute a SQL query, there is no deserialization at all. Ignite stores data in binary format that allows to lookup fields without full deserialization. So only required fields are fetched from the objects.

https://apacheignite.readme.io/docs/binary-marshaller

-Val
Loading...