off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

classic Classic list List threaded Threaded
31 messages Options
12
tomk tomk
Reply | Threaded
Open this post in threaded view
|

off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

Hello,
in doces it is written:  
To improve performance of SQL queries with off-heap enabled, you can try to increase value of property CacheConfiguration.setSqlOnheapRowCacheSize which can be low by default 10 000.


(in my xml file I set (only for server configuration - no jdbc client)                    
)


How does it work ? How much this number should be ?
vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

Are you using offheap? What is your data size?

Generally, I would recommend to use on-heap with SQL queries if this possible (unless you have a very big data sets and want to avoid having large heap sizes). If you still have to use offheap, you can try playing with this parameter and see what performance you get with different values. The optimal value depends on a particular application.

-Val
Alexei Scherbakov Alexei Scherbakov
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

Hi,

Generally speaking, settings setSqlOnheapRowCacheSize to larger value increases
SQL performance in OFFHEAP_TIERED mode, but also means more job for GC,
so it should be used with care.

The value should be set to the size of your application's working(frequently accessed) data set.

2016-05-23 13:07 GMT+03:00 vkulichenko <[hidden email]>:
Are you using offheap? What is your data size?

Generally, I would recommend to use on-heap with SQL queries if this
possible (unless you have a very big data sets and want to avoid having
large heap sizes). If you still have to use offheap, you can try playing
with this parameter and see what performance you get with different values.
The optimal value depends on a particular application.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/off-heap-indexes-setSqlOnheapRowCacheSize-how-does-it-improve-efficiency-tp5070p5092.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.



--

Best regards,
Alexei Scherbakov
tomk tomk
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

Ok, I am going to use OFF_HEAP.     


On each node I am going to use about 10 GB.  (My ram is 16GB). 
Can you help me adjust configuration for this aim ? 
It is very important for me. 
Aim: Extremely fast sql quries.


2016-05-23 18:13 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Hi,

Generally speaking, settings setSqlOnheapRowCacheSize to larger value increases
SQL performance in OFFHEAP_TIERED mode, but also means more job for GC,
so it should be used with care.

The value should be set to the size of your application's working(frequently accessed) data set.

2016-05-23 13:07 GMT+03:00 vkulichenko <[hidden email]>:
Are you using offheap? What is your data size?

Generally, I would recommend to use on-heap with SQL queries if this
possible (unless you have a very big data sets and want to avoid having
large heap sizes). If you still have to use offheap, you can try playing
with this parameter and see what performance you get with different values.
The optimal value depends on a particular application.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/off-heap-indexes-setSqlOnheapRowCacheSize-how-does-it-improve-efficiency-tp5070p5092.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.



--

Best regards,
Alexei Scherbakov

Alexei Scherbakov Alexei Scherbakov
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

Try to start with some larger number, if default value is too low for you. 
On example, set it to 50000, and see if the performance is OK.
If not, increase to 100000 etc.
I can't help you further without knowing your data access patterns.

BTW, for 10G heap it is probably better to use ONHEAP_TIERED, as Val suggested.
Don't forget to tune GC as described here:



2016-05-23 22:05 GMT+03:00 Tomek W <[hidden email]>:
Ok, I am going to use OFF_HEAP.     


On each node I am going to use about 10 GB.  (My ram is 16GB). 
Can you help me adjust configuration for this aim ? 
It is very important for me. 
Aim: Extremely fast sql quries.


2016-05-23 18:13 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Hi,

Generally speaking, settings setSqlOnheapRowCacheSize to larger value increases
SQL performance in OFFHEAP_TIERED mode, but also means more job for GC,
so it should be used with care.

The value should be set to the size of your application's working(frequently accessed) data set.

2016-05-23 13:07 GMT+03:00 vkulichenko <[hidden email]>:
Are you using offheap? What is your data size?

Generally, I would recommend to use on-heap with SQL queries if this
possible (unless you have a very big data sets and want to avoid having
large heap sizes). If you still have to use offheap, you can try playing
with this parameter and see what performance you get with different values.
The optimal value depends on a particular application.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/off-heap-indexes-setSqlOnheapRowCacheSize-how-does-it-improve-efficiency-tp5070p5092.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.



--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov
tomk tomk
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

Sorry, 
I made a mistake - I wanted to say that I am going to use ON_HEAP.
Can you suggest my more details about tuning ? 
I have client (which run hot loading data from postgresql) and server node (keep cache - data).  
Now client also requires ~4GB data. Why ?   After all, it doesn't keep data, only run hot loading. 





2016-05-24 13:44 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Try to start with some larger number, if default value is too low for you. 
On example, set it to 50000, and see if the performance is OK.
If not, increase to 100000 etc.
I can't help you further without knowing your data access patterns.

BTW, for 10G heap it is probably better to use ONHEAP_TIERED, as Val suggested.
Don't forget to tune GC as described here:



2016-05-23 22:05 GMT+03:00 Tomek W <[hidden email]>:
Ok, I am going to use OFF_HEAP.     


On each node I am going to use about 10 GB.  (My ram is 16GB). 
Can you help me adjust configuration for this aim ? 
It is very important for me. 
Aim: Extremely fast sql quries.


2016-05-23 18:13 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Hi,

Generally speaking, settings setSqlOnheapRowCacheSize to larger value increases
SQL performance in OFFHEAP_TIERED mode, but also means more job for GC,
so it should be used with care.

The value should be set to the size of your application's working(frequently accessed) data set.

2016-05-23 13:07 GMT+03:00 vkulichenko <[hidden email]>:
Are you using offheap? What is your data size?

Generally, I would recommend to use on-heap with SQL queries if this
possible (unless you have a very big data sets and want to avoid having
large heap sizes). If you still have to use offheap, you can try playing
with this parameter and see what performance you get with different values.
The optimal value depends on a particular application.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/off-heap-indexes-setSqlOnheapRowCacheSize-how-does-it-improve-efficiency-tp5070p5092.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.



--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov

Alexei Scherbakov Alexei Scherbakov
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

Just use this:

-server -Xms10G -Xmx10G -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+UseTLAB -XX:NewSize=128m -XX:MaxNewSize=128m -XX:MaxTenuringThreshold=0 -XX:SurvivorRatio=1024 -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+DisableExplicitGC

How do you measure client memory usage ?

2016-05-24 15:04 GMT+03:00 Tomek W <[hidden email]>:
Sorry, 
I made a mistake - I wanted to say that I am going to use ON_HEAP.
Can you suggest my more details about tuning ? 
I have client (which run hot loading data from postgresql) and server node (keep cache - data).  
Now client also requires ~4GB data. Why ?   After all, it doesn't keep data, only run hot loading. 





2016-05-24 13:44 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Try to start with some larger number, if default value is too low for you. 
On example, set it to 50000, and see if the performance is OK.
If not, increase to 100000 etc.
I can't help you further without knowing your data access patterns.

BTW, for 10G heap it is probably better to use ONHEAP_TIERED, as Val suggested.
Don't forget to tune GC as described here:



2016-05-23 22:05 GMT+03:00 Tomek W <[hidden email]>:
Ok, I am going to use OFF_HEAP.     


On each node I am going to use about 10 GB.  (My ram is 16GB). 
Can you help me adjust configuration for this aim ? 
It is very important for me. 
Aim: Extremely fast sql quries.


2016-05-23 18:13 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Hi,

Generally speaking, settings setSqlOnheapRowCacheSize to larger value increases
SQL performance in OFFHEAP_TIERED mode, but also means more job for GC,
so it should be used with care.

The value should be set to the size of your application's working(frequently accessed) data set.

2016-05-23 13:07 GMT+03:00 vkulichenko <[hidden email]>:
Are you using offheap? What is your data size?

Generally, I would recommend to use on-heap with SQL queries if this
possible (unless you have a very big data sets and want to avoid having
large heap sizes). If you still have to use offheap, you can try playing
with this parameter and see what performance you get with different values.
The optimal value depends on a particular application.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/off-heap-indexes-setSqlOnheapRowCacheSize-how-does-it-improve-efficiency-tp5070p5092.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.



--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov
tomk tomk
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

I didn't measure it.  I got GC overstack error, so I increased until it works. 

Nevertheless, it is still unclear why it requires so much memory.


2016-05-24 14:28 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Just use this:

-server -Xms10G -Xmx10G -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+UseTLAB -XX:NewSize=128m -XX:MaxNewSize=128m -XX:MaxTenuringThreshold=0 -XX:SurvivorRatio=1024 -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+DisableExplicitGC

How do you measure client memory usage ?

2016-05-24 15:04 GMT+03:00 Tomek W <[hidden email]>:
Sorry, 
I made a mistake - I wanted to say that I am going to use ON_HEAP.
Can you suggest my more details about tuning ? 
I have client (which run hot loading data from postgresql) and server node (keep cache - data).  
Now client also requires ~4GB data. Why ?   After all, it doesn't keep data, only run hot loading. 





2016-05-24 13:44 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Try to start with some larger number, if default value is too low for you. 
On example, set it to 50000, and see if the performance is OK.
If not, increase to 100000 etc.
I can't help you further without knowing your data access patterns.

BTW, for 10G heap it is probably better to use ONHEAP_TIERED, as Val suggested.
Don't forget to tune GC as described here:



2016-05-23 22:05 GMT+03:00 Tomek W <[hidden email]>:
Ok, I am going to use OFF_HEAP.     


On each node I am going to use about 10 GB.  (My ram is 16GB). 
Can you help me adjust configuration for this aim ? 
It is very important for me. 
Aim: Extremely fast sql quries.


2016-05-23 18:13 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Hi,

Generally speaking, settings setSqlOnheapRowCacheSize to larger value increases
SQL performance in OFFHEAP_TIERED mode, but also means more job for GC,
so it should be used with care.

The value should be set to the size of your application's working(frequently accessed) data set.

2016-05-23 13:07 GMT+03:00 vkulichenko <[hidden email]>:
Are you using offheap? What is your data size?

Generally, I would recommend to use on-heap with SQL queries if this
possible (unless you have a very big data sets and want to avoid having
large heap sizes). If you still have to use offheap, you can try playing
with this parameter and see what performance you get with different values.
The optimal value depends on a particular application.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/off-heap-indexes-setSqlOnheapRowCacheSize-how-does-it-improve-efficiency-tp5070p5092.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.



--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov

tomk tomk
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

In reply to this post by Alexei Scherbakov
Thanks for you answer.


I didn't measure it. Simply, I was getting GC overflow error, so I succesively increased it. 
How to measure it ? Why client requires so much memory ?

2016-05-24 14:28 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Just use this:

-server -Xms10G -Xmx10G -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+UseTLAB -XX:NewSize=128m -XX:MaxNewSize=128m -XX:MaxTenuringThreshold=0 -XX:SurvivorRatio=1024 -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+DisableExplicitGC

How do you measure client memory usage ?

2016-05-24 15:04 GMT+03:00 Tomek W <[hidden email]>:
Sorry, 
I made a mistake - I wanted to say that I am going to use ON_HEAP.
Can you suggest my more details about tuning ? 
I have client (which run hot loading data from postgresql) and server node (keep cache - data).  
Now client also requires ~4GB data. Why ?   After all, it doesn't keep data, only run hot loading. 





2016-05-24 13:44 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Try to start with some larger number, if default value is too low for you. 
On example, set it to 50000, and see if the performance is OK.
If not, increase to 100000 etc.
I can't help you further without knowing your data access patterns.

BTW, for 10G heap it is probably better to use ONHEAP_TIERED, as Val suggested.
Don't forget to tune GC as described here:



2016-05-23 22:05 GMT+03:00 Tomek W <[hidden email]>:
Ok, I am going to use OFF_HEAP.     


On each node I am going to use about 10 GB.  (My ram is 16GB). 
Can you help me adjust configuration for this aim ? 
It is very important for me. 
Aim: Extremely fast sql quries.


2016-05-23 18:13 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Hi,

Generally speaking, settings setSqlOnheapRowCacheSize to larger value increases
SQL performance in OFFHEAP_TIERED mode, but also means more job for GC,
so it should be used with care.

The value should be set to the size of your application's working(frequently accessed) data set.

2016-05-23 13:07 GMT+03:00 vkulichenko <[hidden email]>:
Are you using offheap? What is your data size?

Generally, I would recommend to use on-heap with SQL queries if this
possible (unless you have a very big data sets and want to avoid having
large heap sizes). If you still have to use offheap, you can try playing
with this parameter and see what performance you get with different values.
The optimal value depends on a particular application.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/off-heap-indexes-setSqlOnheapRowCacheSize-how-does-it-improve-efficiency-tp5070p5092.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.



--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov

Alexei Scherbakov Alexei Scherbakov
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

Have you configured near cache on the client?
Do you buffer data somewere ?
Share the details of the loading process.


2016-05-24 16:07 GMT+03:00 Tomek W <[hidden email]>:
Thanks for you answer.


I didn't measure it. Simply, I was getting GC overflow error, so I succesively increased it. 
How to measure it ? Why client requires so much memory ?

2016-05-24 14:28 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Just use this:

-server -Xms10G -Xmx10G -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+UseTLAB -XX:NewSize=128m -XX:MaxNewSize=128m -XX:MaxTenuringThreshold=0 -XX:SurvivorRatio=1024 -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+DisableExplicitGC

How do you measure client memory usage ?

2016-05-24 15:04 GMT+03:00 Tomek W <[hidden email]>:
Sorry, 
I made a mistake - I wanted to say that I am going to use ON_HEAP.
Can you suggest my more details about tuning ? 
I have client (which run hot loading data from postgresql) and server node (keep cache - data).  
Now client also requires ~4GB data. Why ?   After all, it doesn't keep data, only run hot loading. 





2016-05-24 13:44 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Try to start with some larger number, if default value is too low for you. 
On example, set it to 50000, and see if the performance is OK.
If not, increase to 100000 etc.
I can't help you further without knowing your data access patterns.

BTW, for 10G heap it is probably better to use ONHEAP_TIERED, as Val suggested.
Don't forget to tune GC as described here:



2016-05-23 22:05 GMT+03:00 Tomek W <[hidden email]>:
Ok, I am going to use OFF_HEAP.     


On each node I am going to use about 10 GB.  (My ram is 16GB). 
Can you help me adjust configuration for this aim ? 
It is very important for me. 
Aim: Extremely fast sql quries.


2016-05-23 18:13 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Hi,

Generally speaking, settings setSqlOnheapRowCacheSize to larger value increases
SQL performance in OFFHEAP_TIERED mode, but also means more job for GC,
so it should be used with care.

The value should be set to the size of your application's working(frequently accessed) data set.

2016-05-23 13:07 GMT+03:00 vkulichenko <[hidden email]>:
Are you using offheap? What is your data size?

Generally, I would recommend to use on-heap with SQL queries if this
possible (unless you have a very big data sets and want to avoid having
large heap sizes). If you still have to use offheap, you can try playing
with this parameter and see what performance you get with different values.
The optimal value depends on a particular application.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/off-heap-indexes-setSqlOnheapRowCacheSize-how-does-it-improve-efficiency-tp5070p5092.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.



--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov
tomk tomk
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

firstly,
look at fragment of my config clieny file that  bother me:
    <bean id="nearCacheBean" class="org.apache.ignite.configuration.NearCacheConfiguration">
    </bean>

    <bean class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="clientMode" value="true"/>

               
          <property name="cacheConfiguration">
            <list>
                <bean class="org.apache.ignite.configuration.CacheConfiguration">
                    <property name="name" value="my_table_cache"/>
                    <property name="cacheMode" value="PARTITIONED"/>
                    <property name="atomicityMode" value="TRANSACTIONAL"/>
                    <property name="backups" value="0"/>
                    <property name="readFromBackup" value="true"/>
                    <property name="copyOnRead" value="true"/>


2016-05-24 15:22 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Have you configured near cache on the client?
Do you buffer data somewere ?
Share the details of the loading process.


2016-05-24 16:07 GMT+03:00 Tomek W <[hidden email]>:
Thanks for you answer.


I didn't measure it. Simply, I was getting GC overflow error, so I succesively increased it. 
How to measure it ? Why client requires so much memory ?

2016-05-24 14:28 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Just use this:

-server -Xms10G -Xmx10G -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+UseTLAB -XX:NewSize=128m -XX:MaxNewSize=128m -XX:MaxTenuringThreshold=0 -XX:SurvivorRatio=1024 -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+DisableExplicitGC

How do you measure client memory usage ?

2016-05-24 15:04 GMT+03:00 Tomek W <[hidden email]>:
Sorry, 
I made a mistake - I wanted to say that I am going to use ON_HEAP.
Can you suggest my more details about tuning ? 
I have client (which run hot loading data from postgresql) and server node (keep cache - data).  
Now client also requires ~4GB data. Why ?   After all, it doesn't keep data, only run hot loading. 





2016-05-24 13:44 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Try to start with some larger number, if default value is too low for you. 
On example, set it to 50000, and see if the performance is OK.
If not, increase to 100000 etc.
I can't help you further without knowing your data access patterns.

BTW, for 10G heap it is probably better to use ONHEAP_TIERED, as Val suggested.
Don't forget to tune GC as described here:



2016-05-23 22:05 GMT+03:00 Tomek W <[hidden email]>:
Ok, I am going to use OFF_HEAP.     


On each node I am going to use about 10 GB.  (My ram is 16GB). 
Can you help me adjust configuration for this aim ? 
It is very important for me. 
Aim: Extremely fast sql quries.


2016-05-23 18:13 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Hi,

Generally speaking, settings setSqlOnheapRowCacheSize to larger value increases
SQL performance in OFFHEAP_TIERED mode, but also means more job for GC,
so it should be used with care.

The value should be set to the size of your application's working(frequently accessed) data set.

2016-05-23 13:07 GMT+03:00 vkulichenko <[hidden email]>:
Are you using offheap? What is your data size?

Generally, I would recommend to use on-heap with SQL queries if this
possible (unless you have a very big data sets and want to avoid having
large heap sizes). If you still have to use offheap, you can try playing
with this parameter and see what performance you get with different values.
The optimal value depends on a particular application.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/off-heap-indexes-setSqlOnheapRowCacheSize-how-does-it-improve-efficiency-tp5070p5092.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.



--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov

Alexei Scherbakov Alexei Scherbakov
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

First, you have near cache configured and it consumes memory.
Second, you defined copyOnRead=true so every cache get returns a copy of the value, increasing memory usage.
Try to remove NearCacheConfiguration bean, set copyOnRead=false and see it it helps with the client memory problem.

2016-05-24 17:56 GMT+03:00 Tomek W <[hidden email]>:
firstly,
look at fragment of my config clieny file that  bother me:
    <bean id="nearCacheBean" class="org.apache.ignite.configuration.NearCacheConfiguration">
    </bean>

    <bean class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="clientMode" value="true"/>

               
          <property name="cacheConfiguration">
            <list>
                <bean class="org.apache.ignite.configuration.CacheConfiguration">
                    <property name="name" value="my_table_cache"/>
                    <property name="cacheMode" value="PARTITIONED"/>
                    <property name="atomicityMode" value="TRANSACTIONAL"/>
                    <property name="backups" value="0"/>
                    <property name="readFromBackup" value="true"/>
                    <property name="copyOnRead" value="true"/>


2016-05-24 15:22 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Have you configured near cache on the client?
Do you buffer data somewere ?
Share the details of the loading process.


2016-05-24 16:07 GMT+03:00 Tomek W <[hidden email]>:
Thanks for you answer.


I didn't measure it. Simply, I was getting GC overflow error, so I succesively increased it. 
How to measure it ? Why client requires so much memory ?

2016-05-24 14:28 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Just use this:

-server -Xms10G -Xmx10G -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+UseTLAB -XX:NewSize=128m -XX:MaxNewSize=128m -XX:MaxTenuringThreshold=0 -XX:SurvivorRatio=1024 -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+DisableExplicitGC

How do you measure client memory usage ?

2016-05-24 15:04 GMT+03:00 Tomek W <[hidden email]>:
Sorry, 
I made a mistake - I wanted to say that I am going to use ON_HEAP.
Can you suggest my more details about tuning ? 
I have client (which run hot loading data from postgresql) and server node (keep cache - data).  
Now client also requires ~4GB data. Why ?   After all, it doesn't keep data, only run hot loading. 





2016-05-24 13:44 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Try to start with some larger number, if default value is too low for you. 
On example, set it to 50000, and see if the performance is OK.
If not, increase to 100000 etc.
I can't help you further without knowing your data access patterns.

BTW, for 10G heap it is probably better to use ONHEAP_TIERED, as Val suggested.
Don't forget to tune GC as described here:



2016-05-23 22:05 GMT+03:00 Tomek W <[hidden email]>:
Ok, I am going to use OFF_HEAP.     


On each node I am going to use about 10 GB.  (My ram is 16GB). 
Can you help me adjust configuration for this aim ? 
It is very important for me. 
Aim: Extremely fast sql quries.


2016-05-23 18:13 GMT+02:00 Alexei Scherbakov <[hidden email]>:
Hi,

Generally speaking, settings setSqlOnheapRowCacheSize to larger value increases
SQL performance in OFFHEAP_TIERED mode, but also means more job for GC,
so it should be used with care.

The value should be set to the size of your application's working(frequently accessed) data set.

2016-05-23 13:07 GMT+03:00 vkulichenko <[hidden email]>:
Are you using offheap? What is your data size?

Generally, I would recommend to use on-heap with SQL queries if this
possible (unless you have a very big data sets and want to avoid having
large heap sizes). If you still have to use offheap, you can try playing
with this parameter and see what performance you get with different values.
The optimal value depends on a particular application.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/off-heap-indexes-setSqlOnheapRowCacheSize-how-does-it-improve-efficiency-tp5070p5092.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.



--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov
tomk tomk
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

+==============================================================================================+
|     Node ID8(@), IP      | CPUs | Heap Used | CPU Load |   Up Time   |  Size   | Hi/Mi/Rd/Wr |
+==============================================================================================+
| 0F0AAF99(@n0), 127.0.0.1 | 8    | 54.50 %   | 3.23 %   | 00:13:13:49 | 3000000 | Hi: 0       |
|                          |      |           |          |             |         | Mi: 0       |
|                          |      |           |          |             |         | Rd: 0       |
|                          |      |           |          |             |         | Wr: 0       |
+----------------------------------------------------------------------------------------------+


I followed your hints. Actually, client doesn't require such many memory as before - thanks for it.


When it comes to configuration of server, I also followed your hints, results:

Querying is done by JDBC Client.  In ignite and postgresql I have single index on column A.

Ignite: SELECT * FROM table WHERE A > 1345 takes 6s.
Postgres: SELECT * FROM table WHERE A > 1345 takes 4s.

As you  can see, postgres is still bettter than Ignite.  I show you significant fragments of my configuration:
http://pastebin.com/EQC4JPWR

And xml for server file:
http://pastebin.com/enR9h5J4


Try to consider why postgresql is still better, please.

Denis Magda Denis Magda
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

In general Ignite is designed to be used in a distributed environment when gigabytes or terabytes of dataset is spread across many cluster nodes and SQL queries executed across the cluster should be faster since resources of all the machines will be used and as a result a query should be completed quicker. In your scenario you just have only a single cluster node and in fact comparing performance of PostgreSQL and H2 (engine that is used by Ignite SQL) and I can consider that Ignite SQL can work slightly slowly but this in is not Ignite usage scenario.

However if you try to create a cluster of several nodes running on different physical machines, pre-load gigabytes of data there and compare Ignite SQL and PostgresSQL you should see performance improvements on Ignite side.

In any case taking into account the advise above do the following:
- execute “EXPLAIN” query to see that the index is chose properly [1];
- H2 console will allow you to see how fast a query is presently executed on a single node removing several Ignite layers [2];
- check if you have any GC pauses during query execution since it can affect execution time [3]

Also share the objects you use as keys and values.


Denis

On May 25, 2016, at 3:23 AM, Tomek W <[hidden email]> wrote:

+==============================================================================================+
|     Node ID8(@), IP      | CPUs | Heap Used | CPU Load |   Up Time   |  Size   | Hi/Mi/Rd/Wr |
+==============================================================================================+
| 0F0AAF99(@n0), 127.0.0.1 | 8    | 54.50 %   | 3.23 %   | 00:13:13:49 | 3000000 | Hi: 0       |
|                          |      |           |          |             |         | Mi: 0       |
|                          |      |           |          |             |         | Rd: 0       |
|                          |      |           |          |             |         | Wr: 0       |
+----------------------------------------------------------------------------------------------+


I followed your hints. Actually, client doesn't require such many memory as before - thanks for it.


When it comes to configuration of server, I also followed your hints, results:

Querying is done by JDBC Client.  In ignite and postgresql I have single index on column A.

Ignite: SELECT * FROM table WHERE A > 1345 takes 6s.
Postgres: SELECT * FROM table WHERE A > 1345 takes 4s.

As you  can see, postgres is still bettter than Ignite.  I show you significant fragments of my configuration:
http://pastebin.com/EQC4JPWR

And xml for server file:
http://pastebin.com/enR9h5J4


Try to consider why postgresql is still better, please.


Alexei Scherbakov Alexei Scherbakov
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

SELECT * is not really a good test query.
It's result can be affected not only by engine performance.

How many entries are downloaded to the client in both cases?
Do the both queries involve network I/O ?

2016-05-25 7:58 GMT+03:00 Denis Magda <[hidden email]>:
In general Ignite is designed to be used in a distributed environment when gigabytes or terabytes of dataset is spread across many cluster nodes and SQL queries executed across the cluster should be faster since resources of all the machines will be used and as a result a query should be completed quicker. In your scenario you just have only a single cluster node and in fact comparing performance of PostgreSQL and H2 (engine that is used by Ignite SQL) and I can consider that Ignite SQL can work slightly slowly but this in is not Ignite usage scenario.

However if you try to create a cluster of several nodes running on different physical machines, pre-load gigabytes of data there and compare Ignite SQL and PostgresSQL you should see performance improvements on Ignite side.

In any case taking into account the advise above do the following:
- execute “EXPLAIN” query to see that the index is chose properly [1];
- H2 console will allow you to see how fast a query is presently executed on a single node removing several Ignite layers [2];
- check if you have any GC pauses during query execution since it can affect execution time [3]

Also share the objects you use as keys and values.


Denis

On May 25, 2016, at 3:23 AM, Tomek W <[hidden email]> wrote:

+==============================================================================================+
|     Node ID8(@), IP      | CPUs | Heap Used | CPU Load |   Up Time   |  Size   | Hi/Mi/Rd/Wr |
+==============================================================================================+
| 0F0AAF99(@n0), 127.0.0.1 | 8    | 54.50 %   | 3.23 %   | 00:13:13:49 | 3000000 | Hi: 0       |
|                          |      |           |          |             |         | Mi: 0       |
|                          |      |           |          |             |         | Rd: 0       |
|                          |      |           |          |             |         | Wr: 0       |
+----------------------------------------------------------------------------------------------+


I followed your hints. Actually, client doesn't require such many memory as before - thanks for it.


When it comes to configuration of server, I also followed your hints, results:

Querying is done by JDBC Client.  In ignite and postgresql I have single index on column A.

Ignite: SELECT * FROM table WHERE A > 1345 takes 6s.
Postgres: SELECT * FROM table WHERE A > 1345 takes 4s.

As you  can see, postgres is still bettter than Ignite.  I show you significant fragments of my configuration:
http://pastebin.com/EQC4JPWR

And xml for server file:
http://pastebin.com/enR9h5J4


Try to consider why postgresql is still better, please.





--

Best regards,
Alexei Scherbakov
tomk tomk
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

| How many entries are downloaded to the client in both cases?
3000 000

| Do the both queries involve network I/O ?
No, I have only local one server (for testing purpose).


2016-05-25 9:59 GMT+02:00 Alexei Scherbakov <[hidden email]>:
SELECT * is not really a good test query.
It's result can be affected not only by engine performance.

How many entries are downloaded to the client in both cases?
Do the both queries involve network I/O ?

2016-05-25 7:58 GMT+03:00 Denis Magda <[hidden email]>:
In general Ignite is designed to be used in a distributed environment when gigabytes or terabytes of dataset is spread across many cluster nodes and SQL queries executed across the cluster should be faster since resources of all the machines will be used and as a result a query should be completed quicker. In your scenario you just have only a single cluster node and in fact comparing performance of PostgreSQL and H2 (engine that is used by Ignite SQL) and I can consider that Ignite SQL can work slightly slowly but this in is not Ignite usage scenario.

However if you try to create a cluster of several nodes running on different physical machines, pre-load gigabytes of data there and compare Ignite SQL and PostgresSQL you should see performance improvements on Ignite side.

In any case taking into account the advise above do the following:
- execute “EXPLAIN” query to see that the index is chose properly [1];
- H2 console will allow you to see how fast a query is presently executed on a single node removing several Ignite layers [2];
- check if you have any GC pauses during query execution since it can affect execution time [3]

Also share the objects you use as keys and values.


Denis

On May 25, 2016, at 3:23 AM, Tomek W <[hidden email]> wrote:

+==============================================================================================+
|     Node ID8(@), IP      | CPUs | Heap Used | CPU Load |   Up Time   |  Size   | Hi/Mi/Rd/Wr |
+==============================================================================================+
| 0F0AAF99(@n0), 127.0.0.1 | 8    | 54.50 %   | 3.23 %   | 00:13:13:49 | 3000000 | Hi: 0       |
|                          |      |           |          |             |         | Mi: 0       |
|                          |      |           |          |             |         | Rd: 0       |
|                          |      |           |          |             |         | Wr: 0       |
+----------------------------------------------------------------------------------------------+


I followed your hints. Actually, client doesn't require such many memory as before - thanks for it.


When it comes to configuration of server, I also followed your hints, results:

Querying is done by JDBC Client.  In ignite and postgresql I have single index on column A.

Ignite: SELECT * FROM table WHERE A > 1345 takes 6s.
Postgres: SELECT * FROM table WHERE A > 1345 takes 4s.

As you  can see, postgres is still bettter than Ignite.  I show you significant fragments of my configuration:
http://pastebin.com/EQC4JPWR

And xml for server file:
http://pastebin.com/enR9h5J4


Try to consider why postgresql is still better, please.





--

Best regards,
Alexei Scherbakov

Alexei Scherbakov Alexei Scherbakov
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

What's the batch size for postgresql ?
What's the size of one entry ?
Could you provide the test code for both postgres and Ignite (just the query + read with the time estimation) ?

2016-05-25 11:13 GMT+03:00 Tomek W <[hidden email]>:
| How many entries are downloaded to the client in both cases?
3000 000

| Do the both queries involve network I/O ?
No, I have only local one server (for testing purpose).


2016-05-25 9:59 GMT+02:00 Alexei Scherbakov <[hidden email]>:
SELECT * is not really a good test query.
It's result can be affected not only by engine performance.

How many entries are downloaded to the client in both cases?
Do the both queries involve network I/O ?

2016-05-25 7:58 GMT+03:00 Denis Magda <[hidden email]>:
In general Ignite is designed to be used in a distributed environment when gigabytes or terabytes of dataset is spread across many cluster nodes and SQL queries executed across the cluster should be faster since resources of all the machines will be used and as a result a query should be completed quicker. In your scenario you just have only a single cluster node and in fact comparing performance of PostgreSQL and H2 (engine that is used by Ignite SQL) and I can consider that Ignite SQL can work slightly slowly but this in is not Ignite usage scenario.

However if you try to create a cluster of several nodes running on different physical machines, pre-load gigabytes of data there and compare Ignite SQL and PostgresSQL you should see performance improvements on Ignite side.

In any case taking into account the advise above do the following:
- execute “EXPLAIN” query to see that the index is chose properly [1];
- H2 console will allow you to see how fast a query is presently executed on a single node removing several Ignite layers [2];
- check if you have any GC pauses during query execution since it can affect execution time [3]

Also share the objects you use as keys and values.


Denis

On May 25, 2016, at 3:23 AM, Tomek W <[hidden email]> wrote:

+==============================================================================================+
|     Node ID8(@), IP      | CPUs | Heap Used | CPU Load |   Up Time   |  Size   | Hi/Mi/Rd/Wr |
+==============================================================================================+
| 0F0AAF99(@n0), 127.0.0.1 | 8    | 54.50 %   | 3.23 %   | 00:13:13:49 | 3000000 | Hi: 0       |
|                          |      |           |          |             |         | Mi: 0       |
|                          |      |           |          |             |         | Rd: 0       |
|                          |      |           |          |             |         | Wr: 0       |
+----------------------------------------------------------------------------------------------+


I followed your hints. Actually, client doesn't require such many memory as before - thanks for it.


When it comes to configuration of server, I also followed your hints, results:

Querying is done by JDBC Client.  In ignite and postgresql I have single index on column A.

Ignite: SELECT * FROM table WHERE A > 1345 takes 6s.
Postgres: SELECT * FROM table WHERE A > 1345 takes 4s.

As you  can see, postgres is still bettter than Ignite.  I show you significant fragments of my configuration:
http://pastebin.com/EQC4JPWR

And xml for server file:
http://pastebin.com/enR9h5J4


Try to consider why postgresql is still better, please.





--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov
tomk tomk
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

Obraz w treści 1

What code do you mean ? JDBC client ?

Additional test:

Before, I tested with 3000 000 rows. Now, I tested with 5000 000 rows.

SELECT * FROM table WHERE A> 1614289770;

Postgresql: 38s.
Ignite: 10s.

So, it seems that Ignite defeated postgresql.
 What do yout think about this results ? 




2016-05-25 10:25 GMT+02:00 Alexei Scherbakov <[hidden email]>:
What's the batch size for postgresql ?
What's the size of one entry ?
Could you provide the test code for both postgres and Ignite (just the query + read with the time estimation) ?

2016-05-25 11:13 GMT+03:00 Tomek W <[hidden email]>:
| How many entries are downloaded to the client in both cases?
3000 000

| Do the both queries involve network I/O ?
No, I have only local one server (for testing purpose).


2016-05-25 9:59 GMT+02:00 Alexei Scherbakov <[hidden email]>:
SELECT * is not really a good test query.
It's result can be affected not only by engine performance.

How many entries are downloaded to the client in both cases?
Do the both queries involve network I/O ?

2016-05-25 7:58 GMT+03:00 Denis Magda <[hidden email]>:
In general Ignite is designed to be used in a distributed environment when gigabytes or terabytes of dataset is spread across many cluster nodes and SQL queries executed across the cluster should be faster since resources of all the machines will be used and as a result a query should be completed quicker. In your scenario you just have only a single cluster node and in fact comparing performance of PostgreSQL and H2 (engine that is used by Ignite SQL) and I can consider that Ignite SQL can work slightly slowly but this in is not Ignite usage scenario.

However if you try to create a cluster of several nodes running on different physical machines, pre-load gigabytes of data there and compare Ignite SQL and PostgresSQL you should see performance improvements on Ignite side.

In any case taking into account the advise above do the following:
- execute “EXPLAIN” query to see that the index is chose properly [1];
- H2 console will allow you to see how fast a query is presently executed on a single node removing several Ignite layers [2];
- check if you have any GC pauses during query execution since it can affect execution time [3]

Also share the objects you use as keys and values.


Denis

On May 25, 2016, at 3:23 AM, Tomek W <[hidden email]> wrote:

+==============================================================================================+
|     Node ID8(@), IP      | CPUs | Heap Used | CPU Load |   Up Time   |  Size   | Hi/Mi/Rd/Wr |
+==============================================================================================+
| 0F0AAF99(@n0), 127.0.0.1 | 8    | 54.50 %   | 3.23 %   | 00:13:13:49 | 3000000 | Hi: 0       |
|                          |      |           |          |             |         | Mi: 0       |
|                          |      |           |          |             |         | Rd: 0       |
|                          |      |           |          |             |         | Wr: 0       |
+----------------------------------------------------------------------------------------------+


I followed your hints. Actually, client doesn't require such many memory as before - thanks for it.


When it comes to configuration of server, I also followed your hints, results:

Querying is done by JDBC Client.  In ignite and postgresql I have single index on column A.

Ignite: SELECT * FROM table WHERE A > 1345 takes 6s.
Postgres: SELECT * FROM table WHERE A > 1345 takes 4s.

As you  can see, postgres is still bettter than Ignite.  I show you significant fragments of my configuration:
http://pastebin.com/EQC4JPWR

And xml for server file:
http://pastebin.com/enR9h5J4


Try to consider why postgresql is still better, please.





--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov

tomk tomk
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

In reply to this post by Alexei Scherbakov
Obraz w treści 1

What code do you mean ? JDBC client ?

2016-05-25 10:25 GMT+02:00 Alexei Scherbakov <[hidden email]>:
What's the batch size for postgresql ?
What's the size of one entry ?
Could you provide the test code for both postgres and Ignite (just the query + read with the time estimation) ?

2016-05-25 11:13 GMT+03:00 Tomek W <[hidden email]>:
| How many entries are downloaded to the client in both cases?
3000 000

| Do the both queries involve network I/O ?
No, I have only local one server (for testing purpose).


2016-05-25 9:59 GMT+02:00 Alexei Scherbakov <[hidden email]>:
SELECT * is not really a good test query.
It's result can be affected not only by engine performance.

How many entries are downloaded to the client in both cases?
Do the both queries involve network I/O ?

2016-05-25 7:58 GMT+03:00 Denis Magda <[hidden email]>:
In general Ignite is designed to be used in a distributed environment when gigabytes or terabytes of dataset is spread across many cluster nodes and SQL queries executed across the cluster should be faster since resources of all the machines will be used and as a result a query should be completed quicker. In your scenario you just have only a single cluster node and in fact comparing performance of PostgreSQL and H2 (engine that is used by Ignite SQL) and I can consider that Ignite SQL can work slightly slowly but this in is not Ignite usage scenario.

However if you try to create a cluster of several nodes running on different physical machines, pre-load gigabytes of data there and compare Ignite SQL and PostgresSQL you should see performance improvements on Ignite side.

In any case taking into account the advise above do the following:
- execute “EXPLAIN” query to see that the index is chose properly [1];
- H2 console will allow you to see how fast a query is presently executed on a single node removing several Ignite layers [2];
- check if you have any GC pauses during query execution since it can affect execution time [3]

Also share the objects you use as keys and values.


Denis

On May 25, 2016, at 3:23 AM, Tomek W <[hidden email]> wrote:

+==============================================================================================+
|     Node ID8(@), IP      | CPUs | Heap Used | CPU Load |   Up Time   |  Size   | Hi/Mi/Rd/Wr |
+==============================================================================================+
| 0F0AAF99(@n0), 127.0.0.1 | 8    | 54.50 %   | 3.23 %   | 00:13:13:49 | 3000000 | Hi: 0       |
|                          |      |           |          |             |         | Mi: 0       |
|                          |      |           |          |             |         | Rd: 0       |
|                          |      |           |          |             |         | Wr: 0       |
+----------------------------------------------------------------------------------------------+


I followed your hints. Actually, client doesn't require such many memory as before - thanks for it.


When it comes to configuration of server, I also followed your hints, results:

Querying is done by JDBC Client.  In ignite and postgresql I have single index on column A.

Ignite: SELECT * FROM table WHERE A > 1345 takes 6s.
Postgres: SELECT * FROM table WHERE A > 1345 takes 4s.

As you  can see, postgres is still bettter than Ignite.  I show you significant fragments of my configuration:
http://pastebin.com/EQC4JPWR

And xml for server file:
http://pastebin.com/enR9h5J4


Try to consider why postgresql is still better, please.





--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov

Alexei Scherbakov Alexei Scherbakov
Reply | Threaded
Open this post in threaded view
|

Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?

For postgres test I mean initial jdbc query and result set traversal.
For Ignite I mean sql query and iterator traversal.
Also it would be interesting to see result of 
SELECT count(*) from the query above in both cases.

2016-05-25 12:00 GMT+03:00 Tomek W <[hidden email]>:
Obraz w treści 1

What code do you mean ? JDBC client ?

2016-05-25 10:25 GMT+02:00 Alexei Scherbakov <[hidden email]>:
What's the batch size for postgresql ?
What's the size of one entry ?
Could you provide the test code for both postgres and Ignite (just the query + read with the time estimation) ?

2016-05-25 11:13 GMT+03:00 Tomek W <[hidden email]>:
| How many entries are downloaded to the client in both cases?
3000 000

| Do the both queries involve network I/O ?
No, I have only local one server (for testing purpose).


2016-05-25 9:59 GMT+02:00 Alexei Scherbakov <[hidden email]>:
SELECT * is not really a good test query.
It's result can be affected not only by engine performance.

How many entries are downloaded to the client in both cases?
Do the both queries involve network I/O ?

2016-05-25 7:58 GMT+03:00 Denis Magda <[hidden email]>:
In general Ignite is designed to be used in a distributed environment when gigabytes or terabytes of dataset is spread across many cluster nodes and SQL queries executed across the cluster should be faster since resources of all the machines will be used and as a result a query should be completed quicker. In your scenario you just have only a single cluster node and in fact comparing performance of PostgreSQL and H2 (engine that is used by Ignite SQL) and I can consider that Ignite SQL can work slightly slowly but this in is not Ignite usage scenario.

However if you try to create a cluster of several nodes running on different physical machines, pre-load gigabytes of data there and compare Ignite SQL and PostgresSQL you should see performance improvements on Ignite side.

In any case taking into account the advise above do the following:
- execute “EXPLAIN” query to see that the index is chose properly [1];
- H2 console will allow you to see how fast a query is presently executed on a single node removing several Ignite layers [2];
- check if you have any GC pauses during query execution since it can affect execution time [3]

Also share the objects you use as keys and values.


Denis

On May 25, 2016, at 3:23 AM, Tomek W <[hidden email]> wrote:

+==============================================================================================+
|     Node ID8(@), IP      | CPUs | Heap Used | CPU Load |   Up Time   |  Size   | Hi/Mi/Rd/Wr |
+==============================================================================================+
| 0F0AAF99(@n0), 127.0.0.1 | 8    | 54.50 %   | 3.23 %   | 00:13:13:49 | 3000000 | Hi: 0       |
|                          |      |           |          |             |         | Mi: 0       |
|                          |      |           |          |             |         | Rd: 0       |
|                          |      |           |          |             |         | Wr: 0       |
+----------------------------------------------------------------------------------------------+


I followed your hints. Actually, client doesn't require such many memory as before - thanks for it.


When it comes to configuration of server, I also followed your hints, results:

Querying is done by JDBC Client.  In ignite and postgresql I have single index on column A.

Ignite: SELECT * FROM table WHERE A > 1345 takes 6s.
Postgres: SELECT * FROM table WHERE A > 1345 takes 4s.

As you  can see, postgres is still bettter than Ignite.  I show you significant fragments of my configuration:
http://pastebin.com/EQC4JPWR

And xml for server file:
http://pastebin.com/enR9h5J4


Try to consider why postgresql is still better, please.





--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov




--

Best regards,
Alexei Scherbakov
12