Data region LRU offheap algo not working

classic Classic list List threaded Threaded
7 messages Options
rick_tem rick_tem
Reply | Threaded
Open this post in threaded view
|

Data region LRU offheap algo not working

Hi,

I am trying to find out why it appears the RANDOM_LRU algo doesn't seem to
work with the following config.  Log attached, as well...after the log of
the below...

2019-09-09 11:04:03.557 WARN  [sys-stripe-5-#6%TemenosGrid%]
IgniteCacheDatabaseSharedManager - Page-based evictions started. Consider
increasing 'maxSize' on Data Region configuration: 1G_Region

after a few minutes you see memory steadily decrease.  What information in
the log will help me determine how many pages are freed, etc?

Thanks,
Rick

dateRepo1.out
<http://apache-ignite-users.70518.x6.nabble.com/file/t869/dateRepo1.out>  
dataRepo2.out
<http://apache-ignite-users.70518.x6.nabble.com/file/t869/dataRepo2.out>  


                                <property name="dataRegionConfigurations">
                                        <list>
                                               
                                                <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                                                        <property name="name" value="1G_Region" />

                                                       
                                                        <property name="initialSize" value="#{100L * 1024 * 1024}" />

                                                       
                                                        <property name="maxSize" value="#{1000L * 1024 * 1024}" />
                                                        <property name="persistenceEnabled" value="false" />
                                                        <property name="pageEvictionMode" value="RANDOM_LRU" />
                                                        <property
name="evictionThreshold" value="0.7"  />
                                                </bean>
                                               
                                        </list>
                                </property>





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Michaelikus Michaelikus
Reply | Threaded
Open this post in threaded view
|

Re: Data region LRU offheap algo not working

This post was updated on .
what is your ignite version?



-----
Russian telegram Ignite chat at https://t.me/RU_Ignite
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Russian telegram Ignite chat at https://t.me/RU_Ignite
rick_tem rick_tem
Reply | Threaded
Open this post in threaded view
|

Re: Data region LRU offheap algo not working

Hi, it's in the log 2.7.5...Thanks for any help here!
Thanks,
Rick



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Alexandr Shapkin Alexandr Shapkin
Reply | Threaded
Open this post in threaded view
|

RE: Data region LRU offheap algo not working

In reply to this post by rick_tem

Hello!

 

As far as my understanding goes, I see that eviction works ok.

 

>2019-09-09 11:04:03.557 WARN  [sys-stripe-5-#6%TemenosGrid%]

>IgniteCacheDatabaseSharedManager - Page-based evictions started. Consider increasing 'maxSize' on Data Region configuration: 1G_Region

 

This message tells that now eviction process is started and will free pages according to the algorithm.

it means that when Ignite requests for a new page and there are no available ones, then Ignite will apply eviction first and then use those pages for a new values.

So you will not see any significant memory changes during eviction.

I.e. Ignite will show that the data region is full, but still allows you to write new values.

 

There are no any special messages log messages regarding evicted pages. Only the first one, when eviction comes to the play.

I think you may try to look into getEvictionRate() metrics [1] for details.

Also you should be able to see it in Visor GUI.

 

[1] - https://apacheignite.readme.io/docs/memory-metrics#section-data-region-metrics

 

From: [hidden email]
Sent: Tuesday, September 10, 2019 10:12 AM
To: [hidden email]
Subject: Data region LRU offheap algo not working

 

Hi,

 

I am trying to find out why it appears the RANDOM_LRU algo doesn't seem to

work with the following config.  Log attached, as well...after the log of

the below...

 

2019-09-09 11:04:03.557 WARN  [sys-stripe-5-#6%TemenosGrid%]

IgniteCacheDatabaseSharedManager - Page-based evictions started. Consider

increasing 'maxSize' on Data Region configuration: 1G_Region

 

after a few minutes you see memory steadily decrease.  What information in

the log will help me determine how many pages are freed, etc?

 

Thanks,

Rick

 

dateRepo1.out

<http://apache-ignite-users.70518.x6.nabble.com/file/t869/dateRepo1.out> 

dataRepo2.out

<http://apache-ignite-users.70518.x6.nabble.com/file/t869/dataRepo2.out> 

 

 

                                                                <property name="dataRegionConfigurations">

                                                                                <list>

                                                                                               

                                                                                                <bean class="org.apache.ignite.configuration.DataRegionConfiguration">

                                                                                                                <property name="name" value="1G_Region" />

 

                                                                                                               

                                                                                                                <property name="initialSize" value="#{100L * 1024 * 1024}" />

 

                                                                                                               

                                                                                                                <property name="maxSize" value="#{1000L * 1024 * 1024}" />

                                                                                                                <property name="persistenceEnabled" value="false" />

                                                                                                                <property name="pageEvictionMode" value="RANDOM_LRU" />

                                                        <property

name="evictionThreshold" value="0.7"  />

                                                                                                </bean>

                                                                                               

                                                                                </list>

                                                                </property>

 

 

 

 

 

--

Sent from: http://apache-ignite-users.70518.x6.nabble.com/

 

Alex Shapkin
rick_tem rick_tem
Reply | Threaded
Open this post in threaded view
|

RE: Data region LRU offheap algo not working

Thanks for your reply Alexandr.

However, looking at the log, memory continues to be lost.  You continue to
see loss of memory after this message:
2019-09-09 11:04:03.557 WARN  [sys-stripe-5-#6%TemenosGrid%]
IgniteCacheDatabaseSharedManager - Page-based evictions started. Consider
increasing 'maxSize' on Data Region configuration: 1G_Region

Log is:
http://apache-ignite-users.70518.x6.nabble.com/file/t869/dateRepo1.out

While not in this log, this eventually causes an out of memory issue stack
trace that is unrecoverable.  I'll get the log that shows that and turn on
the metrics to get better information.  It's pretty easy for me to
reproduce.

Thanks,
Rick








--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
rick_tem rick_tem
Reply | Threaded
Open this post in threaded view
|

RE: Data region LRU offheap algo not working

Hi,

Please find enclosed the memory crash with the following configuration along
with the data region metrics just before the crash.  This should be easy to
reproduce.  Simply keep injecting data into the cache at a high rate with a
small data region size.

Thanks,
Rick

                                <property name="dataRegionConfigurations">
                                        <list>
                                                <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                                                        <property name="name" value="1G_Region" />
                                                        <property name="initialSize" value="#{100L * 1024 * 1024}" />
                                                        <property name="maxSize" value="#{400L * 1024 * 1024}" />
                                                        <property name="persistenceEnabled" value="false" />
                                                        <property name="pageEvictionMode" value="RANDOM_2_LRU" />
                                                        <property name="evictionThreshold" value="0.6" />
                                                        <property name="metricsEnabled" value="true"/>
                                                </bean>
                                                 
                                        </list>
                                </property>
Ignite-dataregion.png
<http://apache-ignite-users.70518.x6.nabble.com/file/t869/Ignite-dataregion.png>  
Ignite.log
<http://apache-ignite-users.70518.x6.nabble.com/file/t869/Ignite.log>  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
rick_tem rick_tem
Reply | Threaded
Open this post in threaded view
|

RE: Data region LRU offheap algo not working

Hi,

I have created the following JIRA for this issue.

IGNITE-12215

Thanks,
Rick



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/