Off-Heap Memory - simpler explanation than in docs

classic Classic list List threaded Threaded
6 messages Options
tomk tomk
Reply | Threaded
Open this post in threaded view
|

Off-Heap Memory - simpler explanation than in docs

Hello,  
Could someone explain this in simpler way ? Maybe some intuition ?
Pavel Tupitsyn Pavel Tupitsyn
Reply | Threaded
Open this post in threaded view
|

Re: Off-Heap Memory - simpler explanation than in docs

Hi, what exactly is not clear?

When you create objects in Java, they end up on JVM heap, in managed memory. These objects are subject to automatic garbage collection.
Ignite Off-Heap Memory feature stores cache data in unmanaged memory (manual allocation/deallocation, like you do in C and other unmanaged languages).
This reduces GC load (better performance, less GC pauses).

Let me know if this helps.

Pavel.

On Thu, May 5, 2016 at 5:08 PM, tomk <[hidden email]> wrote:
Hello,
Could someone explain this in simpler way ? Maybe some intuition ?



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Off-Heap-Memory-simpler-explanation-than-in-docs-tp4790.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

tomk tomk
Reply | Threaded
Open this post in threaded view
|

Re: Off-Heap Memory - simpler explanation than in docs

Thanks for your answer, nevertheless I still don't understand.  
I know that nodes in ignite are launched in jvm.  
It seems to me that off heap make it possible to cache large dataset (such large as RAM).  Why it is not possible with on heap ?

Why off heap is faster than two others layers ?
Denis Magda Denis Magda
Reply | Threaded
Open this post in threaded view
|

Re: Off-Heap Memory - simpler explanation than in docs

 Why it is not possible with on heap ? 
Well-known garbage collectors that are available in Oracle/Open/IBM JDKs works not to nice with heaps of size > 20 GB on high-performant applications like the ones that are built on top of Ignite. The reason is that stop-the-world pauses can become lengthy measuring in tens or hundreds of seconds. Go though this discussion to get more details [1].

The only known garbage collector that can keep up with the heaps of hundreds of gygabates and that avoids stop-the-world pauses is Zing from Azul. [2]

Since you need to pay to Azul if you want to use its JVM the only possible way is to fall back to off-heap space in Ignite that will let to avoid lengthy GC pauses.

Why off heap is faster than two others layers ?

It depends. Usually off-heap based solutions show comparable performance if properly tuned. However there can be minor performance degradation in high performant degradation since objects have to be moved from off-heap to on-heap when you need them (cache.get/results sets of SQL query/etc.). But as a benefit you won’t have long gc pauses.


Denis

On May 11, 2016, at 7:59 PM, tomk <[hidden email]> wrote:

Thanks for your answer, nevertheless I still don't understand.  
I know that nodes in ignite are launched in jvm.  
It seems to me that off heap make it possible to cache large dataset (such
large as RAM).  Why it is not possible with on heap ?

Why off heap is faster than two others layers ?



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Off-Heap-Memory-simpler-explanation-than-in-docs-tp4790p4877.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

tomk tomk
Reply | Threaded
Open this post in threaded view
|

Re: Off-Heap Memory - simpler explanation than in docs

So,
If each my node(machine)  has less  than 20 (in real 16) GB then I should use on-heap ?
Denis Magda Denis Magda
Reply | Threaded
Open this post in threaded view
|

Re: Off-Heap Memory - simpler explanation than in docs

You can give it a try and switch to OFFHEAP if it won’t work well for your use case.

However don’t forget to tune garbage collector appropriately. You can refer to this page [1] as an example.


Denis

On May 21, 2016, at 6:09 PM, tomk <[hidden email]> wrote:

So,
If each my node(machine)  has less  than 20 (in real 16) GB then I should
use on-heap ?



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Off-Heap-Memory-simpler-explanation-than-in-docs-tp4790p5069.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.