Still confused... I already looked at off heap docs this is why I asked the question. Lets forget about off heap for 2 seconds... I currently start my application (ignite embedded) with -Xmx100g. This works nice and fine. But gc pauses kick in around 60g of heap usage. So once I enable tiered off heap settings in my XML configuration do I still need to start my app with -Xmx100g?
John, you can start your application with minimal heap settings for your application, as if Ignite was not even there. For example, if your application requires 5GB of heap to function, you can start it with -Xmx5g setting.
Make sure to configure off-heap memory mode and, if needed, eviction policy, as specified in Off-Heap Memory section.
Sorry for sounding silly! But something is not sticking here with me I did the 2 following tests. And as I said I have read and fully and understand the offheap memory docs :) On both tests I have the grid setup as offheap_tiered set to 100g So...
If I start with Xmx5g then I can only fit about 10 million records before the application stops responding. If I start with Xmx100g then I can fit 90 million records before the application stops responding.
So if I need 100g of offheap it seems I need Xmx100g to be setup. They seem to go hand in hand.
It does not sound like the cache is using off-heap at all in your scenario. Can you please provide your configuration file? Also, please make sure that you are populating exactly the same cache you are configuring.
John, we tested similar scenarios internally and were able to fit larger number of entries in a smaller 2Gb heap. Can you repeat your test with Xmx5G, take a heap dump and check what objects fill the heap?
I have setup a test of -Xmx1g and offheap of 81GB. Looking at heap stats with yourkit. It will take a while to reproduce the error.
So far yourkit show PS OLD GEN growing and the top 3 objects in terms of object count are...
io.netty...ThreadCache$$Entry at 1,000,000 objects (Thats my HTTP server) constant doesn't grow.
org.apache.ignite.internal.util.snaptree.SnapTreeMap$Node at 650,000 objects and seems to keep groing
org.apache.ignite.internal.util.GridCircularbuffer$Item at 420,000 objects seems constant not growing.