When putting data on Off heap Cache Consuming all RAM

classic Classic list List threaded Threaded
2 messages Options
Abhishek M Abhishek M
Reply | Threaded
Open this post in threaded view
|

When putting data on Off heap Cache Consuming all RAM

This post has NOT been accepted by the mailing list yet.
This post was updated on .
Hi ,

I have 25 Millions Records. When i am putting into Off_Heap cache its consuming 14Gb Memory(near about 10Gb Ram + 4Gb Swap). Please let me know how i can reduce the memory size. their is any option so that it will consume less memory? While i am putting same data in Java heap(in HashMap) its taking near about 6GB Ram.

Code:

CacheConfiguration<Long, CreativeLineItemAdslot> cfg = new CacheConfiguration<Long, CreativeLineItemAdslot>();
cfg.setCacheMode(CacheMode.PARTITIONED);
cfg.setName(CACHE_NAME);
cfg.setMemoryMode(CacheMemoryMode.OFFHEAP_TIERED);
cfg.setIndexedTypes(Long.class, CreativeLineItemAdslot.class);
cfg.setSwapEnabled(Boolean.TRUE);
cfg.setOffHeapMaxMemory(offHeapMemoryMaxSize);
cfg.setSqlOnheapRowCacheSize(100);
cfg.setBackups(1);
cache = ignite.getOrCreateCache(cfg);



Also facing Issue while fetching the data. 95% time its responding with in 10Ms. But some time its takes near about 1-2 Min. Please let me know what can be a issue.

I am using below code:

String sql = "lineItemId = ?";
                Iterable<Entry<Long, CreativeLineItemAdslot>> col = cache.query(
                                new SqlQuery<Long, CreativeLineItemAdslot>(CreativeLineItemAdslot.class, sql).setArgs(lineItemId)).getAll();

and lineItemId  is
@QuerySqlField(index = true)
        private long lineItemId;
dsetrakyan dsetrakyan
Reply | Threaded
Open this post in threaded view
|

Re: When putting data on Off heap Cache Consuming all RAM

This post was updated on .
Abhishek M wrote
I have 25 Millions Records. When i am putting into Off_Heap cache its consuming 14Gb Memory(near about 10Gb Ram + 4Gb Swap). Please let me know how i can reduce the memory size. their is any option so that it will consume less memory? While i am putting same data in Java heap(in HashMap) its taking near about 6GB Ram.
You should specify on-heap memory size at the startup time of your JVM. For example, the following JVM setting "-Xmx1g" will allocate 1GB on heap.

Abhishek M wrote
Also facing Issue while fetching the data. 95% time its responding with in 10Ms. But some time its takes near about 1-2 Min. Please let me know what can be a issue.
This sounds like a GC pause caused by large heap space. Once you apply the JVM configuration setting above, it should go away.