OOM in Heap though offHeap is available/free

classic Classic list List threaded Threaded
4 messages Options
Amit Pundir Amit Pundir
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

OOM in Heap though offHeap is available/free

Hi,

I am using onHeap along with the default offHeap memory in my Ignite 2.0 server nodes. I am getting out of memory error even though there is enough memory available offHeap.

My understanding (based on my earlier posts) is that onHeap is just a cache for offHeap. If that's how it works then the heap space should never fill up if there is space available offHeap.

Could you please help me understand how these 2 memories work with each other. The exception I have got is pasted below.


Thanks


[19:30:14,582][SEVERE][grid-nio-worker-tcp-comm-2-#23%NRO%][GridDirectParser] Failed to read message [msg=GridIoMessage [plc=0, topic=null, topicOrd=-1, ordered=false, timeout=0, skipOnTimeout=false, msg=null], buf=java.nio.DirectByteBuffer[pos=176 lim=20272 cap=32768], reader=DirectMessageReader [state=DirectMessageState [pos=0, stack=[StateItem [stream=DirectByteBufferStreamImplV2 [buf=java.nio.DirectByteBuffer[pos=176 lim=20272 cap=32768], baseOff=139667108316592, arrOff=-1, tmpArrOff=0, tmpArrBytes=0, msgTypeDone=true, msg=GridDhtTxPrepareRequest [nearNodeId=null, futId=null, miniId=0, topVer=null, invalidateNearEntries=null, nearWrites=null, owned=null, nearXidVer=null, subjId=null, taskNameHash=0, preloadKeys=null, super=GridDistributedTxPrepareRequest [threadId=164, concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, writeVer=GridCacheVersion [topVer=113311658, order=1501950546676, nodeOrder=6], timeout=56261, reads=null, writes=null, dhtVers=null, txSize=0, plc=2, txState=null, flags=onePhaselast, super=GridDistributedBaseMessage [ver=GridCacheVersion [topVer=113311658, order=1501950546597, nodeOrder=6], committedVers=null, rolledbackVers=null, cnt=0, super=GridCacheMessage [msgId=1294187, depInfo=null, err=null, skipPrepare=false, cacheId=0, cacheId=0]]]], mapIt=null, it=null, arrPos=-1, keyDone=false, readSize=-1, readItems=0, prim=0, primShift=0, uuidState=0, uuidMost=0, uuidLeast=0, uuidLocId=0, lastFinished=true], state=0], StateItem [stream=DirectByteBufferStreamImplV2 [buf=java.nio.DirectByteBuffer[pos=176 lim=20272 cap=32768], baseOff=139667108316592, arrOff=-1, tmpArrOff=0, tmpArrBytes=0, msgTypeDone=true, msg=IgniteTxEntry [key=BinaryObjectImpl [arr= true, ctx=false, start=0], cacheId=1593518529, txKey=null, val=[op=NOOP, val=null], prevVal=[op=NOOP, val=null], oldVal=[op=NOOP, val=null], entryProcessorsCol=null, ttl=-1, conflictExpireTime=-1, conflictVer=null, explicitVer=null, dhtVer=null, filters=null, filtersPassed=false, filtersSet=false, entry=null, prepared=0, locked=false, nodeId=null, locMapped=false, expiryPlc=null, transferExpiryPlc=false, flags=0, partUpdateCntr=0, serReadVer=null, xidVer=null], mapIt=null, it=null, arrPos=-1, keyDone=false, readSize=1, readItems=0, prim=0, primShift=0, uuidState=0, uuidMost=0, uuidLeast=0, uuidLocId=0, lastFinished=true], state=0], StateItem [stream=DirectByteBufferStreamImplV2 [buf=java.nio.DirectByteBuffer[pos=176 lim=20272 cap=32768], baseOff=139667108316592, arrOff=-1, tmpArrOff=0, tmpArrBytes=0, msgTypeDone=true, msg=[op=UPDATE, val=null], mapIt=null, it=null, arrPos=-1, keyDone=false, readSize=-1, readItems=0, prim=0, primShift=0, uuidState=0, uuidMost=0, uuidLeast=0, uuidLocId=0, lastFinished=true], state=0], StateItem [stream=DirectByteBufferStreamImplV2 [buf=java.nio.DirectByteBuffer[pos=176 lim=20272 cap=32768], baseOff=139667108316592, arrOff=-1, tmpArrOff=0, tmpArrBytes=0, msgTypeDone=true, msg=BinaryObjectImpl [arr= false, ctx=false, start=0], mapIt=null, it=null, arrPos=-1, keyDone=false, readSize=-1, readItems=0, prim=0, primShift=0, uuidState=0, uuidMost=0, uuidLeast=0, uuidLocId=0, lastFinished=true], state=0], StateItem [stream=DirectByteBufferStreamImplV2 [buf=java.nio.DirectByteBuffer[pos=176 lim=20272 cap=32768], baseOff=139667108316592, arrOff=-1, tmpArrOff=0, tmpArrBytes=0, msgTypeDone=false, msg=null, mapIt=null, it=null, arrPos=-1, keyDone=false, readSize=-1, readItems=0, prim=0, primShift=0, uuidState=0, uuidMost=0, uuidLeast=0, uuidLocId=0, lastFinished=true], state=0], null, null, null, null, null]], lastRead=true], ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [selector=sun.nio.ch.EPollSelectorImpl@31cec787, idx=2, bytesRcvd=428406858156, bytesSent=278366, bytesRcvd0=0, bytesSent0=0, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-2, igniteInstanceName=NRO, finished=false, hashCode=426420385, interrupted=false, runner=grid-nio-worker-tcp-comm-2-#23%NRO%]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=176 lim=20272 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=139784, sentCnt=0, reserved=true, lastAck=139776, nodeLeft=false, node=TcpDiscoveryNode [id=02c2d1ae-3676-4c21-a953-60d81a41c47d, addrs=[10.120.249.50, 127.0.0.1], sockAddrs=[/127.0.0.1:47500, /10.120.249.50:47500], discPort=47500, order=6, intOrder=6, lastExchangeTime=1501831543578, loc=false, ver=2.0.0#20170430-sha1:d4eef3c6, isClient=false], connected=true, connectCnt=0, queueLimit=131072, reserveCnt=2, pairedConnections=true], outRecovery=null, super=GridNioSessionImpl [locAddr=/10.120.194.51:47100, rmtAddr=/10.120.249.50:35310, createTime=1502047810795, closeTime=0, bytesSent=28, bytesRcvd=20314, bytesSent0=0, bytesRcvd0=0, sndSchedTime=1502047810795, lastSndTime=1502047811434, lastRcvTime=1502047811434, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=o.a.i.i.util.nio.GridDirectParser@16786b96, directMode=true], GridConnectionBytesVerifyFilter], accepted=true]]]
java.lang.OutOfMemoryError: Java heap space
[19:30:14,581][SEVERE][query-#23216%NRO%][GridCacheIoManager] Failed to process message [senderId=9d1131b7-a590-4e7f-b0ab-29bcbd5716d0, messageType=class o.a.i.i.processors.cache.query.GridCacheQueryRequest]
java.lang.OutOfMemoryError: Java heap space
        at org.apache.ignite.internal.util.io.GridUnsafeDataOutput.array(GridUnsafeDataOutput.java:98)
        at org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.marshal0(OptimizedMarshaller.java:203)
        at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
        at org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:9969)
        at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:179)
        at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
        at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
        at org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:245)
        at org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:82)
        at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
        at org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:9969)
        at org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10033)
        at org.apache.ignite.internal.processors.cache.GridCacheUtils.marshal(GridCacheUtils.java:879)
        at org.apache.ignite.internal.processors.cache.GridCacheUtils.marshal(GridCacheUtils.java:852)
        at org.apache.ignite.internal.processors.cache.GridCacheMessage.marshalCollection(GridCacheMessage.java:492)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponse.prepareMarshal(GridCacheQueryResponse.java:137)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onSend(GridCacheIoManager.java:912)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.sendOrderedMessage(GridCacheIoManager.java:1108)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager.sendQueryResponse(GridCacheDistributedQueryManager.java:324)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager.onPageReady(GridCacheDistributedQueryManager.java:469)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.runQuery(GridCacheQueryManager.java:1349)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager.processQueryRequest(GridCacheDistributedQueryManager.java:231)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$2.apply(GridCacheDistributedQueryManager.java:109)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$2.apply(GridCacheDistributedQueryManager.java:107)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:863)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:386)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:308)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:100)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:253)
        at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1257)
        at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:885)
        at org.apache.ignite.internal.managers.communication.GridIoManager.access$2100(GridIoManager.java:114)
Exception in thread "query-#23216%NRO%" java.lang.OutOfMemoryError: Java heap space
        at org.apache.ignite.internal.util.io.GridUnsafeDataOutput.array(GridUnsafeDataOutput.java:98)
        at org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.marshal0(OptimizedMarshaller.java:203)
        at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
        at org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:9969)
        at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:179)
        at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
        at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
        at org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:245)
        at org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:82)
        at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
        at org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:9969)
        at org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10033)
        at org.apache.ignite.internal.processors.cache.GridCacheUtils.marshal(GridCacheUtils.java:879)
        at org.apache.ignite.internal.processors.cache.GridCacheUtils.marshal(GridCacheUtils.java:852)
        at org.apache.ignite.internal.processors.cache.GridCacheMessage.marshalCollection(GridCacheMessage.java:492)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponse.prepareMarshal(GridCacheQueryResponse.java:137)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onSend(GridCacheIoManager.java:912)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.sendOrderedMessage(GridCacheIoManager.java:1108)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager.sendQueryResponse(GridCacheDistributedQueryManager.java:324)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager.onPageReady(GridCacheDistributedQueryManager.java:469)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.runQuery(GridCacheQueryManager.java:1349)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager.processQueryRequest(GridCacheDistributedQueryManager.java:231)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$2.apply(GridCacheDistributedQueryManager.java:109)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$2.apply(GridCacheDistributedQueryManager.java:107)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:863)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:386)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:308)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:100)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:253)
        at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1257)
        at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:885)
        at org.apache.ignite.internal.managers.communication.GridIoManager.access$2100(GridIoManager.java:114)
afedotov afedotov
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OOM in Heap though offHeap is available/free

Hi Amit,

Your understanding is correct, an on-heap cache is like a frontend for an off-heap cache.
It's useful in cases when a lot of cache reads are expected.
The only thing that should be considered is setting an eviction policy for on-heap cache to
prevent it growing unrestrictedly.

Please find the links below for your reference:

Kind regards,
Alex.

On Wed, Aug 9, 2017 at 5:48 AM, Amit Pundir [via Apache Ignite Users] <[hidden email]> wrote:
Hi,

I am using onHeap along with the default offHeap memory in my Ignite 2.0 server nodes. I am getting out of memory error even though there is enough memory available offHeap.

My understanding (based on my earlier posts) is that onHeap is just a cache for offHeap. If that's how it works then the heap space should never fill up if there is space available offHeap.

Could you please help me understand how these 2 memories work with each other. The exception I have got is pasted below.


Thanks


[19:30:14,582][SEVERE][grid-nio-worker-tcp-comm-2-#23%NRO%][GridDirectParser] Failed to read message [msg=GridIoMessage [plc=0, topic=null, topicOrd=-1, ordered=false, timeout=0, skipOnTimeout=false, msg=null], buf=java.nio.DirectByteBuffer[pos=176 lim=20272 cap=32768], reader=DirectMessageReader [state=DirectMessageState [pos=0, stack=[StateItem [stream=DirectByteBufferStreamImplV2 [buf=java.nio.DirectByteBuffer[pos=176 lim=20272 cap=32768], baseOff=139667108316592, arrOff=-1, tmpArrOff=0, tmpArrBytes=0, msgTypeDone=true, msg=GridDhtTxPrepareRequest [nearNodeId=null, futId=null, miniId=0, topVer=null, invalidateNearEntries=null, nearWrites=null, owned=null, nearXidVer=null, subjId=null, taskNameHash=0, preloadKeys=null, super=GridDistributedTxPrepareRequest [threadId=164, concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, writeVer=GridCacheVersion [topVer=113311658, order=1501950546676, nodeOrder=6], timeout=56261, reads=null, writes=null, dhtVers=null, txSize=0, plc=2, txState=null, flags=onePhaselast, super=GridDistributedBaseMessage [ver=GridCacheVersion [topVer=113311658, order=1501950546597, nodeOrder=6], committedVers=null, rolledbackVers=null, cnt=0, super=GridCacheMessage [msgId=1294187, depInfo=null, err=null, skipPrepare=false, cacheId=0, cacheId=0]]]], mapIt=null, it=null, arrPos=-1, keyDone=false, readSize=-1, readItems=0, prim=0, primShift=0, uuidState=0, uuidMost=0, uuidLeast=0, uuidLocId=0, lastFinished=true], state=0], StateItem [stream=DirectByteBufferStreamImplV2 [buf=java.nio.DirectByteBuffer[pos=176 lim=20272 cap=32768], baseOff=139667108316592, arrOff=-1, tmpArrOff=0, tmpArrBytes=0, msgTypeDone=true, msg=IgniteTxEntry [key=BinaryObjectImpl [arr= true, ctx=false, start=0], cacheId=1593518529, txKey=null, val=[op=NOOP, val=null], prevVal=[op=NOOP, val=null], oldVal=[op=NOOP, val=null], entryProcessorsCol=null, ttl=-1, conflictExpireTime=-1, conflictVer=null, explicitVer=null, dhtVer=null, filters=null, filtersPassed=false, filtersSet=false, entry=null, prepared=0, locked=false, nodeId=null, locMapped=false, expiryPlc=null, transferExpiryPlc=false, flags=0, partUpdateCntr=0, serReadVer=null, xidVer=null], mapIt=null, it=null, arrPos=-1, keyDone=false, readSize=1, readItems=0, prim=0, primShift=0, uuidState=0, uuidMost=0, uuidLeast=0, uuidLocId=0, lastFinished=true], state=0], StateItem [stream=DirectByteBufferStreamImplV2 [buf=java.nio.DirectByteBuffer[pos=176 lim=20272 cap=32768], baseOff=139667108316592, arrOff=-1, tmpArrOff=0, tmpArrBytes=0, msgTypeDone=true, msg=[op=UPDATE, val=null], mapIt=null, it=null, arrPos=-1, keyDone=false, readSize=-1, readItems=0, prim=0, primShift=0, uuidState=0, uuidMost=0, uuidLeast=0, uuidLocId=0, lastFinished=true], state=0], StateItem [stream=DirectByteBufferStreamImplV2 [buf=java.nio.DirectByteBuffer[pos=176 lim=20272 cap=32768], baseOff=139667108316592, arrOff=-1, tmpArrOff=0, tmpArrBytes=0, msgTypeDone=true, msg=BinaryObjectImpl [arr= false, ctx=false, start=0], mapIt=null, it=null, arrPos=-1, keyDone=false, readSize=-1, readItems=0, prim=0, primShift=0, uuidState=0, uuidMost=0, uuidLeast=0, uuidLocId=0, lastFinished=true], state=0], StateItem [stream=DirectByteBufferStreamImplV2 [buf=java.nio.DirectByteBuffer[pos=176 lim=20272 cap=32768], baseOff=139667108316592, arrOff=-1, tmpArrOff=0, tmpArrBytes=0, msgTypeDone=false, msg=null, mapIt=null, it=null, arrPos=-1, keyDone=false, readSize=-1, readItems=0, prim=0, primShift=0, uuidState=0, uuidMost=0, uuidLeast=0, uuidLocId=0, lastFinished=true], state=0], null, null, null, null, null]], lastRead=true], ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [selector=sun.nio.ch.EPollSelectorImpl@31cec787, idx=2, bytesRcvd=428406858156, bytesSent=278366, bytesRcvd0=0, bytesSent0=0, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-2, igniteInstanceName=NRO, finished=false, hashCode=426420385, interrupted=false, runner=grid-nio-worker-tcp-comm-2-#23%NRO%]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=176 lim=20272 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=139784, sentCnt=0, reserved=true, lastAck=139776, nodeLeft=false, node=TcpDiscoveryNode [id=02c2d1ae-3676-4c21-a953-60d81a41c47d, addrs=[10.120.249.50, 127.0.0.1], sockAddrs=[/127.0.0.1:47500, /10.120.249.50:47500], discPort=47500, order=6, intOrder=6, lastExchangeTime=1501831543578, loc=false, ver=2.0.0#20170430-sha1:d4eef3c6, isClient=false], connected=true, connectCnt=0, queueLimit=131072, reserveCnt=2, pairedConnections=true], outRecovery=null, super=GridNioSessionImpl [locAddr=/10.120.194.51:47100, rmtAddr=/10.120.249.50:35310, createTime=1502047810795, closeTime=0, bytesSent=28, bytesRcvd=20314, bytesSent0=0, bytesRcvd0=0, sndSchedTime=1502047810795, lastSndTime=1502047811434, lastRcvTime=1502047811434, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=o.a.i.i.util.nio.GridDirectParser@16786b96, directMode=true], GridConnectionBytesVerifyFilter], accepted=true]]]
java.lang.OutOfMemoryError: Java heap space
[19:30:14,581][SEVERE][query-#23216%NRO%][GridCacheIoManager] Failed to process message [senderId=9d1131b7-a590-4e7f-b0ab-29bcbd5716d0, messageType=class o.a.i.i.processors.cache.query.GridCacheQueryRequest]
java.lang.OutOfMemoryError: Java heap space
        at org.apache.ignite.internal.util.io.GridUnsafeDataOutput.array(GridUnsafeDataOutput.java:98)
        at org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.marshal0(OptimizedMarshaller.java:203)
        at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
        at org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:9969)
        at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:179)
        at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
        at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
        at org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:245)
        at org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:82)
        at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
        at org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:9969)
        at org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10033)
        at org.apache.ignite.internal.processors.cache.GridCacheUtils.marshal(GridCacheUtils.java:879)
        at org.apache.ignite.internal.processors.cache.GridCacheUtils.marshal(GridCacheUtils.java:852)
        at org.apache.ignite.internal.processors.cache.GridCacheMessage.marshalCollection(GridCacheMessage.java:492)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponse.prepareMarshal(GridCacheQueryResponse.java:137)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onSend(GridCacheIoManager.java:912)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.sendOrderedMessage(GridCacheIoManager.java:1108)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager.sendQueryResponse(GridCacheDistributedQueryManager.java:324)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager.onPageReady(GridCacheDistributedQueryManager.java:469)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.runQuery(GridCacheQueryManager.java:1349)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager.processQueryRequest(GridCacheDistributedQueryManager.java:231)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$2.apply(GridCacheDistributedQueryManager.java:109)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$2.apply(GridCacheDistributedQueryManager.java:107)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:863)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:386)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:308)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:100)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:253)
        at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1257)
        at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:885)
        at org.apache.ignite.internal.managers.communication.GridIoManager.access$2100(GridIoManager.java:114)
Exception in thread "query-#23216%NRO%" java.lang.OutOfMemoryError: Java heap space
        at org.apache.ignite.internal.util.io.GridUnsafeDataOutput.array(GridUnsafeDataOutput.java:98)
        at org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.marshal0(OptimizedMarshaller.java:203)
        at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
        at org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:9969)
        at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:179)
        at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
        at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
        at org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:245)
        at org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:82)
        at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
        at org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:9969)
        at org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10033)
        at org.apache.ignite.internal.processors.cache.GridCacheUtils.marshal(GridCacheUtils.java:879)
        at org.apache.ignite.internal.processors.cache.GridCacheUtils.marshal(GridCacheUtils.java:852)
        at org.apache.ignite.internal.processors.cache.GridCacheMessage.marshalCollection(GridCacheMessage.java:492)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponse.prepareMarshal(GridCacheQueryResponse.java:137)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onSend(GridCacheIoManager.java:912)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.sendOrderedMessage(GridCacheIoManager.java:1108)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager.sendQueryResponse(GridCacheDistributedQueryManager.java:324)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager.onPageReady(GridCacheDistributedQueryManager.java:469)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.runQuery(GridCacheQueryManager.java:1349)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager.processQueryRequest(GridCacheDistributedQueryManager.java:231)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$2.apply(GridCacheDistributedQueryManager.java:109)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$2.apply(GridCacheDistributedQueryManager.java:107)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:863)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:386)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:308)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:100)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:253)
        at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1257)
        at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:885)
        at org.apache.ignite.internal.managers.communication.GridIoManager.access$2100(GridIoManager.java:114)


If you reply to this email, your message will be added to the discussion below:
http://apache-ignite-users.70518.x6.nabble.com/OOM-in-Heap-though-offHeap-is-available-free-tp16070.html
To start a new topic under Apache Ignite Users, email [hidden email]
To unsubscribe from Apache Ignite Users, click here.
NAML

Amit Pundir Amit Pundir
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OOM in Heap though offHeap is available/free

Thanks for the response.

So does that mean with onHeap enabled, the normal JVM Memory and GC tuning parameters don't work as expected? I am using CMS garbage collector and was hoping that it will take care of cache-keys eviction from the onHeap memory.
vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OOM in Heap though offHeap is available/free

When you enable the on-heap cache, is saves anything that you access. If there is no eviction policy and you constantly update/read new keys, you will very likely run out of heap memory. Configure eviction policy if you want to limit the size of on-heap cache.

-Val
Loading...