I'm trying to understand the memory footprint for ON_HEAP caches in Ignite with the goal of sizing server nodes properly.
I have set up a simple cache test with a single server node and 2,000,000 entries of type <Long, PortfolioAsset> and collected a Heap Dump. Each instance of PortfolioAsset is 32 bytes, and Long is 24 bytes.
When looking at the Heap Histogram and seeing the amount of memory overhead I would like to know if there if there is a way to reduce/eliminate it. In picture shown below I can account for the rows in green: my keys, my values, the serialized version of the values (BinaryObjectImpl), and some overhead for the cache entries. This part makes total sense to me. However, I can't quite make sense of the 3 entries in red.
How can I tweak my cache/ignite configuration parameters to remove this overhead? The data and entries only take 260 MB while the other stuff takes 380 MB. Here's my config:
I would suggest you to follow the following algorithm when calculating memory usage by Ignite.
CALCULATING THE MEMORY USAGE
1. Calculate primary data size: multiply the size of one entry in bytes by the total number of entries. Note, that Ignite will typically add around 200 bytes overhead to each entry.
2. If you have backups, multiply by their number
3. Indexes also require memory. Basic use cases will add a 30% increase
4. Add around 20MB per cache. This value can be reduced if to decrease partitions count and/or IgniteSystemProperties.IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE
5. Add around 200-300MB per node for internal memory and reasonable amount of memory for JVM and GC to operate efficiently