Question about Ignite persistence disk space used after clearing cache

classic Classic list List threaded Threaded
3 messages Options
Scott Scott
Reply | Threaded
Open this post in threaded view
|

Question about Ignite persistence disk space used after clearing cache

I have a question about how the off-heap usage is reported when Ignite persistence is configured.  I have a single node set up.  I stored about 1GB of items in the cache, then cleared the cache (remotely, using the Java thin client:  ClientCache.clear()).

I then verified that the items were no longer in the cache.

However, when I look at the Ignite log, I do not see that the disk space was freed:

[2020-09-25T11:17:36,299][INFO ][grid-timeout-worker-#23][IgniteKernal]
Metrics for local node (to disable set 'metricsLogFrequency' to 0)
    ^-- Node [id=db4ed295, uptime=00:34:00.176]
    ^-- H/N/C [hosts=1, nodes=1, CPUs=8]
    ^-- CPU [cur=0.2%, avg=0.3%, GC=0%]
    ^-- PageMemory [pages=250315]
    ^-- Heap [used=180MB, free=94.85%, comm=438MB]
    ^-- Off-heap [used=989MB, free=88.35%, comm=8392MB]
    ^--   sysMemPlc region [used=0MB, free=99.99%, comm=100MB]
    ^--   default_region region [used=989MB, free=87.92%, comm=8192MB]
    ^--   metastoreMemPlc region [used=0MB, free=99.94%, comm=0MB]
    ^--   TxLog region [used=0MB, free=100%, comm=100MB]
    ^-- Ignite persistence [used=998MB]
    ^--   sysMemPlc region [used=0MB]
    ^--   default_region region [used=998MB]
    ^--   metastoreMemPlc region [used=0MB]
    ^--   TxLog region [used=0MB]
    ^-- Outbound messages queue [size=0]
    ^-- Public thread pool [active=0, idle=0, qSize=0]
    ^-- System thread pool [active=0, idle=6, qSize=0] 

"Ignite persistence [used=998MB]" seems to indicate that 1GB of data is still in the cache.  Is this simply a report of the disk space *allocated*, or is actual disk space in use?  Is there a way to get both measurements?

thanks,

-- Scott 
dmagda dmagda
Reply | Threaded
Open this post in threaded view
|

Re: Question about Ignite persistence disk space used after clearing cache

Hi Scott,

The disk space is not compacted even after you clear the entire cache. The compaction feature will be introduced to Ignite soon. So, the metric shows the allocated size. This doc section suggests an approach for the actual size calculation:



-
Denis


On Fri, Sep 25, 2020 at 1:52 PM Scott Prater <[hidden email]> wrote:
I have a question about how the off-heap usage is reported when Ignite persistence is configured.  I have a single node set up.  I stored about 1GB of items in the cache, then cleared the cache (remotely, using the Java thin client:  ClientCache.clear()).

I then verified that the items were no longer in the cache.

However, when I look at the Ignite log, I do not see that the disk space was freed:

[2020-09-25T11:17:36,299][INFO ][grid-timeout-worker-#23][IgniteKernal]
Metrics for local node (to disable set 'metricsLogFrequency' to 0)
    ^-- Node [id=db4ed295, uptime=00:34:00.176]
    ^-- H/N/C [hosts=1, nodes=1, CPUs=8]
    ^-- CPU [cur=0.2%, avg=0.3%, GC=0%]
    ^-- PageMemory [pages=250315]
    ^-- Heap [used=180MB, free=94.85%, comm=438MB]
    ^-- Off-heap [used=989MB, free=88.35%, comm=8392MB]
    ^--   sysMemPlc region [used=0MB, free=99.99%, comm=100MB]
    ^--   default_region region [used=989MB, free=87.92%, comm=8192MB]
    ^--   metastoreMemPlc region [used=0MB, free=99.94%, comm=0MB]
    ^--   TxLog region [used=0MB, free=100%, comm=100MB]
    ^-- Ignite persistence [used=998MB]
    ^--   sysMemPlc region [used=0MB]
    ^--   default_region region [used=998MB]
    ^--   metastoreMemPlc region [used=0MB]
    ^--   TxLog region [used=0MB]
    ^-- Outbound messages queue [size=0]
    ^-- Public thread pool [active=0, idle=0, qSize=0]
    ^-- System thread pool [active=0, idle=6, qSize=0] 

"Ignite persistence [used=998MB]" seems to indicate that 1GB of data is still in the cache.  Is this simply a report of the disk space *allocated*, or is actual disk space in use?  Is there a way to get both measurements?

thanks,

-- Scott 
Scott Scott
Reply | Threaded
Open this post in threaded view
|

Re: Question about Ignite persistence disk space used after clearing cache

Thanks, Denis.  I'll take a look at that documentation.

On Fri, Sep 25, 2020 at 6:30 PM Denis Magda <[hidden email]> wrote:
Hi Scott,

The disk space is not compacted even after you clear the entire cache. The compaction feature will be introduced to Ignite soon. So, the metric shows the allocated size. This doc section suggests an approach for the actual size calculation:



-
Denis


On Fri, Sep 25, 2020 at 1:52 PM Scott Prater <[hidden email]> wrote:
I have a question about how the off-heap usage is reported when Ignite persistence is configured.  I have a single node set up.  I stored about 1GB of items in the cache, then cleared the cache (remotely, using the Java thin client:  ClientCache.clear()).

I then verified that the items were no longer in the cache.

However, when I look at the Ignite log, I do not see that the disk space was freed:

[2020-09-25T11:17:36,299][INFO ][grid-timeout-worker-#23][IgniteKernal]
Metrics for local node (to disable set 'metricsLogFrequency' to 0)
    ^-- Node [id=db4ed295, uptime=00:34:00.176]
    ^-- H/N/C [hosts=1, nodes=1, CPUs=8]
    ^-- CPU [cur=0.2%, avg=0.3%, GC=0%]
    ^-- PageMemory [pages=250315]
    ^-- Heap [used=180MB, free=94.85%, comm=438MB]
    ^-- Off-heap [used=989MB, free=88.35%, comm=8392MB]
    ^--   sysMemPlc region [used=0MB, free=99.99%, comm=100MB]
    ^--   default_region region [used=989MB, free=87.92%, comm=8192MB]
    ^--   metastoreMemPlc region [used=0MB, free=99.94%, comm=0MB]
    ^--   TxLog region [used=0MB, free=100%, comm=100MB]
    ^-- Ignite persistence [used=998MB]
    ^--   sysMemPlc region [used=0MB]
    ^--   default_region region [used=998MB]
    ^--   metastoreMemPlc region [used=0MB]
    ^--   TxLog region [used=0MB]
    ^-- Outbound messages queue [size=0]
    ^-- Public thread pool [active=0, idle=0, qSize=0]
    ^-- System thread pool [active=0, idle=6, qSize=0] 

"Ignite persistence [used=998MB]" seems to indicate that 1GB of data is still in the cache.  Is this simply a report of the disk space *allocated*, or is actual disk space in use?  Is there a way to get both measurements?

thanks,

-- Scott