Unable to free disk space after keys are removed in ignite 2.7.0

[albar]

Hi, I have a simple local installation of ignite 2.7.0: one server and one spring boot client application, all running on my laptop.

I’ve configured Persistence on the server node

 

<property name="dataStorageConfiguration">

   <bean class="org.apache.ignite.configuration.DataStorageConfiguration">

       <property name="defaultDataRegionConfiguration">

            <bean class="org.apache.ignite.configuration.DataRegionConfiguration">

                <property name="initialSize" value="#{100L * 1024 * 1024}"/>

                <property name="maxSize" value="#{1L * 1024 * 1024 * 1024}"/>

                <property name="persistenceEnabled" value="true"/>

                <property name="metricsEnabled" value="true"/>

            </bean>

        </property>

    </bean>

</property>

 

My Spring app creates a cache via ignite.getOrCreateCache(CACHE_NAME), writes 1.000.000 entries via a DataStreamer, then reads and removes those keys via a cache.get(key) and a cache.remove(key).

With the ignitevisor command I’ve verified that the cache fills up during the execution and at the end, after the removals, is empty.

 

I’ve a problem with the Persistence: the filesystem under work\db\node-<uid>\cache-<cachename> grows during the execution, but the space is never released. After the test, when the cache is empty, I still find more or less a thousand of files named “part-xxx.bin”. The only way I’ve found to free space is destroy() the cache that it is not what I’d like to do.

 

How the system is supposed to free disk space? Am I missing something? Any suggestion or pointer to documentation is highly appreciated.

 

Alberto

 

Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione, Grazie.

This e-mail and any attachments is confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorised. If you are not the intended recipient, please delete this message and any attachments and advise the sender by return e-mail, Thanks.

Rispetta l'ambiente. Non stampare questa mail se non è necessario.

[ilya.kasnacheev]

Hello!

> How the system is supposed to free disk space?

It's not supposed to!

Ignite does not make effort to return disk space (or off-heap memory) to system since it is not usually needed.

Regards,

пн, 27 мая 2019 г. в 16:38, Baravaglio Alberto <[hidden email]>:

Hi, I have a simple local installation of ignite 2.7.0: one server and one spring boot client application, all running on my laptop.

I’ve configured Persistence on the server node

 

<property name="dataStorageConfiguration">

   <bean class="org.apache.ignite.configuration.DataStorageConfiguration">

       <property name="defaultDataRegionConfiguration">

            <bean class="org.apache.ignite.configuration.DataRegionConfiguration">

                <property name="initialSize" value="#{100L * 1024 * 1024}"/>

                <property name="maxSize" value="#{1L * 1024 * 1024 * 1024}"/>

                <property name="persistenceEnabled" value="true"/>

                <property name="metricsEnabled" value="true"/>

            </bean>

        </property>

    </bean>

</property>

 

My Spring app creates a cache via ignite.getOrCreateCache(CACHE_NAME), writes 1.000.000 entries via a DataStreamer, then reads and removes those keys via a cache.get(key) and a cache.remove(key).

With the ignitevisor command I’ve verified that the cache fills up during the execution and at the end, after the removals, is empty.

 

I’ve a problem with the Persistence: the filesystem under work\db\node-<uid>\cache-<cachename> grows during the execution, but the space is never released. After the test, when the cache is empty, I still find more or less a thousand of files named “part-xxx.bin”. The only way I’ve found to free space is destroy() the cache that it is not what I’d like to do.

 

How the system is supposed to free disk space? Am I missing something? Any suggestion or pointer to documentation is highly appreciated.

 

Alberto

 

Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione, Grazie.

This e-mail and any attachments is confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorised. If you are not the intended recipient, please delete this message and any attachments and advise the sender by return e-mail, Thanks.

Rispetta l'ambiente. Non stampare questa mail se non è necessario.

[dmagda]

Ilya, could you please share more details on why it works this way. It will be useful to know.

On Mon, May 27, 2019 at 6:53 AM Ilya Kasnacheev <[hidden email]> wrote:

Hello!

> How the system is supposed to free disk space?

It's not supposed to!

Ignite does not make effort to return disk space (or off-heap memory) to system since it is not usually needed.

Regards,

пн, 27 мая 2019 г. в 16:38, Baravaglio Alberto <[hidden email]>:

Hi, I have a simple local installation of ignite 2.7.0: one server and one spring boot client application, all running on my laptop.

I’ve configured Persistence on the server node

 

<property name="dataStorageConfiguration">

   <bean class="org.apache.ignite.configuration.DataStorageConfiguration">

       <property name="defaultDataRegionConfiguration">

            <bean class="org.apache.ignite.configuration.DataRegionConfiguration">

                <property name="initialSize" value="#{100L * 1024 * 1024}"/>

                <property name="maxSize" value="#{1L * 1024 * 1024 * 1024}"/>

                <property name="persistenceEnabled" value="true"/>

                <property name="metricsEnabled" value="true"/>

            </bean>

        </property>

    </bean>

</property>

 

My Spring app creates a cache via ignite.getOrCreateCache(CACHE_NAME), writes 1.000.000 entries via a DataStreamer, then reads and removes those keys via a cache.get(key) and a cache.remove(key).

With the ignitevisor command I’ve verified that the cache fills up during the execution and at the end, after the removals, is empty.

 

I’ve a problem with the Persistence: the filesystem under work\db\node-<uid>\cache-<cachename> grows during the execution, but the space is never released. After the test, when the cache is empty, I still find more or less a thousand of files named “part-xxx.bin”. The only way I’ve found to free space is destroy() the cache that it is not what I’d like to do.

 

How the system is supposed to free disk space? Am I missing something? Any suggestion or pointer to documentation is highly appreciated.

 

Alberto

 

Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione, Grazie.

This e-mail and any attachments is confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorised. If you are not the intended recipient, please delete this message and any attachments and advise the sender by return e-mail, Thanks.

Rispetta l'ambiente. Non stampare questa mail se non è necessario.

[ilya.kasnacheev]

Hello!

When you delete data from caches, Ignite will likely not free Durable Memory pages, and even if it does, it will likely not "un-commit" memory pages and return them to OS.

Then, WAL will usually only grow and db/ will grow too, they will not shrink from the deletion of data from caches. Dropping a cache will free space.

This is tolerable because modern server instances are usually single-purpose. You expect an instance's storage to be used for Ignite data only and the amount of data is expected to be stable ot grow monotonously.

It is expected that you will destroy VM or delete Ignite's dirs when changing usage pattern, rather than just starting new apps alongside and expecting Ignite to downsize while keeping it running.

Regards,

пн, 10 июн. 2019 г. в 20:30, Denis Magda <[hidden email]>:

Ilya, could you please share more details on why it works this way. It will be useful to know.

On Mon, May 27, 2019 at 6:53 AM Ilya Kasnacheev <[hidden email]> wrote:

Hello!

> How the system is supposed to free disk space?

It's not supposed to!

Ignite does not make effort to return disk space (or off-heap memory) to system since it is not usually needed.

Regards,

пн, 27 мая 2019 г. в 16:38, Baravaglio Alberto <[hidden email]>:

Hi, I have a simple local installation of ignite 2.7.0: one server and one spring boot client application, all running on my laptop.

I’ve configured Persistence on the server node

 

<property name="dataStorageConfiguration">

   <bean class="org.apache.ignite.configuration.DataStorageConfiguration">

       <property name="defaultDataRegionConfiguration">

            <bean class="org.apache.ignite.configuration.DataRegionConfiguration">

                <property name="initialSize" value="#{100L * 1024 * 1024}"/>

                <property name="maxSize" value="#{1L * 1024 * 1024 * 1024}"/>

                <property name="persistenceEnabled" value="true"/>

                <property name="metricsEnabled" value="true"/>

            </bean>

        </property>

    </bean>

</property>

 

My Spring app creates a cache via ignite.getOrCreateCache(CACHE_NAME), writes 1.000.000 entries via a DataStreamer, then reads and removes those keys via a cache.get(key) and a cache.remove(key).

With the ignitevisor command I’ve verified that the cache fills up during the execution and at the end, after the removals, is empty.

 

I’ve a problem with the Persistence: the filesystem under work\db\node-<uid>\cache-<cachename> grows during the execution, but the space is never released. After the test, when the cache is empty, I still find more or less a thousand of files named “part-xxx.bin”. The only way I’ve found to free space is destroy() the cache that it is not what I’d like to do.

 

How the system is supposed to free disk space? Am I missing something? Any suggestion or pointer to documentation is highly appreciated.

 

Alberto

 

Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione, Grazie.

This e-mail and any attachments is confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorised. If you are not the intended recipient, please delete this message and any attachments and advise the sender by return e-mail, Thanks.

Rispetta l'ambiente. Non stampare questa mail se non è necessario.

[VeenaMithare]

Hi Ilya,

I have a question related to this :
>>Then, WAL will usually only grow and db/ will grow too, they will not
shrink from the deletion of data from caches. Dropping a cache will free
space.

>>This is tolerable because modern server instances are usually
single-purpose. You expect an instance's storage to be used for Ignite data
only and the amount of data is expected to be stable not grow monotonously.

We store audit data for our cache modifications. And audit data will grow
monotonously over a period of time. What is the best way of housekeeping
audit records ?

regards,
Veena.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

[ilya.kasnacheev]

Hello!

You may delete old records, then this disk space will be reused when adding new records.

Regards,

чт, 10 дек. 2020 г. в 19:08, VeenaMithare <[hidden email]>:

Hi Ilya,

I have a question related to this :
>>Then, WAL will usually only grow and db/ will grow too, they will not
shrink from the deletion of data from caches. Dropping a cache will free
space.

>>This is tolerable because modern server instances are usually
single-purpose. You expect an instance's storage to be used for Ignite data
only and the amount of data is expected to be stable not grow monotonously.

We store audit data for our cache modifications. And audit data will grow
monotonously over a period of time. What is the best way of housekeeping
audit records ?

regards,
Veena.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

[VeenaMithare]

Thank you .. !



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/