Enabling swapPath causes invoking shutdown hook

classic Classic list List threaded Threaded
6 messages Options
38797715 38797715
Reply | Threaded
Open this post in threaded view
|

Enabling swapPath causes invoking shutdown hook

Hi community,

When swapPath is configured in DataRegionConfiguration and maxSize is greater than the physical memory (that is, swap space is enabled), if the amount of data exceeds the physical memory, a node failure will occur. The log is as follows:

[08:29:14,212][INFO][Thread-24][G] Invoking shutdown hook...

I think the node process may be killed by the OS. What parameters can be adjusted?


aealexsandrov aealexsandrov
Reply | Threaded
Open this post in threaded view
|

Re: Enabling swapPath causes invoking shutdown hook

Hi,

Can you please clarify your expectations? You expected that JVM process will
be killed instead of gracefully stopping? What you are going to achieve?

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
38797715 38797715
Reply | Threaded
Open this post in threaded view
|

Re: Enabling swapPath causes invoking shutdown hook

Hi,

https://apacheignite.readme.io/docs/swap-space

According to the above document, if the physical memory is small, you can solve this problem by opening the swap space,The specific method is to configure maxSize to a larger value (i.e. larger than the physical memory), and the swapPath property needs to be configured.

But from the test results, the node is terminated.

I think the correct result should be that even if the amount of data exceeds the physical memory, the node should still be able to run normally, but the data is exchanged to the disk.

I want to know what parameters affect the behavior of this configuration? vm.swappiness or others?

在 2020/7/24 下午9:55, aealexsandrov 写道:
Hi,

Can you please clarify your expectations? You expected that JVM process will
be killed instead of gracefully stopping? What you are going to achieve?

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
aealexsandrov aealexsandrov
Reply | Threaded
Open this post in threaded view
|

Re: Enabling swapPath causes invoking shutdown hook

Hi,

Documentation looks not very clean for me. Probably you should prepare your
operating system to use swap space. Can you try to prepare the swap file
first as following:

https://linuxize.com/post/how-to-add-swap-space-on-ubuntu-18-04/

And then choose this file.

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Enabling swapPath causes invoking shutdown hook

In reply to this post by 38797715
Hello!

From the docs:

To avoid this situation with the swapping capabilities, you need to :

  • Set maxSize = bigger_ than_RAM_size, in which case, the OS will take care of the swapping.
  • Enable swapping by setting the DataRegionConfiguration.swapPath property.

I actually think these are either-or. You should either do the first (and configure OS swapping) or the second part.

Having said that, I recommend setting proper Native Persistence instead.

Regards,
-- 
Ilya Kasnacheev


сб, 25 июл. 2020 г. в 04:49, 38797715 <[hidden email]>:

Hi,

https://apacheignite.readme.io/docs/swap-space

According to the above document, if the physical memory is small, you can solve this problem by opening the swap space,The specific method is to configure maxSize to a larger value (i.e. larger than the physical memory), and the swapPath property needs to be configured.

But from the test results, the node is terminated.

I think the correct result should be that even if the amount of data exceeds the physical memory, the node should still be able to run normally, but the data is exchanged to the disk.

I want to know what parameters affect the behavior of this configuration? vm.swappiness or others?

在 2020/7/24 下午9:55, aealexsandrov 写道:
Hi,

Can you please clarify your expectations? You expected that JVM process will
be killed instead of gracefully stopping? What you are going to achieve?

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
38797715 38797715
Reply | Threaded
Open this post in threaded view
|

Re: Enabling swapPath causes invoking shutdown hook

Hi Ilya,

If so, there are two ways to implement ignite's swap space:
1. maxSize > physical memory, which will use the swap mechanism of the OS, can be used vm.swappiness Adjust.
2. Configure the swapPath property, which is implemented by Ignite itself, is independent of the OS and has no optimization parameters.

There's a choice between these two models, right? Then I think there may be many problems in the description of the document. I hope you can check it again:
https://apacheignite.readme.io/docs/swap-space

After our initial testing, the performance of swap space is much better than native persistence, so I think this pattern is valuable in some scenarios.

在 2020/8/4 下午10:16, Ilya Kasnacheev 写道:
Hello!

From the docs:

To avoid this situation with the swapping capabilities, you need to :

  • Set maxSize = bigger_ than_RAM_size, in which case, the OS will take care of the swapping.
  • Enable swapping by setting the DataRegionConfiguration.swapPath property.

I actually think these are either-or. You should either do the first (and configure OS swapping) or the second part.

Having said that, I recommend setting proper Native Persistence instead.

Regards,
-- 
Ilya Kasnacheev


сб, 25 июл. 2020 г. в 04:49, 38797715 <[hidden email]>:

Hi,

https://apacheignite.readme.io/docs/swap-space

According to the above document, if the physical memory is small, you can solve this problem by opening the swap space,The specific method is to configure maxSize to a larger value (i.e. larger than the physical memory), and the swapPath property needs to be configured.

But from the test results, the node is terminated.

I think the correct result should be that even if the amount of data exceeds the physical memory, the node should still be able to run normally, but the data is exchanged to the disk.

I want to know what parameters affect the behavior of this configuration? vm.swappiness or others?

在 2020/7/24 下午9:55, aealexsandrov 写道:
Hi,

Can you please clarify your expectations? You expected that JVM process will
be killed instead of gracefully stopping? What you are going to achieve?

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/