Understanding Swap Space feature-- how it works?

classic Classic list List threaded Threaded
6 messages Options
edwardk edwardk
Reply | Threaded
Open this post in threaded view
|

Understanding Swap Space feature-- how it works?

Hi,

I am trying to understand the support in Ignite for spilling over to disk when memory limits are reached.

I see there is a feature of swap space as in the link below,
https://apacheignite.readme.io/docs/off-heap-memory#swap-space

But, I am not clear on how it works.

When we configure swap-space it appears the cached data would be stored in swap space.

But, what would the change be from the perspective of client. Is it completely transparent to the client.

I do understand that there would be a performance impact if it is enabled, as it would slow the cache operations as some data would be in memory and some in disk.

Does this feature work and can we rely on it for critical scenarios where we expect data size to be more which cannot be accommodated within the configured memory limits.

Also, is there a way we can configure it per cache and not globally so that for some items alone it can spill over to disk

vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|

Re: Understanding Swap Space feature-- how it works?

Hi,

Yes, swap space is transparent. Only keep in mind that whenever you read an entry from swap, it has to be unswapped first. So it's useful for the data that is rarely used, but still can't be accessed sometimes. If all you data is active, you will end up with a lot of swaps/unswaps which will seriously drop performance.

Swap can be enabled or disabled on per cache level via CacheConfiguration.setSwapEnabled property.

-Val
rushi_rashi rushi_rashi
Reply | Threaded
Open this post in threaded view
|

Re: Understanding Swap Space feature-- how it works?

Hi,
I have question related to swap memory.
If we have set setSwapEnabled to true,
a. When will be the data is written to disc?
b. If by default data is written to disc only when free space is not available in RAM, then can we limit the usage of RAM, lets say write to disc when 80% of RAM is used. Although eviction policy writes data to off heap memory but not to disc.
vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|

Re: Understanding Swap Space feature-- how it works?

Basically, swap space is another storage layer. You have on-heap, off-heap and then swap. So in case of on-heap mode only entry will be written in swap when it's evicted from cache due to eviction policy. With off-heap enabled it will be written to swap when it's evicted from there due to off-heap memory limit.

-Val
martinlundaskoe martinlundaskoe
Reply | Threaded
Open this post in threaded view
|

Re: Understanding Swap Space feature-- how it works?

Hi Valentin

TL/DR; How is data evicted from swap then?

I know this is an old issue, but I am also trying to understand the swap
feature.

This may be outdated by now due to the addition of Native Persistence, but I
have a use case where having swap instead of native persistence would be
ideal.

I first thought that the off-heap storage was defined per dataregion, and
the data retained in RAM would be limited by the XX:MaxDirectMemorySize, the
rest would be written to swap.

Now I al really confused, so now my question is now, how to I get the data
off swap?
I don't see anything related to eviction of data from the cache that was
written to swap... plus does it even make sense to have evicted data in
swap?


vkulichenko wrote
> Basically, swap space is another storage layer. You have on-heap, off-heap
> and then swap. So in case of on-heap mode only entry will be written in
> swap when it's evicted from cache due to eviction policy. With off-heap
> enabled it will be written to swap when it's evicted from there due to
> off-heap memory limit.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Understanding Swap Space feature-- how it works?

Hello!

As far as my understanding goes, you do not need to do anything special to get data off swap. You can just access any data in the cache, whether swapped out or not.

I'm not sure how it would interact with eviction.

Regards,
--
Ilya Kasnacheev


ср, 24 мар. 2021 г. в 10:26, martinlundaskoe <[hidden email]>:
Hi Valentin

TL/DR; How is data evicted from swap then?

I know this is an old issue, but I am also trying to understand the swap
feature.

This may be outdated by now due to the addition of Native Persistence, but I
have a use case where having swap instead of native persistence would be
ideal.

I first thought that the off-heap storage was defined per dataregion, and
the data retained in RAM would be limited by the XX:MaxDirectMemorySize, the
rest would be written to swap.

Now I al really confused, so now my question is now, how to I get the data
off swap?
I don't see anything related to eviction of data from the cache that was
written to swap... plus does it even make sense to have evicted data in
swap?


vkulichenko wrote
> Basically, swap space is another storage layer. You have on-heap, off-heap
> and then swap. So in case of on-heap mode only entry will be written in
> swap when it's evicted from cache due to eviction policy. With off-heap
> enabled it will be written to swap when it's evicted from there due to
> off-heap memory limit.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/