Ignite Client Node OOM Issue

classic Classic list List threaded Threaded
5 messages Options
Ravi Makwana Ravi Makwana
Reply | Threaded
Open this post in threaded view
|

Ignite Client Node OOM Issue

HI,

We are using Apache Ignite 2.7.0 binary and servers are using Linux OS & app servers are using Windows OS.We are using Apache Ignite .Net APIs.

Recently we have noticed that our application (Client Node) is stopping with OOM error. App server has 32 GB RAM & we are specifying JVM Heap = 8 GB.

App server utilization at the OOM error was 80%  and we have still 20% memory left, still we are facing this issue.

1) How to check memory usage for the client node? 
2)  Is there any way by looking at logs we can understand, OOM might occur? 
3)  Could you please suggest, how to overcome this issue?

 I am sharing an application crash log. 
 
Thanks & Regards,

OOM-Error.rar (12K) Download Attachment
Vladimir Pligin Vladimir Pligin
Reply | Threaded
Open this post in threaded view
|

Re: Ignite Client Node OOM Issue

What load profile do you have? Do you use SQL queries? Is it possible to
share your client node configuration?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Ravi Makwana Ravi Makwana
Reply | Threaded
Open this post in threaded view
|

Re: Ignite Client Node OOM Issue

Hi,

1) What load profile do you have? 
Ans: We have 2 clusters, each having 2 nodes, one cluster is having approx 15 GB data (Replication) & second cluster is having approx 5 GB data (Partitioned) with eviction policy.

2) Do you use SQL queries?
Ans: Yes, We are using.

3) Is it possible to share your client node configuration?
Ans: Yes, I have attached below.

Thanks,


On Fri, 6 Nov 2020 at 18:50, Vladimir Pligin <[hidden email]> wrote:
What load profile do you have? Do you use SQL queries? Is it possible to
share your client node configuration?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

ClinetConfiguration.txt (2K) Download Attachment
Taras Ledkov Taras Ledkov
Reply | Threaded
Open this post in threaded view
|

Re: Ignite Client Node OOM Issue

Hi,

SQL query may be potential cause of the OOM.

Please check that:
1. Simple scan+filter queries (e.g. SELECT * FROM <src> WHERE <cond>) that may produce big results run in lazy mode (SqlFieldsQuery#setLazy);
2. Queries that requires not indexing sort or group aggregates don't produce  big result set (lazy mode isn't applied).

On 06.11.2020 17:07, Ravi Makwana wrote:
Hi,

1) What load profile do you have? 
Ans: We have 2 clusters, each having 2 nodes, one cluster is having approx 15 GB data (Replication) & second cluster is having approx 5 GB data (Partitioned) with eviction policy.

2) Do you use SQL queries?
Ans: Yes, We are using.

3) Is it possible to share your client node configuration?
Ans: Yes, I have attached below.

Thanks,


On Fri, 6 Nov 2020 at 18:50, Vladimir Pligin <[hidden email]> wrote:
What load profile do you have? Do you use SQL queries? Is it possible to
share your client node configuration?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
-- 
Taras Ledkov
Mail-To: [hidden email]
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Ignite Client Node OOM Issue

In reply to this post by Ravi Makwana
Hello!

I recommend taking a heap dump from this node when it is nearing OOM, checking it with e.g. Eclipse MAT to see what is the leak suspect/class histogram.

Then share the findings with us.
--
Ilya Kasnacheev


ср, 4 нояб. 2020 г. в 19:45, Ravi Makwana <[hidden email]>:
HI,

We are using Apache Ignite 2.7.0 binary and servers are using Linux OS & app servers are using Windows OS.We are using Apache Ignite .Net APIs.

Recently we have noticed that our application (Client Node) is stopping with OOM error. App server has 32 GB RAM & we are specifying JVM Heap = 8 GB.

App server utilization at the OOM error was 80%  and we have still 20% memory left, still we are facing this issue.

1) How to check memory usage for the client node? 
2)  Is there any way by looking at logs we can understand, OOM might occur? 
3)  Could you please suggest, how to overcome this issue?

 I am sharing an application crash log. 
 
Thanks & Regards,