WAL is growing exponentially

classic Classic list List threaded Threaded
2 messages Options
rihan123 rihan123
Reply | Threaded
Open this post in threaded view
|

WAL is growing exponentially

Hi
I am doing POC to see if Ignite is good for my usecase and i achieved
required data ingestion rate but after ingesting huge data i am seeing WAL
disk is growing exponentially.
To check the number of records ingested i just connected to visor after this
i saw WAL is growing exponentially.
There are lot of messeges like this --


Could not clear historyMap due to WAL reservation on cp: CheckpointEntry
[id=c1f85a50-c218-4e38-a692-0aa87380d295, timestamp=1606979161926,
ptr=FileWALPointer [idx=1427, fileOff=51216143, len=9667]], history map size
is 4
Could not clear historyMap due to WAL reservation on cp: CheckpointEntry
[id=c1f85a50-c218-4e38-a692-0aa87380d295, timestamp=1606979161926,
ptr=FileWALPointer [idx=1427, fileOff=51216143, len=9667]], history map size
is 5
Could not clear historyMap due to WAL reservation on cp: CheckpointEntry
[id=c1f85a50-c218-4e38-a692-0aa87380d295, timestamp=1606979161926,
ptr=FileWALPointer [idx=1427, fileOff=51216143, len=9667]], history map size
is 6
Could not clear historyMap due to WAL reservation on cp: CheckpointEntry
[id=c1f85a50-c218-4e38-a692-0aa87380d295, timestamp=1606979161926,
ptr=FileWALPointer [idx=1427, fileOff=51216143, len=9667]], history map size
is 7
Could not clear historyMap due to WAL reservation on cp: CheckpointEntry
[id=5b4cfe31-99e6-4c00-b306-9c590df8fdfb, timestamp=1606979174328,
ptr=FileWALPointer [idx=1520, fileOff=59350658, len=10066]], history map
size is 2
Could not clear historyMap due to WAL reservation on cp: CheckpointEntry
[id=5b4cfe31-99e6-4c00-b306-9c590df8fdfb, timestamp=1606979174328,
ptr=FileWALPointer [idx=1520, fileOff=59350658, len=10066]], history map
size is 4
Could not clear historyMap due to WAL reservation on cp: CheckpointEntry
[id=5b4cfe31-99e6-4c00-b306-9c590df8fdfb, timestamp=1606979174328,
ptr=FileWALPointer [idx=1520, fileOff=59350658, len=10066]], history map
size is 5


Any suggestions on this issue is greatly appreciated



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Alexandr Shapkin Alexandr Shapkin
Reply | Threaded
Open this post in threaded view
|

RE: WAL is growing exponentially

Hi,

 

What version do you use?

Is it possible to share the full log files?

 

From: [hidden email]
Sent: Friday, December 4, 2020 2:32 PM
To: [hidden email]
Subject: WAL is growing exponentially

 

Hi

I am doing POC to see if Ignite is good for my usecase and i achieved

required data ingestion rate but after ingesting huge data i am seeing WAL

disk is growing exponentially.

To check the number of records ingested i just connected to visor after this

i saw WAL is growing exponentially.

There are lot of messeges like this --

 

 

Could not clear historyMap due to WAL reservation on cp: CheckpointEntry

[id=c1f85a50-c218-4e38-a692-0aa87380d295, timestamp=1606979161926,

ptr=FileWALPointer [idx=1427, fileOff=51216143, len=9667]], history map size

is 4

Could not clear historyMap due to WAL reservation on cp: CheckpointEntry

[id=c1f85a50-c218-4e38-a692-0aa87380d295, timestamp=1606979161926,

ptr=FileWALPointer [idx=1427, fileOff=51216143, len=9667]], history map size

is 5

Could not clear historyMap due to WAL reservation on cp: CheckpointEntry

[id=c1f85a50-c218-4e38-a692-0aa87380d295, timestamp=1606979161926,

ptr=FileWALPointer [idx=1427, fileOff=51216143, len=9667]], history map size

is 6

Could not clear historyMap due to WAL reservation on cp: CheckpointEntry

[id=c1f85a50-c218-4e38-a692-0aa87380d295, timestamp=1606979161926,

ptr=FileWALPointer [idx=1427, fileOff=51216143, len=9667]], history map size

is 7

Could not clear historyMap due to WAL reservation on cp: CheckpointEntry

[id=5b4cfe31-99e6-4c00-b306-9c590df8fdfb, timestamp=1606979174328,

ptr=FileWALPointer [idx=1520, fileOff=59350658, len=10066]], history map

size is 2

Could not clear historyMap due to WAL reservation on cp: CheckpointEntry

[id=5b4cfe31-99e6-4c00-b306-9c590df8fdfb, timestamp=1606979174328,

ptr=FileWALPointer [idx=1520, fileOff=59350658, len=10066]], history map

size is 4

Could not clear historyMap due to WAL reservation on cp: CheckpointEntry

[id=5b4cfe31-99e6-4c00-b306-9c590df8fdfb, timestamp=1606979174328,

ptr=FileWALPointer [idx=1520, fileOff=59350658, len=10066]], history map

size is 5

 

 

Any suggestions on this issue is greatly appreciated

 

 

 

--

Sent from: http://apache-ignite-users.70518.x6.nabble.com/

 

Alex Shapkin