Write behind problems

classic Classic list List threaded Threaded
7 messages Options
amits amits
Reply | Threaded
Open this post in threaded view
|

Write behind problems

This post has NOT been accepted by the mailing list yet.
Hello,
I'm have a cache that is used to store binary data(byte[]), the key of that cache is a string and the value is a struct with the following fields :
        boolean isNew - indexed
        String id
        byte[] buffer

I utilize a CPU intensive write behind process for this cache (the process compress the buffer before saving it the disk) .
I've encountered 2 problems with the write behind process :
1.it's seems that while the write behind process is running, all the other requests (get/put) for this cache are halted until the write behind process is done.
2.the write behind process is handling the expired items serially resulting slow performances as only one core is used to execute the CPU intensive write behind process.

Here is the code i use to subscribe to EVT_CACHE_OBJECT_EXPIRED events :
                _ignite = Ignition.start(Constants.GRID_GAIN_CONFIG_FILEPATH);
                IgniteBiPredicate<UUID, CacheEvent> lsnr = new IgniteBiPredicate<UUID, CacheEvent>() {
                @Override public boolean apply(UUID uu, CacheEvent evt) {
                Tile expiredObject = (Tile)evt.oldValue();
                 if (expiredObject.getIsNew()){
                  PersistencyManager.write(expiredObject, Constants.BUFFER_PATH);
                }
               
                    return true;
                }
                };
                _ignite.events(_ignite.cluster()).remoteListen(lsnr , null,EventType.EVT_CACHE_OBJECT_EXPIRED);

Is there a way to solve those issues?
Thanks,
Amit.
vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|

Re: Write behind problems

Hi Amit,

I sounds like a misconfiguration issue. Updating a persistent store in write-behind mode always happens in the background and in multithreaded fashion. Can you share your cache configuration so that we can take a look and suggest proper changes?

Also here is the documentation paragraph about write-behind caching: http://apacheignite.readme.io/docs/persistent-store#write-behind-caching

-Val
Ognen Duzlevski Ognen Duzlevski
Reply | Threaded
Open this post in threaded view
|

Re: Write behind problems

Hey guys,

I still think the user list is misconfigured. I am quoting the reply below by Val to a message that I never saw pass by in the user list.

Ognen

On Tue, Jul 28, 2015 at 5:57 PM, vkulichenko <[hidden email]> wrote:
Hi Amit,

I sounds like a misconfiguration issue. Updating a persistent store in
write-behind mode always happens in the background and in multithreaded
fashion. Can you share your cache configuration so that we can take a look
and suggest proper changes?

Also here is the documentation paragraph about write-behind caching:
http://apacheignite.readme.io/docs/persistent-store#write-behind-caching

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Write-behind-problems-tp734p739.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Konstantin Boudnik Konstantin Boudnik
Reply | Threaded
Open this post in threaded view
|

Re: Write behind problems

Same here. Looks like nabble's messages aren't get copied to this list.

Cos

On Tue, Jul 28, 2015 at 06:30PM, Ognen Duzlevski wrote:

>    Hey guys,
>    I still think the user list is misconfigured. I am quoting the reply below
>    by Val to a message that I never saw pass by in the user list.
>    Ognen
>    On Tue, Jul 28, 2015 at 5:57 PM, vkulichenko
>    <[hidden email]> wrote:
>
>      Hi Amit,
>
>      I sounds like a misconfiguration issue. Updating a persistent store in
>      write-behind mode always happens in the background and in multithreaded
>      fashion. Can you share your cache configuration so that we can take a
>      look
>      and suggest proper changes?
>
>      Also here is the documentation paragraph about write-behind caching:
>      http://apacheignite.readme.io/docs/persistent-store#write-behind-caching
>
>      -Val
>
>      --
>      View this message in context:
>      http://apache-ignite-users.70518.x6.nabble.com/Write-behind-problems-tp734p739.html
>      Sent from the Apache Ignite Users mailing list archive at Nabble.com.
dsetrakyan dsetrakyan
Reply | Threaded
Open this post in threaded view
|

Re: Write behind problems

In reply to this post by Ognen Duzlevski
Ognen,

Such posts usually occur on the nabble forum, when users do not subscribe to the user list: http://apache-ignite-users.70518.x6.nabble.com/Write-behind-problems-td734.html

D.

On Tue, Jul 28, 2015 at 4:30 PM, Ognen Duzlevski <[hidden email]> wrote:
Hey guys,

I still think the user list is misconfigured. I am quoting the reply below by Val to a message that I never saw pass by in the user list.

Ognen

On Tue, Jul 28, 2015 at 5:57 PM, vkulichenko <[hidden email]> wrote:
Hi Amit,

I sounds like a misconfiguration issue. Updating a persistent store in
write-behind mode always happens in the background and in multithreaded
fashion. Can you share your cache configuration so that we can take a look
and suggest proper changes?

Also here is the documentation paragraph about write-behind caching:
http://apacheignite.readme.io/docs/persistent-store#write-behind-caching

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Write-behind-problems-tp734p739.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.


Ognen Duzlevski Ognen Duzlevski
Reply | Threaded
Open this post in threaded view
|

Re: Write behind problems

On Tue, Jul 28, 2015 at 6:48 PM, Dmitriy Setrakyan <[hidden email]> wrote:
Ognen,

Such posts usually occur on the nabble forum, when users do not subscribe to the user list: http://apache-ignite-users.70518.x6.nabble.com/Write-behind-problems-td734.html

D.

I think we should only let people post to the user list if they subscribe to it - this is how almost every project I know of functions. This Nabble business is annoying, to say the least and makes the user list less useful :-)

Ognen



On Tue, Jul 28, 2015 at 4:30 PM, Ognen Duzlevski <[hidden email]> wrote:
Hey guys,

I still think the user list is misconfigured. I am quoting the reply below by Val to a message that I never saw pass by in the user list.

Ognen

On Tue, Jul 28, 2015 at 5:57 PM, vkulichenko <[hidden email]> wrote:
Hi Amit,

I sounds like a misconfiguration issue. Updating a persistent store in
write-behind mode always happens in the background and in multithreaded
fashion. Can you share your cache configuration so that we can take a look
and suggest proper changes?

Also here is the documentation paragraph about write-behind caching:
http://apacheignite.readme.io/docs/persistent-store#write-behind-caching

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Write-behind-problems-tp734p739.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.



Konstantin Boudnik Konstantin Boudnik
Reply | Threaded
Open this post in threaded view
|

Re: Post from unsubscribed users

It seems the issue here that the initial posts aren't even sent to the list's
moderator. So none of us have a chance to accept the initial post. I wonder if
this is something with how this forum is configured on nabble?

Cos

On Thu, Jul 30, 2015 at 10:52AM, Ognen Duzlevski wrote:

> On Tue, Jul 28, 2015 at 6:48 PM, Dmitriy Setrakyan <[hidden email]>
>  wrote:
>
> > Ognen,
> >
> > Such posts usually occur on the nabble forum, when users do not subscribe
> > to the user list:
> > http://apache-ignite-users.70518.x6.nabble.com/Write-behind-problems-td734.html
> >
> > D.
> >
>
> I think we should only let people post to the user list if they subscribe
> to it - this is how almost every project I know of functions. This Nabble
> business is annoying, to say the least and makes the user list less useful
> :-)
>
> Ognen
>
>
>
> > On Tue, Jul 28, 2015 at 4:30 PM, Ognen Duzlevski <
> > [hidden email]> wrote:
> >
> >> Hey guys,
> >>
> >> I still think the user list is misconfigured. I am quoting the reply
> >> below by Val to a message that I never saw pass by in the user list.
> >>
> >> Ognen