Unsafe usage of org.h2.util.JdbcUtils in Ignite

classic Classic list List threaded Threaded
10 messages Options
Andrey Davydov Andrey Davydov
Reply | Threaded
Open this post in threaded view
|

Unsafe usage of org.h2.util.JdbcUtils in Ignite

Hello,


org.h2.util.JdbcUtils is utility class with all static methods  and configured via System.properties. So it system wide resource. It is incorrect inject Ignite specific settings in it.

 

this     - value: org.apache.ignite.internal.IgniteKernal #1

<- grid     - class: org.apache.ignite.internal.GridKernalContextImpl, value: org.apache.ignite.internal.IgniteKernal #1

  <- ctx     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing, value: org.apache.ignite.internal.GridKernalContextImpl #2

   <- this$0     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing #2

    <- serializer     - class: org.h2.util.JdbcUtils, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10 #1

     <- [5395]     - class: java.lang.Object[], value: org.h2.util.JdbcUtils class JdbcUtils

      <- elementData     - class: java.util.Vector, value: java.lang.Object[] #37309

       <- classes     - class: sun.misc.Launcher$AppClassLoader, value: java.util.Vector #31

        <- contextClassLoader (thread object)     - class: java.lang.Thread, value: sun.misc.Launcher$AppClassLoader #1

 

  1. It cause problems, if it need to work with H2 databases from same JVM where ignite run.
  2. It cause problems, when some Ignites run in same JVM
  3. It makes closed IgniteKernal reachable from GC root.

 

I think it is bad architecture solution to use this class and use H2 related system properties at all.

 

Andrey.

 

ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

Hello!

As far as my understanding goes:

1) It is H2's decision to exhibit JdbcUtil.serializer as their public API, they have a public system property to override it:

/**
* System property <code>h2.javaObjectSerializer</code>
* (default: null).<br />
* The JavaObjectSerializer class name for java objects being stored in
* column of type OTHER. It must be the same on client and server to work
* correctly.
*/
public static final String JAVA_OBJECT_SERIALIZER =
Utils.getProperty("h2.javaObjectSerializer", null);

Obviously, this is not designed for multi-tenancy of H2 in mind.

If you really need multi-tenancy, I recommend starting H2 in a separate class loader inherited from root class loader and isolated from any Ignite classes.

Regards,
--
Ilya Kasnacheev


ср, 18 мар. 2020 г. в 18:54, Andrey Davydov <[hidden email]>:

Hello,


org.h2.util.JdbcUtils is utility class with all static methods  and configured via System.properties. So it system wide resource. It is incorrect inject Ignite specific settings in it.

 

this     - value: org.apache.ignite.internal.IgniteKernal #1

<- grid     - class: org.apache.ignite.internal.GridKernalContextImpl, value: org.apache.ignite.internal.IgniteKernal #1

  <- ctx     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing, value: org.apache.ignite.internal.GridKernalContextImpl #2

   <- this$0     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing #2

    <- serializer     - class: org.h2.util.JdbcUtils, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10 #1

     <- [5395]     - class: java.lang.Object[], value: org.h2.util.JdbcUtils class JdbcUtils

      <- elementData     - class: java.util.Vector, value: java.lang.Object[] #37309

       <- classes     - class: sun.misc.Launcher$AppClassLoader, value: java.util.Vector #31

        <- contextClassLoader (thread object)     - class: java.lang.Thread, value: sun.misc.Launcher$AppClassLoader #1

 

  1. It cause problems, if it need to work with H2 databases from same JVM where ignite run.
  2. It cause problems, when some Ignites run in same JVM
  3. It makes closed IgniteKernal reachable from GC root.

 

I think it is bad architecture solution to use this class and use H2 related system properties at all.

 

Andrey.

 

Andrey Davydov Andrey Davydov
Reply | Threaded
Open this post in threaded view
|

RE: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

In reply to this post by Andrey Davydov

I have some RnD with Apache Felix this week to found workaround for multi-tenancy of H2.

 

But there is problem with some Ignites in same JVM.

 

As I see in org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing latest started Ignite will visible via JdbcUtils.serializer, and it can be already closed and it workdir can be deleted.

 

Line 2105:

 

 

        if (JdbcUtils.serializer != null)

            U.warn(log, "Custom H2 serialization is already configured, will override.");

 

        JdbcUtils.serializer = h2Serializer();

 

Line 2268:

 

    private JavaObjectSerializer h2Serializer() {

        return new JavaObjectSerializer() {  //nested class has link to parent IgniteH2Indexing and to ingnite instance transitively

            @Override public byte[] serialize(Object obj) throws Exception {

                return U.marshal(marshaller, obj); //In common case, binary marshaller logic depends on work dir

            }

 

            @Override public Object deserialize(byte[] bytes) throws Exception {

                ClassLoader clsLdr = ctx != null ? U.resolveClassLoader(ctx.config()) : null; //only configuration need, but all ctx leaked

 

                return U.unmarshal(marshaller, bytes, clsLdr);

            }

        };

    }

 

 

Andrey.

 

От: [hidden email]
Отправлено: 19 марта 2020 г. в 14:37
Кому: [hidden email]
Тема: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

Hello!

 

As far as my understanding goes:

 

1) It is H2's decision to exhibit JdbcUtil.serializer as their public API, they have a public system property to override it:

 

/**
* System property <code>h2.javaObjectSerializer</code>
* (default: null).<br />
* The JavaObjectSerializer class name for java objects being stored in
* column of type OTHER. It must be the same on client and server to work
* correctly.
*/
public static final String JAVA_OBJECT_SERIALIZER =
        Utils.getProperty(
"h2.javaObjectSerializer", null);

 

Obviously, this is not designed for multi-tenancy of H2 in mind.

 

If you really need multi-tenancy, I recommend starting H2 in a separate class loader inherited from root class loader and isolated from any Ignite classes.

 

Regards,

--

Ilya Kasnacheev

 

 

ср, 18 мар. 2020 г. в 18:54, Andrey Davydov <[hidden email]>:

Hello,

 

org.h2.util.JdbcUtils is utility class with all static methods  and configured via System.properties. So it system wide resource. It is incorrect inject Ignite specific settings in it.

 

this     - value: org.apache.ignite.internal.IgniteKernal #1

<- grid     - class: org.apache.ignite.internal.GridKernalContextImpl, value: org.apache.ignite.internal.IgniteKernal #1

  <- ctx     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing, value: org.apache.ignite.internal.GridKernalContextImpl #2

   <- this$0     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing #2

    <- serializer     - class: org.h2.util.JdbcUtils, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10 #1

     <- [5395]     - class: java.lang.Object[], value: org.h2.util.JdbcUtils class JdbcUtils

      <- elementData     - class: java.util.Vector, value: java.lang.Object[] #37309

       <- classes     - class: sun.misc.Launcher$AppClassLoader, value: java.util.Vector #31

        <- contextClassLoader (thread object)     - class: java.lang.Thread, value: sun.misc.Launcher$AppClassLoader #1

 

  1. It cause problems, if it need to work with H2 databases from same JVM where ignite run.
  2. It cause problems, when some Ignites run in same JVM
  3. It makes closed IgniteKernal reachable from GC root.

 

I think it is bad architecture solution to use this class and use H2 related system properties at all.

 

Andrey.

 

 

Andrey Davydov Andrey Davydov
Reply | Threaded
Open this post in threaded view
|

RE: RE: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

In reply to this post by Andrey Davydov

Seem that refactor h2serilizer method in following manner will be safe for marshallers which not depends on ignite instance and will be faster anyway, due to single clsLdr resolving. For binary marshaller solution is still unsafe =(((

   

    private JavaObjectSerializer h2Serializer() {

        ClassLoader clsLdr = ctx != null ? U.resolveClassLoader(ctx.config()) : null;

        return new CustomJavaObjectSerializer(marshaller, clsLdr);

    }

 

    private static final class CustomJavaObjectSerializer implements JavaObjectSerializer {

        private final Marshaller marshaller;

        private final ClassLoader clsLdr;

       

        CustomJavaObjectSerializer(Marshaller marshaller, ClassLoader clsLdr) {

            this.marshaller = marshaller;

            this.clsLdr = clsLdr;

        }

       

        @Override public byte[] serialize(Object obj) throws Exception {

            return U.marshal(marshaller, obj);

        }

 

        @Override public Object deserialize(byte[] bytes) throws Exception {

            return U.unmarshal(marshaller, bytes, clsLdr);

        }

    }

 

Andrey.

 

От: [hidden email]
Отправлено: 19 марта 2020 г. в 15:43
Кому: [hidden email]
Тема: RE: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

I have some RnD with Apache Felix this week to found workaround for multi-tenancy of H2.

 

But there is problem with some Ignites in same JVM.

 

As I see in org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing latest started Ignite will visible via JdbcUtils.serializer, and it can be already closed and it workdir can be deleted.

 

Line 2105:

 

 

        if (JdbcUtils.serializer != null)

            U.warn(log, "Custom H2 serialization is already configured, will override.");

 

        JdbcUtils.serializer = h2Serializer();

 

Line 2268:

 

    private JavaObjectSerializer h2Serializer() {

        return new JavaObjectSerializer() {  //nested class has link to parent IgniteH2Indexing and to ingnite instance transitively

            @Override public byte[] serialize(Object obj) throws Exception {

                return U.marshal(marshaller, obj); //In common case, binary marshaller logic depends on work dir

            }

 

            @Override public Object deserialize(byte[] bytes) throws Exception {

                ClassLoader clsLdr = ctx != null ? U.resolveClassLoader(ctx.config()) : null; //only configuration need, but all ctx leaked

 

                return U.unmarshal(marshaller, bytes, clsLdr);

            }

        };

    }

 

 

Andrey.

 

От: [hidden email]
Отправлено: 19 марта 2020 г. в 14:37
Кому: [hidden email]
Тема: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

Hello!

 

As far as my understanding goes:

 

1) It is H2's decision to exhibit JdbcUtil.serializer as their public API, they have a public system property to override it:

 

/**
* System property <code>h2.javaObjectSerializer</code>
* (default: null).<br />
* The JavaObjectSerializer class name for java objects being stored in
* column of type OTHER. It must be the same on client and server to work
* correctly.
*/
public static final String JAVA_OBJECT_SERIALIZER =
        Utils.getProperty(
"h2.javaObjectSerializer", null);

 

Obviously, this is not designed for multi-tenancy of H2 in mind.

 

If you really need multi-tenancy, I recommend starting H2 in a separate class loader inherited from root class loader and isolated from any Ignite classes.

 

Regards,

--

Ilya Kasnacheev

 

 

ср, 18 мар. 2020 г. в 18:54, Andrey Davydov <[hidden email]>:

Hello,

 

org.h2.util.JdbcUtils is utility class with all static methods  and configured via System.properties. So it system wide resource. It is incorrect inject Ignite specific settings in it.

 

this     - value: org.apache.ignite.internal.IgniteKernal #1

<- grid     - class: org.apache.ignite.internal.GridKernalContextImpl, value: org.apache.ignite.internal.IgniteKernal #1

  <- ctx     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing, value: org.apache.ignite.internal.GridKernalContextImpl #2

   <- this$0     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing #2

    <- serializer     - class: org.h2.util.JdbcUtils, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10 #1

     <- [5395]     - class: java.lang.Object[], value: org.h2.util.JdbcUtils class JdbcUtils

      <- elementData     - class: java.util.Vector, value: java.lang.Object[] #37309

       <- classes     - class: sun.misc.Launcher$AppClassLoader, value: java.util.Vector #31

        <- contextClassLoader (thread object)     - class: java.lang.Thread, value: sun.misc.Launcher$AppClassLoader #1

 

1.It cause problems, if it need to work with H2 databases from same JVM where ignite run.

2.It cause problems, when some Ignites run in same JVM

3.It makes closed IgniteKernal reachable from GC root.

 

I think it is bad architecture solution to use this class and use H2 related system properties at all.

 

Andrey.

 

 

 

ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

In reply to this post by Andrey Davydov
Hello!

I suggest raising these issues on developer list and/or filing tickets against IGNITE.

Regards,
--
Ilya Kasnacheev


чт, 19 мар. 2020 г. в 15:43, Andrey Davydov <[hidden email]>:

I have some RnD with Apache Felix this week to found workaround for multi-tenancy of H2.

 

But there is problem with some Ignites in same JVM.

 

As I see in org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing latest started Ignite will visible via JdbcUtils.serializer, and it can be already closed and it workdir can be deleted.

 

Line 2105:

 

 

        if (JdbcUtils.serializer != null)

            U.warn(log, "Custom H2 serialization is already configured, will override.");

 

        JdbcUtils.serializer = h2Serializer();

 

Line 2268:

 

    private JavaObjectSerializer h2Serializer() {

        return new JavaObjectSerializer() {  //nested class has link to parent IgniteH2Indexing and to ingnite instance transitively

            @Override public byte[] serialize(Object obj) throws Exception {

                return U.marshal(marshaller, obj); //In common case, binary marshaller logic depends on work dir

            }

 

            @Override public Object deserialize(byte[] bytes) throws Exception {

                ClassLoader clsLdr = ctx != null ? U.resolveClassLoader(ctx.config()) : null; //only configuration need, but all ctx leaked

 

                return U.unmarshal(marshaller, bytes, clsLdr);

            }

        };

    }

 

 

Andrey.

 

От: [hidden email]
Отправлено: 19 марта 2020 г. в 14:37
Кому: [hidden email]
Тема: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

Hello!

 

As far as my understanding goes:

 

1) It is H2's decision to exhibit JdbcUtil.serializer as their public API, they have a public system property to override it:

 

/**
* System property <code>h2.javaObjectSerializer</code>
* (default: null).<br />
* The JavaObjectSerializer class name for java objects being stored in
* column of type OTHER. It must be the same on client and server to work
* correctly.
*/
public static final String JAVA_OBJECT_SERIALIZER =
        Utils.getProperty(
"h2.javaObjectSerializer", null);

 

Obviously, this is not designed for multi-tenancy of H2 in mind.

 

If you really need multi-tenancy, I recommend starting H2 in a separate class loader inherited from root class loader and isolated from any Ignite classes.

 

Regards,

--

Ilya Kasnacheev

 

 

ср, 18 мар. 2020 г. в 18:54, Andrey Davydov <[hidden email]>:

Hello,

 

org.h2.util.JdbcUtils is utility class with all static methods  and configured via System.properties. So it system wide resource. It is incorrect inject Ignite specific settings in it.

 

this     - value: org.apache.ignite.internal.IgniteKernal #1

<- grid     - class: org.apache.ignite.internal.GridKernalContextImpl, value: org.apache.ignite.internal.IgniteKernal #1

  <- ctx     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing, value: org.apache.ignite.internal.GridKernalContextImpl #2

   <- this$0     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing #2

    <- serializer     - class: org.h2.util.JdbcUtils, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10 #1

     <- [5395]     - class: java.lang.Object[], value: org.h2.util.JdbcUtils class JdbcUtils

      <- elementData     - class: java.util.Vector, value: java.lang.Object[] #37309

       <- classes     - class: sun.misc.Launcher$AppClassLoader, value: java.util.Vector #31

        <- contextClassLoader (thread object)     - class: java.lang.Thread, value: sun.misc.Launcher$AppClassLoader #1

 

  1. It cause problems, if it need to work with H2 databases from same JVM where ignite run.
  2. It cause problems, when some Ignites run in same JVM
  3. It makes closed IgniteKernal reachable from GC root.

 

I think it is bad architecture solution to use this class and use H2 related system properties at all.

 

Andrey.

 

 

Andrew Mashenkov Andrew Mashenkov
Reply | Threaded
Open this post in threaded view
|

Re: RE: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

In reply to this post by Andrey Davydov
Hi,

In Apache Ignite master branch I see a separate class H2JavaObjectSerializer that implements JavaObjectSerializer.
Seems, this won't be released in 2.8

On Thu, Mar 19, 2020 at 4:03 PM Andrey Davydov <[hidden email]> wrote:

Seem that refactor h2serilizer method in following manner will be safe for marshallers which not depends on ignite instance and will be faster anyway, due to single clsLdr resolving. For binary marshaller solution is still unsafe =(((

   

    private JavaObjectSerializer h2Serializer() {

        ClassLoader clsLdr = ctx != null ? U.resolveClassLoader(ctx.config()) : null;

        return new CustomJavaObjectSerializer(marshaller, clsLdr);

    }

 

    private static final class CustomJavaObjectSerializer implements JavaObjectSerializer {

        private final Marshaller marshaller;

        private final ClassLoader clsLdr;

       

        CustomJavaObjectSerializer(Marshaller marshaller, ClassLoader clsLdr) {

            this.marshaller = marshaller;

            this.clsLdr = clsLdr;

        }

       

        @Override public byte[] serialize(Object obj) throws Exception {

            return U.marshal(marshaller, obj);

        }

 

        @Override public Object deserialize(byte[] bytes) throws Exception {

            return U.unmarshal(marshaller, bytes, clsLdr);

        }

    }

 

Andrey.

 

От: [hidden email]
Отправлено: 19 марта 2020 г. в 15:43
Кому: [hidden email]
Тема: RE: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

I have some RnD with Apache Felix this week to found workaround for multi-tenancy of H2.

 

But there is problem with some Ignites in same JVM.

 

As I see in org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing latest started Ignite will visible via JdbcUtils.serializer, and it can be already closed and it workdir can be deleted.

 

Line 2105:

 

 

        if (JdbcUtils.serializer != null)

            U.warn(log, "Custom H2 serialization is already configured, will override.");

 

        JdbcUtils.serializer = h2Serializer();

 

Line 2268:

 

    private JavaObjectSerializer h2Serializer() {

        return new JavaObjectSerializer() {  //nested class has link to parent IgniteH2Indexing and to ingnite instance transitively

            @Override public byte[] serialize(Object obj) throws Exception {

                return U.marshal(marshaller, obj); //In common case, binary marshaller logic depends on work dir

            }

 

            @Override public Object deserialize(byte[] bytes) throws Exception {

                ClassLoader clsLdr = ctx != null ? U.resolveClassLoader(ctx.config()) : null; //only configuration need, but all ctx leaked

 

                return U.unmarshal(marshaller, bytes, clsLdr);

            }

        };

    }

 

 

Andrey.

 

От: [hidden email]
Отправлено: 19 марта 2020 г. в 14:37
Кому: [hidden email]
Тема: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

Hello!

 

As far as my understanding goes:

 

1) It is H2's decision to exhibit JdbcUtil.serializer as their public API, they have a public system property to override it:

 

/**
* System property <code>h2.javaObjectSerializer</code>
* (default: null).<br />
* The JavaObjectSerializer class name for java objects being stored in
* column of type OTHER. It must be the same on client and server to work
* correctly.
*/
public static final String JAVA_OBJECT_SERIALIZER =
        Utils.getProperty(
"h2.javaObjectSerializer", null);

 

Obviously, this is not designed for multi-tenancy of H2 in mind.

 

If you really need multi-tenancy, I recommend starting H2 in a separate class loader inherited from root class loader and isolated from any Ignite classes.

 

Regards,

--

Ilya Kasnacheev

 

 

ср, 18 мар. 2020 г. в 18:54, Andrey Davydov <[hidden email]>:

Hello,

 

org.h2.util.JdbcUtils is utility class with all static methods  and configured via System.properties. So it system wide resource. It is incorrect inject Ignite specific settings in it.

 

this     - value: org.apache.ignite.internal.IgniteKernal #1

<- grid     - class: org.apache.ignite.internal.GridKernalContextImpl, value: org.apache.ignite.internal.IgniteKernal #1

  <- ctx     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing, value: org.apache.ignite.internal.GridKernalContextImpl #2

   <- this$0     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing #2

    <- serializer     - class: org.h2.util.JdbcUtils, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10 #1

     <- [5395]     - class: java.lang.Object[], value: org.h2.util.JdbcUtils class JdbcUtils

      <- elementData     - class: java.util.Vector, value: java.lang.Object[] #37309

       <- classes     - class: sun.misc.Launcher$AppClassLoader, value: java.util.Vector #31

        <- contextClassLoader (thread object)     - class: java.lang.Thread, value: sun.misc.Launcher$AppClassLoader #1

 

1.It cause problems, if it need to work with H2 databases from same JVM where ignite run.

2.It cause problems, when some Ignites run in same JVM

3.It makes closed IgniteKernal reachable from GC root.

 

I think it is bad architecture solution to use this class and use H2 related system properties at all.

 

Andrey.

 

 

 



--
Best regards,
Andrey V. Mashenkov
Regards,
Andrew.
Andrey Davydov Andrey Davydov
Reply | Threaded
Open this post in threaded view
|

RE: Re: RE: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

In reply to this post by Andrey Davydov

It seems like moving in right way =) Let wait for release.

 

Andrey.

 

От: [hidden email]
Отправлено: 19 марта 2020 г. в 16:28
Кому: [hidden email]
Тема: Re: RE: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

Hi,

 

In Apache Ignite master branch I see a separate class H2JavaObjectSerializer that implements JavaObjectSerializer.

Seems, this won't be released in 2.8

 

On Thu, Mar 19, 2020 at 4:03 PM Andrey Davydov <[hidden email]> wrote:

Seem that refactor h2serilizer method in following manner will be safe for marshallers which not depends on ignite instance and will be faster anyway, due to single clsLdr resolving. For binary marshaller solution is still unsafe =(((

   

    private JavaObjectSerializer h2Serializer() {

        ClassLoader clsLdr = ctx != null ? U.resolveClassLoader(ctx.config()) : null;

        return new CustomJavaObjectSerializer(marshaller, clsLdr);

    }

 

    private static final class CustomJavaObjectSerializer implements JavaObjectSerializer {

        private final Marshaller marshaller;

        private final ClassLoader clsLdr;

       

        CustomJavaObjectSerializer(Marshaller marshaller, ClassLoader clsLdr) {

            this.marshaller = marshaller;

            this.clsLdr = clsLdr;

        }

       

        @Override public byte[] serialize(Object obj) throws Exception {

            return U.marshal(marshaller, obj);

        }

 

        @Override public Object deserialize(byte[] bytes) throws Exception {

            return U.unmarshal(marshaller, bytes, clsLdr);

        }

    }

 

Andrey.

 

От: [hidden email]
Отправлено: 19 марта 2020 г. в 15:43
Кому: [hidden email]
Тема: RE: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

I have some RnD with Apache Felix this week to found workaround for multi-tenancy of H2.

 

But there is problem with some Ignites in same JVM.

 

As I see in org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing latest started Ignite will visible via JdbcUtils.serializer, and it can be already closed and it workdir can be deleted.

 

Line 2105:

 

 

        if (JdbcUtils.serializer != null)

            U.warn(log, "Custom H2 serialization is already configured, will override.");

 

        JdbcUtils.serializer = h2Serializer();

 

Line 2268:

 

    private JavaObjectSerializer h2Serializer() {

        return new JavaObjectSerializer() {  //nested class has link to parent IgniteH2Indexing and to ingnite instance transitively

            @Override public byte[] serialize(Object obj) throws Exception {

                return U.marshal(marshaller, obj); //In common case, binary marshaller logic depends on work dir

            }

 

            @Override public Object deserialize(byte[] bytes) throws Exception {

                ClassLoader clsLdr = ctx != null ? U.resolveClassLoader(ctx.config()) : null; //only configuration need, but all ctx leaked

 

                return U.unmarshal(marshaller, bytes, clsLdr);

            }

        };

    }

 

 

Andrey.

 

От: [hidden email]
Отправлено: 19 марта 2020 г. в 14:37
Кому: [hidden email]
Тема: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

Hello!

 

As far as my understanding goes:

 

1) It is H2's decision to exhibit JdbcUtil.serializer as their public API, they have a public system property to override it:

 

/**
* System property <code>h2.javaObjectSerializer</code>
* (default: null).<br />
* The JavaObjectSerializer class name for java objects being stored in
* column of type OTHER. It must be the same on client and server to work
* correctly.
*/
public static final String JAVA_OBJECT_SERIALIZER =
        Utils.getProperty(
"h2.javaObjectSerializer", null);

 

Obviously, this is not designed for multi-tenancy of H2 in mind.

 

If you really need multi-tenancy, I recommend starting H2 in a separate class loader inherited from root class loader and isolated from any Ignite classes.

 

Regards,

--

Ilya Kasnacheev

 

 

ср, 18 мар. 2020 г. в 18:54, Andrey Davydov <[hidden email]>:

Hello,

 

org.h2.util.JdbcUtils is utility class with all static methods  and configured via System.properties. So it system wide resource. It is incorrect inject Ignite specific settings in it.

 

this     - value: org.apache.ignite.internal.IgniteKernal #1

<- grid     - class: org.apache.ignite.internal.GridKernalContextImpl, value: org.apache.ignite.internal.IgniteKernal #1

  <- ctx     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing, value: org.apache.ignite.internal.GridKernalContextImpl #2

   <- this$0     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing #2

    <- serializer     - class: org.h2.util.JdbcUtils, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10 #1

     <- [5395]     - class: java.lang.Object[], value: org.h2.util.JdbcUtils class JdbcUtils

      <- elementData     - class: java.util.Vector, value: java.lang.Object[] #37309

       <- classes     - class: sun.misc.Launcher$AppClassLoader, value: java.util.Vector #31

        <- contextClassLoader (thread object)     - class: java.lang.Thread, value: sun.misc.Launcher$AppClassLoader #1

 

1.It cause problems, if it need to work with H2 databases from same JVM where ignite run.

2.It cause problems, when some Ignites run in same JVM

3.It makes closed IgniteKernal reachable from GC root.

 

I think it is bad architecture solution to use this class and use H2 related system properties at all.

 

Andrey.

 

 

 


 

--

Best regards,
Andrey V. Mashenkov

 

Andrey Davydov Andrey Davydov
Reply | Threaded
Open this post in threaded view
|

Re: Re: RE: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

Hello,

Current implementation is really unsafe for multiple Ignite in same JVM.
In tests for our system when we stop/start nodes in different order we get following error:

Caused by: class org.apache.ignite.IgniteCheckedException: Failed to initialize system DB connection: jdbc:h2:mem:fd191fac-c2f1-4398-bf8a-0dcddf651830;LOCK_MODE=3;MULTI_THREADED=1;DB_CLOSE_ON_EXIT=FALSE;DEFAULT_LOCK_TIMEOUT=10000;FUNCTIONS_IN_SCHEMA=true;OPTIMIZE_REUSE_RESULTS=0;QUERY_CACHE_SIZE=0;MAX_OPERATION_MEMORY=0;BATCH_JOINS=1;ROW_FACTORY="org.apache.ignite.internal.processors.query.h2.opt.H2PlainRowFactory";DEFAULT_TABLE_ENGINE=org.apache.ignite.internal.processors.query.h2.opt.GridH2DefaultTableEngine
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1402)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1703)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1117)
at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1035)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:921)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:820)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:690)
at com.example.testutils.TestNode.start(TestNode.java:75)
... 38 more
Caused by: class org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to initialize system DB connection: jdbc:h2:mem:fd191fac-c2f1-4398-bf8a-0dcddf651830;LOCK_MODE=3;MULTI_THREADED=1;DB_CLOSE_ON_EXIT=FALSE;DEFAULT_LOCK_TIMEOUT=10000;FUNCTIONS_IN_SCHEMA=true;OPTIMIZE_REUSE_RESULTS=0;QUERY_CACHE_SIZE=0;MAX_OPERATION_MEMORY=0;BATCH_JOINS=1;ROW_FACTORY="org.apache.ignite.internal.processors.query.h2.opt.H2PlainRowFactory";DEFAULT_TABLE_ENGINE=org.apache.ignite.internal.processors.query.h2.opt.GridH2DefaultTableEngine
at org.apache.ignite.internal.processors.query.h2.ConnectionManager.connectionNoCache(ConnectionManager.java:213)
at org.apache.ignite.internal.processors.query.h2.ConnectionManager.<init>(ConnectionManager.java:152)
at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.start(IgniteH2Indexing.java:2070)
at org.apache.ignite.internal.processors.query.GridQueryProcessor.start(GridQueryProcessor.java:256)
at org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1978)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1212)
... 46 more
Caused by: java.sql.SQLException: No suitable driver found for jdbc:h2:mem:fd191fac-c2f1-4398-bf8a-0dcddf651830;LOCK_MODE=3;MULTI_THREADED=1;DB_CLOSE_ON_EXIT=FALSE;DEFAULT_LOCK_TIMEOUT=10000;FUNCTIONS_IN_SCHEMA=true;OPTIMIZE_REUSE_RESULTS=0;QUERY_CACHE_SIZE=0;MAX_OPERATION_MEMORY=0;BATCH_JOINS=1;ROW_FACTORY="org.apache.ignite.internal.processors.query.h2.opt.H2PlainRowFactory";DEFAULT_TABLE_ENGINE=org.apache.ignite.internal.processors.query.h2.opt.GridH2DefaultTableEngine
at java.sql.DriverManager.getConnection(DriverManager.java:689)
at java.sql.DriverManager.getConnection(DriverManager.java:270)
at org.apache.ignite.internal.processors.query.h2.ConnectionManager.connectionNoCache(ConnectionManager.java:206)
... 51 more


On Thu, Mar 19, 2020 at 6:52 PM Andrey Davydov <[hidden email]> wrote:

It seems like moving in right way =) Let wait for release.

 

Andrey.

 

От: [hidden email]
Отправлено: 19 марта 2020 г. в 16:28
Кому: [hidden email]
Тема: Re: RE: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

Hi,

 

In Apache Ignite master branch I see a separate class H2JavaObjectSerializer that implements JavaObjectSerializer.

Seems, this won't be released in 2.8

 

On Thu, Mar 19, 2020 at 4:03 PM Andrey Davydov <[hidden email]> wrote:

Seem that refactor h2serilizer method in following manner will be safe for marshallers which not depends on ignite instance and will be faster anyway, due to single clsLdr resolving. For binary marshaller solution is still unsafe =(((

   

    private JavaObjectSerializer h2Serializer() {

        ClassLoader clsLdr = ctx != null ? U.resolveClassLoader(ctx.config()) : null;

        return new CustomJavaObjectSerializer(marshaller, clsLdr);

    }

 

    private static final class CustomJavaObjectSerializer implements JavaObjectSerializer {

        private final Marshaller marshaller;

        private final ClassLoader clsLdr;

       

        CustomJavaObjectSerializer(Marshaller marshaller, ClassLoader clsLdr) {

            this.marshaller = marshaller;

            this.clsLdr = clsLdr;

        }

       

        @Override public byte[] serialize(Object obj) throws Exception {

            return U.marshal(marshaller, obj);

        }

 

        @Override public Object deserialize(byte[] bytes) throws Exception {

            return U.unmarshal(marshaller, bytes, clsLdr);

        }

    }

 

Andrey.

 

От: [hidden email]
Отправлено: 19 марта 2020 г. в 15:43
Кому: [hidden email]
Тема: RE: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

I have some RnD with Apache Felix this week to found workaround for multi-tenancy of H2.

 

But there is problem with some Ignites in same JVM.

 

As I see in org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing latest started Ignite will visible via JdbcUtils.serializer, and it can be already closed and it workdir can be deleted.

 

Line 2105:

 

 

        if (JdbcUtils.serializer != null)

            U.warn(log, "Custom H2 serialization is already configured, will override.");

 

        JdbcUtils.serializer = h2Serializer();

 

Line 2268:

 

    private JavaObjectSerializer h2Serializer() {

        return new JavaObjectSerializer() {  //nested class has link to parent IgniteH2Indexing and to ingnite instance transitively

            @Override public byte[] serialize(Object obj) throws Exception {

                return U.marshal(marshaller, obj); //In common case, binary marshaller logic depends on work dir

            }

 

            @Override public Object deserialize(byte[] bytes) throws Exception {

                ClassLoader clsLdr = ctx != null ? U.resolveClassLoader(ctx.config()) : null; //only configuration need, but all ctx leaked

 

                return U.unmarshal(marshaller, bytes, clsLdr);

            }

        };

    }

 

 

Andrey.

 

От: [hidden email]
Отправлено: 19 марта 2020 г. в 14:37
Кому: [hidden email]
Тема: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

Hello!

 

As far as my understanding goes:

 

1) It is H2's decision to exhibit JdbcUtil.serializer as their public API, they have a public system property to override it:

 

/**
* System property <code>h2.javaObjectSerializer</code>
* (default: null).<br />
* The JavaObjectSerializer class name for java objects being stored in
* column of type OTHER. It must be the same on client and server to work
* correctly.
*/
public static final String JAVA_OBJECT_SERIALIZER =
        Utils.getProperty(
"h2.javaObjectSerializer", null);

 

Obviously, this is not designed for multi-tenancy of H2 in mind.

 

If you really need multi-tenancy, I recommend starting H2 in a separate class loader inherited from root class loader and isolated from any Ignite classes.

 

Regards,

--

Ilya Kasnacheev

 

 

ср, 18 мар. 2020 г. в 18:54, Andrey Davydov <[hidden email]>:

Hello,

 

org.h2.util.JdbcUtils is utility class with all static methods  and configured via System.properties. So it system wide resource. It is incorrect inject Ignite specific settings in it.

 

this     - value: org.apache.ignite.internal.IgniteKernal #1

<- grid     - class: org.apache.ignite.internal.GridKernalContextImpl, value: org.apache.ignite.internal.IgniteKernal #1

  <- ctx     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing, value: org.apache.ignite.internal.GridKernalContextImpl #2

   <- this$0     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing #2

    <- serializer     - class: org.h2.util.JdbcUtils, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10 #1

     <- [5395]     - class: java.lang.Object[], value: org.h2.util.JdbcUtils class JdbcUtils

      <- elementData     - class: java.util.Vector, value: java.lang.Object[] #37309

       <- classes     - class: sun.misc.Launcher$AppClassLoader, value: java.util.Vector #31

        <- contextClassLoader (thread object)     - class: java.lang.Thread, value: sun.misc.Launcher$AppClassLoader #1

 

1.It cause problems, if it need to work with H2 databases from same JVM where ignite run.

2.It cause problems, when some Ignites run in same JVM

3.It makes closed IgniteKernal reachable from GC root.

 

I think it is bad architecture solution to use this class and use H2 related system properties at all.

 

Andrey.

 

 

 


 

--

Best regards,
Andrey V. Mashenkov

 

ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Re: RE: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

Hello!

I have never seen a stack trace like this one. Can you provide a reproducer for this behavior?

Regards,
--
Ilya Kasnacheev


пт, 20 мар. 2020 г. в 20:20, Andrey Davydov <[hidden email]>:
Hello,

Current implementation is really unsafe for multiple Ignite in same JVM.
In tests for our system when we stop/start nodes in different order we get following error:

Caused by: class org.apache.ignite.IgniteCheckedException: Failed to initialize system DB connection: jdbc:h2:mem:fd191fac-c2f1-4398-bf8a-0dcddf651830;LOCK_MODE=3;MULTI_THREADED=1;DB_CLOSE_ON_EXIT=FALSE;DEFAULT_LOCK_TIMEOUT=10000;FUNCTIONS_IN_SCHEMA=true;OPTIMIZE_REUSE_RESULTS=0;QUERY_CACHE_SIZE=0;MAX_OPERATION_MEMORY=0;BATCH_JOINS=1;ROW_FACTORY="org.apache.ignite.internal.processors.query.h2.opt.H2PlainRowFactory";DEFAULT_TABLE_ENGINE=org.apache.ignite.internal.processors.query.h2.opt.GridH2DefaultTableEngine
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1402)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1703)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1117)
at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1035)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:921)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:820)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:690)
at com.example.testutils.TestNode.start(TestNode.java:75)
... 38 more
Caused by: class org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to initialize system DB connection: jdbc:h2:mem:fd191fac-c2f1-4398-bf8a-0dcddf651830;LOCK_MODE=3;MULTI_THREADED=1;DB_CLOSE_ON_EXIT=FALSE;DEFAULT_LOCK_TIMEOUT=10000;FUNCTIONS_IN_SCHEMA=true;OPTIMIZE_REUSE_RESULTS=0;QUERY_CACHE_SIZE=0;MAX_OPERATION_MEMORY=0;BATCH_JOINS=1;ROW_FACTORY="org.apache.ignite.internal.processors.query.h2.opt.H2PlainRowFactory";DEFAULT_TABLE_ENGINE=org.apache.ignite.internal.processors.query.h2.opt.GridH2DefaultTableEngine
at org.apache.ignite.internal.processors.query.h2.ConnectionManager.connectionNoCache(ConnectionManager.java:213)
at org.apache.ignite.internal.processors.query.h2.ConnectionManager.<init>(ConnectionManager.java:152)
at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.start(IgniteH2Indexing.java:2070)
at org.apache.ignite.internal.processors.query.GridQueryProcessor.start(GridQueryProcessor.java:256)
at org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1978)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1212)
... 46 more
Caused by: java.sql.SQLException: No suitable driver found for jdbc:h2:mem:fd191fac-c2f1-4398-bf8a-0dcddf651830;LOCK_MODE=3;MULTI_THREADED=1;DB_CLOSE_ON_EXIT=FALSE;DEFAULT_LOCK_TIMEOUT=10000;FUNCTIONS_IN_SCHEMA=true;OPTIMIZE_REUSE_RESULTS=0;QUERY_CACHE_SIZE=0;MAX_OPERATION_MEMORY=0;BATCH_JOINS=1;ROW_FACTORY="org.apache.ignite.internal.processors.query.h2.opt.H2PlainRowFactory";DEFAULT_TABLE_ENGINE=org.apache.ignite.internal.processors.query.h2.opt.GridH2DefaultTableEngine
at java.sql.DriverManager.getConnection(DriverManager.java:689)
at java.sql.DriverManager.getConnection(DriverManager.java:270)
at org.apache.ignite.internal.processors.query.h2.ConnectionManager.connectionNoCache(ConnectionManager.java:206)
... 51 more


On Thu, Mar 19, 2020 at 6:52 PM Andrey Davydov <[hidden email]> wrote:

It seems like moving in right way =) Let wait for release.

 

Andrey.

 

От: [hidden email]
Отправлено: 19 марта 2020 г. в 16:28
Кому: [hidden email]
Тема: Re: RE: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

Hi,

 

In Apache Ignite master branch I see a separate class H2JavaObjectSerializer that implements JavaObjectSerializer.

Seems, this won't be released in 2.8

 

On Thu, Mar 19, 2020 at 4:03 PM Andrey Davydov <[hidden email]> wrote:

Seem that refactor h2serilizer method in following manner will be safe for marshallers which not depends on ignite instance and will be faster anyway, due to single clsLdr resolving. For binary marshaller solution is still unsafe =(((

   

    private JavaObjectSerializer h2Serializer() {

        ClassLoader clsLdr = ctx != null ? U.resolveClassLoader(ctx.config()) : null;

        return new CustomJavaObjectSerializer(marshaller, clsLdr);

    }

 

    private static final class CustomJavaObjectSerializer implements JavaObjectSerializer {

        private final Marshaller marshaller;

        private final ClassLoader clsLdr;

       

        CustomJavaObjectSerializer(Marshaller marshaller, ClassLoader clsLdr) {

            this.marshaller = marshaller;

            this.clsLdr = clsLdr;

        }

       

        @Override public byte[] serialize(Object obj) throws Exception {

            return U.marshal(marshaller, obj);

        }

 

        @Override public Object deserialize(byte[] bytes) throws Exception {

            return U.unmarshal(marshaller, bytes, clsLdr);

        }

    }

 

Andrey.

 

От: [hidden email]
Отправлено: 19 марта 2020 г. в 15:43
Кому: [hidden email]
Тема: RE: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

I have some RnD with Apache Felix this week to found workaround for multi-tenancy of H2.

 

But there is problem with some Ignites in same JVM.

 

As I see in org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing latest started Ignite will visible via JdbcUtils.serializer, and it can be already closed and it workdir can be deleted.

 

Line 2105:

 

 

        if (JdbcUtils.serializer != null)

            U.warn(log, "Custom H2 serialization is already configured, will override.");

 

        JdbcUtils.serializer = h2Serializer();

 

Line 2268:

 

    private JavaObjectSerializer h2Serializer() {

        return new JavaObjectSerializer() {  //nested class has link to parent IgniteH2Indexing and to ingnite instance transitively

            @Override public byte[] serialize(Object obj) throws Exception {

                return U.marshal(marshaller, obj); //In common case, binary marshaller logic depends on work dir

            }

 

            @Override public Object deserialize(byte[] bytes) throws Exception {

                ClassLoader clsLdr = ctx != null ? U.resolveClassLoader(ctx.config()) : null; //only configuration need, but all ctx leaked

 

                return U.unmarshal(marshaller, bytes, clsLdr);

            }

        };

    }

 

 

Andrey.

 

От: [hidden email]
Отправлено: 19 марта 2020 г. в 14:37
Кому: [hidden email]
Тема: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

Hello!

 

As far as my understanding goes:

 

1) It is H2's decision to exhibit JdbcUtil.serializer as their public API, they have a public system property to override it:

 

/**
* System property <code>h2.javaObjectSerializer</code>
* (default: null).<br />
* The JavaObjectSerializer class name for java objects being stored in
* column of type OTHER. It must be the same on client and server to work
* correctly.
*/
public static final String JAVA_OBJECT_SERIALIZER =
        Utils.getProperty(
"h2.javaObjectSerializer", null);

 

Obviously, this is not designed for multi-tenancy of H2 in mind.

 

If you really need multi-tenancy, I recommend starting H2 in a separate class loader inherited from root class loader and isolated from any Ignite classes.

 

Regards,

--

Ilya Kasnacheev

 

 

ср, 18 мар. 2020 г. в 18:54, Andrey Davydov <[hidden email]>:

Hello,

 

org.h2.util.JdbcUtils is utility class with all static methods  and configured via System.properties. So it system wide resource. It is incorrect inject Ignite specific settings in it.

 

this     - value: org.apache.ignite.internal.IgniteKernal #1

<- grid     - class: org.apache.ignite.internal.GridKernalContextImpl, value: org.apache.ignite.internal.IgniteKernal #1

  <- ctx     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing, value: org.apache.ignite.internal.GridKernalContextImpl #2

   <- this$0     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing #2

    <- serializer     - class: org.h2.util.JdbcUtils, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10 #1

     <- [5395]     - class: java.lang.Object[], value: org.h2.util.JdbcUtils class JdbcUtils

      <- elementData     - class: java.util.Vector, value: java.lang.Object[] #37309

       <- classes     - class: sun.misc.Launcher$AppClassLoader, value: java.util.Vector #31

        <- contextClassLoader (thread object)     - class: java.lang.Thread, value: sun.misc.Launcher$AppClassLoader #1

 

1.It cause problems, if it need to work with H2 databases from same JVM where ignite run.

2.It cause problems, when some Ignites run in same JVM

3.It makes closed IgniteKernal reachable from GC root.

 

I think it is bad architecture solution to use this class and use H2 related system properties at all.

 

Andrey.

 

 

 


 

--

Best regards,
Andrey V. Mashenkov

 

Andrey Davydov Andrey Davydov
Reply | Threaded
Open this post in threaded view
|

RE: Re: Re: RE: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

In reply to this post by Andrey Davydov

When I got OutOfMemory and become research heapDumps, I have found some bugs and leaks not only in Ignite but in our code too. So I already implement some fixes in our system, and I don’t see this error more. May be it was because of our logic (we register/unregister drivers too). So if I get it again, I will submit reproduce scenario.

 

Andrey.

 

От: [hidden email]
Отправлено: 25 марта 2020 г. в 15:37
Кому: [hidden email]
Тема: Re: Re: RE: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

Hello!

 

I have never seen a stack trace like this one. Can you provide a reproducer for this behavior?

 

Regards,

--

Ilya Kasnacheev

 

 

пт, 20 мар. 2020 г. в 20:20, Andrey Davydov <[hidden email]>:

Hello,

 

Current implementation is really unsafe for multiple Ignite in same JVM.

In tests for our system when we stop/start nodes in different order we get following error:

 

Caused by: class org.apache.ignite.IgniteCheckedException: Failed to initialize system DB connection: jdbc:h2:mem:fd191fac-c2f1-4398-bf8a-0dcddf651830;LOCK_MODE=3;MULTI_THREADED=1;DB_CLOSE_ON_EXIT=FALSE;DEFAULT_LOCK_TIMEOUT=10000;FUNCTIONS_IN_SCHEMA=true;OPTIMIZE_REUSE_RESULTS=0;QUERY_CACHE_SIZE=0;MAX_OPERATION_MEMORY=0;BATCH_JOINS=1;ROW_FACTORY="org.apache.ignite.internal.processors.query.h2.opt.H2PlainRowFactory";DEFAULT_TABLE_ENGINE=org.apache.ignite.internal.processors.query.h2.opt.GridH2DefaultTableEngine
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1402)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1703)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1117)
at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1035)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:921)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:820)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:690)
at com.example.testutils.TestNode.start(TestNode.java:75)
... 38 more
Caused by: class org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to initialize system DB connection: jdbc:h2:mem:fd191fac-c2f1-4398-bf8a-0dcddf651830;LOCK_MODE=3;MULTI_THREADED=1;DB_CLOSE_ON_EXIT=FALSE;DEFAULT_LOCK_TIMEOUT=10000;FUNCTIONS_IN_SCHEMA=true;OPTIMIZE_REUSE_RESULTS=0;QUERY_CACHE_SIZE=0;MAX_OPERATION_MEMORY=0;BATCH_JOINS=1;ROW_FACTORY="org.apache.ignite.internal.processors.query.h2.opt.H2PlainRowFactory";DEFAULT_TABLE_ENGINE=org.apache.ignite.internal.processors.query.h2.opt.GridH2DefaultTableEngine
at org.apache.ignite.internal.processors.query.h2.ConnectionManager.connectionNoCache(ConnectionManager.java:213)
at org.apache.ignite.internal.processors.query.h2.ConnectionManager.<init>(ConnectionManager.java:152)
at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.start(IgniteH2Indexing.java:2070)
at org.apache.ignite.internal.processors.query.GridQueryProcessor.start(GridQueryProcessor.java:256)
at org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1978)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1212)
... 46 more
Caused by: java.sql.SQLException: No suitable driver found for jdbc:h2:mem:fd191fac-c2f1-4398-bf8a-0dcddf651830;LOCK_MODE=3;MULTI_THREADED=1;DB_CLOSE_ON_EXIT=FALSE;DEFAULT_LOCK_TIMEOUT=10000;FUNCTIONS_IN_SCHEMA=true;OPTIMIZE_REUSE_RESULTS=0;QUERY_CACHE_SIZE=0;MAX_OPERATION_MEMORY=0;BATCH_JOINS=1;ROW_FACTORY="org.apache.ignite.internal.processors.query.h2.opt.H2PlainRowFactory";DEFAULT_TABLE_ENGINE=org.apache.ignite.internal.processors.query.h2.opt.GridH2DefaultTableEngine
at java.sql.DriverManager.getConnection(DriverManager.java:689)
at java.sql.DriverManager.getConnection(DriverManager.java:270)
at org.apache.ignite.internal.processors.query.h2.ConnectionManager.connectionNoCache(ConnectionManager.java:206)
... 51 more

 

 

On Thu, Mar 19, 2020 at 6:52 PM Andrey Davydov <[hidden email]> wrote:

It seems like moving in right way =) Let wait for release.

 

Andrey.

 

От: [hidden email]
Отправлено: 19 марта 2020 г. в 16:28
Кому: [hidden email]
Тема: Re: RE: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

Hi,

 

In Apache Ignite master branch I see a separate class H2JavaObjectSerializer that implements JavaObjectSerializer.

Seems, this won't be released in 2.8

 

On Thu, Mar 19, 2020 at 4:03 PM Andrey Davydov <[hidden email]> wrote:

Seem that refactor h2serilizer method in following manner will be safe for marshallers which not depends on ignite instance and will be faster anyway, due to single clsLdr resolving. For binary marshaller solution is still unsafe =(((

   

    private JavaObjectSerializer h2Serializer() {

        ClassLoader clsLdr = ctx != null ? U.resolveClassLoader(ctx.config()) : null;

        return new CustomJavaObjectSerializer(marshaller, clsLdr);

    }

 

    private static final class CustomJavaObjectSerializer implements JavaObjectSerializer {

        private final Marshaller marshaller;

        private final ClassLoader clsLdr;

       

        CustomJavaObjectSerializer(Marshaller marshaller, ClassLoader clsLdr) {

            this.marshaller = marshaller;

            this.clsLdr = clsLdr;

        }

       

        @Override public byte[] serialize(Object obj) throws Exception {

            return U.marshal(marshaller, obj);

        }

 

        @Override public Object deserialize(byte[] bytes) throws Exception {

            return U.unmarshal(marshaller, bytes, clsLdr);

        }

    }

 

Andrey.

 

От: [hidden email]
Отправлено: 19 марта 2020 г. в 15:43
Кому: [hidden email]
Тема: RE: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

I have some RnD with Apache Felix this week to found workaround for multi-tenancy of H2.

 

But there is problem with some Ignites in same JVM.

 

As I see in org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing latest started Ignite will visible via JdbcUtils.serializer, and it can be already closed and it workdir can be deleted.

 

Line 2105:

 

 

        if (JdbcUtils.serializer != null)

            U.warn(log, "Custom H2 serialization is already configured, will override.");

 

        JdbcUtils.serializer = h2Serializer();

 

Line 2268:

 

    private JavaObjectSerializer h2Serializer() {

        return new JavaObjectSerializer() {  //nested class has link to parent IgniteH2Indexing and to ingnite instance transitively

            @Override public byte[] serialize(Object obj) throws Exception {

                return U.marshal(marshaller, obj); //In common case, binary marshaller logic depends on work dir

            }

 

            @Override public Object deserialize(byte[] bytes) throws Exception {

                ClassLoader clsLdr = ctx != null ? U.resolveClassLoader(ctx.config()) : null; //only configuration need, but all ctx leaked

 

                return U.unmarshal(marshaller, bytes, clsLdr);

            }

        };

    }

 

 

Andrey.

 

От: [hidden email]
Отправлено: 19 марта 2020 г. в 14:37
Кому: [hidden email]
Тема: Re: Unsafe usage of org.h2.util.JdbcUtils in Ignite

 

Hello!

 

As far as my understanding goes:

 

1) It is H2's decision to exhibit JdbcUtil.serializer as their public API, they have a public system property to override it:

 

/**
* System property <code>h2.javaObjectSerializer</code>
* (default: null).<br />
* The JavaObjectSerializer class name for java objects being stored in
* column of type OTHER. It must be the same on client and server to work
* correctly.
*/
public static final String JAVA_OBJECT_SERIALIZER =
        Utils.getProperty(
"h2.javaObjectSerializer", null);

 

Obviously, this is not designed for multi-tenancy of H2 in mind.

 

If you really need multi-tenancy, I recommend starting H2 in a separate class loader inherited from root class loader and isolated from any Ignite classes.

 

Regards,

--

Ilya Kasnacheev

 

 

ср, 18 мар. 2020 г. в 18:54, Andrey Davydov <[hidden email]>:

Hello,

 

org.h2.util.JdbcUtils is utility class with all static methods  and configured via System.properties. So it system wide resource. It is incorrect inject Ignite specific settings in it.

 

this     - value: org.apache.ignite.internal.IgniteKernal #1

<- grid     - class: org.apache.ignite.internal.GridKernalContextImpl, value: org.apache.ignite.internal.IgniteKernal #1

  <- ctx     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing, value: org.apache.ignite.internal.GridKernalContextImpl #2

   <- this$0     - class: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing #2

    <- serializer     - class: org.h2.util.JdbcUtils, value: org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$10 #1

     <- [5395]     - class: java.lang.Object[], value: org.h2.util.JdbcUtils class JdbcUtils

      <- elementData     - class: java.util.Vector, value: java.lang.Object[] #37309

       <- classes     - class: sun.misc.Launcher$AppClassLoader, value: java.util.Vector #31

        <- contextClassLoader (thread object)     - class: java.lang.Thread, value: sun.misc.Launcher$AppClassLoader #1

 

1.It cause problems, if it need to work with H2 databases from same JVM where ignite run.

2.It cause problems, when some Ignites run in same JVM

3.It makes closed IgniteKernal reachable from GC root.

 

I think it is bad architecture solution to use this class and use H2 related system properties at all.

 

Andrey.

 

 

 


 

--

Best regards,
Andrey V. Mashenkov