Working Directly with Binary objects

classic Classic list List threaded Threaded
9 messages Options
guill-melo guill-melo
Reply | Threaded
Open this post in threaded view
|

Working Directly with Binary objects

Hello All,
I have just recently started using Ignite, and I am really enjoying it, I am trying to profile the size of some objects and compare it with other types of serializations, is it possible to use the serialization without starting a Ignite process? The only way I could do it was:

       IgniteConfiguration cfg = new IgniteConfiguration();
        Ignite ignite = Ignition.start(cfg);
        Entity entity = Entity
                .build()
                .withId("12345")
                .withField("value field")
                .withOtherField(33)
                .build();
        BinaryObject binaryObject = ignite.binary().toBinary(entity);

Also, Is there a utility to convert the BinaryObject to byte[] ou should I just treat it like any other object?

Thanks !!
Alexei Scherbakov Alexei Scherbakov
Reply | Threaded
Open this post in threaded view
|

Re: Working Directly with Binary objects

Hi,

try something like:

String dir = System.getProperty("java.io.tmpdir");
IgniteUtils.setWorkDirectory(dir, null);
IgniteConfiguration iCfg = new IgniteConfiguration();
BinaryConfiguration bCfg = new BinaryConfiguration();
iCfg.setBinaryConfiguration(bCfg);
BinaryContext ctx = new BinaryContext(BinaryCachingMetadataHandler.create(), iCfg, new NullLogger());
BinaryMarshaller marsh = new BinaryMarshaller();
marsh.setContext(new MarshallerContextImpl(null));
IgniteUtils.invoke(BinaryMarshaller.class, marsh, "setBinaryContext", ctx, iCfg);

byte[] bytes = marsh.marshal(new HashMap<String, String>() {
{
put("1", "1");
}
});

For second question:

((BinaryObjectImpl)bo).array()



2016-05-24 0:05 GMT+03:00 guill-melo <[hidden email]>:
Hello All,
I have just recently started using Ignite, and I am really enjoying it, I am
trying to profile the size of some objects and compare it with other types
of serializations, is it possible to use the serialization without starting
a Ignite process? The only way I could do it was:

       IgniteConfiguration cfg = new IgniteConfiguration();
        Ignite ignite = Ignition.start(cfg);
        Entity entity = Entity
                .build()
                .withId("12345")
                .withField("value field")
                .withOtherField(33)
                .build();
        BinaryObject binaryObject = ignite.binary().toBinary(entity);

Also, Is there a utility to convert the BinaryObject to byte[] ou should I
just treat it like any other object?

Thanks !!



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Working-Directly-with-Binary-objects-tp5131.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.



--

Best regards,
Alexei Scherbakov
guill-melo guill-melo
Reply | Threaded
Open this post in threaded view
|

Re: Working Directly with Binary objects

Hello Alexei,
Thank you very much for the reply, that really helped me.

Just a quick follow up, how can I create a BinaryObject from a byte[]? I playing with a BinaryCacheStore and I want to persist the objects in the serialized BinaryObject format,  as its a lot core compact.

Thanks again.
Alexei Scherbakov Alexei Scherbakov
Reply | Threaded
Open this post in threaded view
|

Re: Working Directly with Binary objects

Hi,

You can marshall and unmarshall BinaryObject using marshaller like any other type.
BTW, what are you trying to achieve ?
If you store cache values as byte arrays, you need to deserialize them on each data access to do something useful.
And how do you conclude "as its a lot core compact" ?


2016-05-24 19:47 GMT+03:00 guill-melo <[hidden email]>:
Hello Alexei,
Thank you very much for the reply, that really helped me.

Just a quick follow up, how can I create a BinaryObject from a byte[]? I
playing with a BinaryCacheStore and I want to persist the objects in the
serialized BinaryObject format,  as its a lot core compact.

Thanks again.



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Working-Directly-with-Binary-objects-tp5131p5152.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.



--

Best regards,
Alexei Scherbakov
Denis Magda Denis Magda
Reply | Threaded
Open this post in threaded view
|

Re: Working Directly with Binary objects

In reply to this post by guill-melo
Hi,

What is your use case and where do you want to apply the techniques suggested in this discussion?

Actually, I would highly discourage you from using this code in production because it's based on private Ignite API that is used by Ignite developers only.

--
Denis
guill-melo guill-melo
Reply | Threaded
Open this post in threaded view
|

Re: Working Directly with Binary objects

In reply to this post by Alexei Scherbakov
Hello Alexei, Thanks again for your time,
by a lot more compact I mean that a object like:
        Entity entity = Entity
                .build()
                .withId("12345")
                .withField("value field")
                .withOtherField(33)
                .build();

using java.io serialization is 237 bytes, while a BinaryObject is only 58, so thats what I ment by "more compact" .

I am trying to  create a CacheStore that stores the objects as binaries, possibly using HBase, or something else that stores byte[].

This is mostly investigating, as I am very curious to create a cache with a 'big data' backing store, and though this would be a good opportunity to improve my Ignite knowledge, I came from the Coherence world.
Hope that makes a little more sense. I don't want to store the objects in the cache as byte[], only in the CacheStore, and the data should only be accessed though the cache.
Thanks.
guill-melo guill-melo
Reply | Threaded
Open this post in threaded view
|

Re: Working Directly with Binary objects

In reply to this post by Denis Magda
Hello Denis,
I am not trying to add this to production, I am investigating the feasibility of having a CacheStore that writes to HBase or something similar in a byte[] representation of the BynaryObject, as it is more efficient than java.io.

I am trying to  see what I can do with the API, as I am very new to it and am quite excited about it.

Thank you Denis
Denis Magda Denis Magda
Reply | Threaded
Open this post in threaded view
|

Re: Working Directly with Binary objects

Hi,

Actually it’s not a difficult task to write a cache entry in a byte representation into a storage if it’s needed. 
The easiest way is just to take an instance of a marshaller that is globally used (ignite.configuration().getMarshaller()) and convert a value and a key of an entry passed to CacheStore.write(Cache.Entry<? extends K, ? extends V> entry) with a call like marshaller.marshal(val.getValue()).

What is more interesting is to implement a CacheStore for some other persistent store. You can refer to existed implementations of 
CacheJdbcBlobStore and CassandraCacheStore.


Denis

On May 25, 2016, at 5:50 PM, guill-melo <[hidden email]> wrote:

Hello Denis,
I am not trying to add this to production, I am investigating the
feasibility of having a CacheStore that writes to HBase or something similar
in a byte[] representation of the BynaryObject, as it is more efficient than
java.io.

I am trying to  see what I can do with the API, as I am very new to it and
am quite excited about it.

Thank you Denis



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Working-Directly-with-Binary-objects-tp5131p5197.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

guill-melo guill-melo
Reply | Threaded
Open this post in threaded view
|

Re: Working Directly with Binary objects

Thank you very much Denis, will look into that.