Unable to read fields value from BinaryObject

classic Classic list List threaded Threaded
3 messages Options
Wasim Bari Wasim Bari
Reply | Threaded
Open this post in threaded view
|

Unable to read fields value from BinaryObject

We are facing issue while retrieving values from binary objects if we use
custom id mapper.

We have two ignite clients.
1) Producer--> Creates cache and add entries to it.
2) Consumer--> Consumes cache entries created by producer. Reads data in
BinaryObject form and extracts values from it.

We notice that if we provide custom id mapper then consumer not able to
fetch values from BinaryObject. It always returns null. However, if we do
not
provide custom id mapper, then we are able to fetch values from BinaryObject
in consumer. Given are the java classes for your reference.

Employee Class
package com.myorg.ignite.producerconsumer;

import java.io.Serializable;
public class Employee implements Serializable{
        private String empName;
        private String deparment;
        private float height;
        private int experience;
       
        public Employee() {
               
        }
        public Employee(String empName, String department, float height, int
experience) {
                this.empName = empName;
                this.deparment = department;
                this.height = height;
                this.experience = experience;
        }
        public String getEmpName() {
                return empName;
        }
        public void setEmpName(String empName) {
                this.empName = empName;
        }
        public String getDeparment() {
                return deparment;
        }
        public void setDeparment(String deparment) {
                this.deparment = deparment;
        }
        public float getHeight() {
                return height;
        }
        public void setHeight(float height) {
                this.height = height;
        }
        public int getExperience() {
                return experience;
        }
        public void setExperience(int experience) {
                this.experience = experience;
        }

}

Utility class for cache creation
package com.myorg.ignite.producerconsumer;

import java.util.Arrays;
import org.apache.ignite.Ignite;
import org.apache.ignite.Ignition;
import org.apache.ignite.configuration.BinaryConfiguration;
import org.apache.ignite.configuration.IgniteConfiguration;
import org.apache.ignite.logger.slf4j.Slf4jLogger;
import org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi;
import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
import
org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;

import com.myorg.ignite.custommapper.CustomBinaryIdMapper;

public class CacheUtil {
        public static String cacheName = "mycache";
        public static String instanceName = "local-ignite";
       
          public static Ignite getIgniteInstance() {
                    IgniteConfiguration cfg = new IgniteConfiguration();
                        cfg.setIgniteInstanceName(instanceName);
                        cfg.setClientMode(true);
                        cfg.setGridLogger(new Slf4jLogger());
                        cfg.setBinaryConfiguration(getBinaryConfiguration());
                       
                       
                        TcpDiscoverySpi spi = new TcpDiscoverySpi();
                        TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
                        ipFinder.setAddresses(Arrays.asList("localhost"));
                        spi.setLocalPort(37508);
                        spi.setLocalPortRange(10);
                        TcpCommunicationSpi commSpi=new TcpCommunicationSpi();

                        commSpi.setLocalPort(37509);
                        spi.setIpFinder(ipFinder);

                        cfg.setDiscoverySpi(spi);
                        cfg.setCommunicationSpi(commSpi);
                       
                    return Ignition.start(cfg);
                }
         
          private static BinaryConfiguration getBinaryConfiguration() {
                    BinaryConfiguration bc = new BinaryConfiguration();
                    CustomBinaryIdMapper imlIdMapper = new CustomBinaryIdMapper();
                    bc.setIdMapper(imlIdMapper);
                    return bc;
          }

}

Producer class

package com.myorg.ignite.producerconsumer;

import java.io.Serializable;
import javax.cache.Cache;
import org.apache.ignite.Ignite;
import org.apache.ignite.configuration.CacheConfiguration;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class CacheProducer {
        private static final Logger LOG =
LoggerFactory.getLogger(CacheProducer.class);
        public static void main(String ...s) {
                Ignite ignite = CacheUtil.getIgniteInstance();
                Cache<String, Serializable> cache = createCache(CacheUtil.cacheName,
ignite);
                cache.getAndPut("emp1", new Employee("David", "Transport", 5.8f, 5));
                cache.close();
               
        }
          private static Cache<String, Serializable> createCache(String cacheName,
Ignite ignite) {
                    CacheConfiguration<String, Serializable> cacheConfig = new
CacheConfiguration<String, Serializable>();
                    cacheConfig.setName(cacheName);
                    cacheConfig.setIndexedTypes(String.class, Serializable.class);
                    Cache<String, Serializable> cache =
ignite.getOrCreateCache(cacheConfig);
                    return cache;
                }

}

Consumer class
package com.myorg.ignite.producerconsumer;

import javax.cache.Cache;
import org.apache.ignite.Ignite;
import org.apache.ignite.binary.BinaryField;
import org.apache.ignite.binary.BinaryObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class CacheConsumer {
        private static final Logger LOG =
LoggerFactory.getLogger(CacheConsumer.class);
        public static void main(String ...s) {
                Ignite ignite = CacheUtil.getIgniteInstance();
                //Cache<String, Serializable> cache = getCache(CacheUtil.cacheName,
ignite);
                Cache<String, BinaryObject> cache =
ignite.cache(CacheUtil.cacheName).withKeepBinary();
                BinaryObject bo = cache.get("emp1");
                traversFields(bo, "empName");
                traversFields(bo, "deparment");
                traversFields(bo, "height");
        }

        private static void traversFields(BinaryObject binaryObject, String
fieldName) {
                BinaryField bField =  binaryObject.type().field(fieldName);
                boolean flag = bField.exists(binaryObject);
                Object val = bField.value(binaryObject);
                //LOG.info("fieldName - {}, doesFieldExist:{}, value:{}", fieldName, flag,
val);
                System.out.println("fieldName - "+fieldName+", doesFieldExist:"+flag+",
value:"+val);
        }
}

CustomID mapper (Note: We created a jar for this class and placed it in libs
folder for apache ignite)
package com.myorg.ignite.custommapper;

import org.apache.ignite.binary.BinaryIdMapper;
public class CustomBinaryIdMapper implements BinaryIdMapper {

        public int typeId(String typeName) {
                // TODO Auto-generated method stub
                System.out.println("Calling typeId, typename:"+typeName);
                int hashCode = 31*typeName.hashCode()+47;
                return hashCode;
        }

        public int fieldId(int typeId, String fieldName) {
                // TODO Auto-generated method stub
                System.out.println("Calling fieldId, fieldName:"+fieldName);
                int hashCode = 31*fieldName.hashCode()+47;
                return hashCode;
        }

}

ignite config file

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd">
    <bean id="ignite.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name = "binaryConfiguration">
            <bean class =
"org.apache.ignite.configuration.BinaryConfiguration">
                <property name = "idMapper">
                    <bean class =
"com.myorg.ignite.custommapper.CustomBinaryIdMapper">
                    </bean>
                </property>
                <property name="compactFooter" value="true"/>
            </bean>
        </property>
       
        <property name="dataStorageConfiguration">
            <bean
class="org.apache.ignite.configuration.DataStorageConfiguration">
                <property name="defaultDataRegionConfiguration">
                    <bean
class="org.apache.ignite.configuration.DataRegionConfiguration">
                        <property name="persistenceEnabled" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>

       
        <property name="discoverySpi">
            <bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                   
                   
                    <bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                        <property name="addresses">
                            <list>
                               
                                <value>127.0.0.1:47500..47502</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
</beans>




-----
Wasim Bari
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Wasim Bari
Alexandr Shapkin Alexandr Shapkin
Reply | Threaded
Open this post in threaded view
|

Re: Unable to read fields value from BinaryObject

Hi,

Yes, I can reproduce the behavior. It could be an issue with the custom
binary mappers.

The weird thing is that the CustomBinaryIdMapper works fine with a single
node, i.e. if it reads and writes the value from a cache, then the mapper is
working. But fails on the other node.

Meanwhile, it's quite interesting, why do you want to replace the default
mapper with a custom one? What tasks are you trying to solve?



-----
Alex Shapkin
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Alex Shapkin
Alexandr Shapkin Alexandr Shapkin
Reply | Threaded
Open this post in threaded view
|

RE: Unable to read fields value from BinaryObject

Hi,

 

Just for the record, I noticed that the following tickets has been filed:

 

https://issues.apache.org/jira/browse/IGNITE-13415

https://issues.apache.org/jira/browse/IGNITE-13436

 

Also seems like the consumer would work if you do at least one cache#put operation.

 

From: [hidden email]
Sent: Thursday, September 10, 2020 3:04 PM
To: [hidden email]
Subject: Re: Unable to read fields value from BinaryObject

 

Hi,

 

Yes, I can reproduce the behavior. It could be an issue with the custom

binary mappers.

 

The weird thing is that the CustomBinaryIdMapper works fine with a single

node, i.e. if it reads and writes the value from a cache, then the mapper is

working. But fails on the other node.

 

Meanwhile, it's quite interesting, why do you want to replace the default

mapper with a custom one? What tasks are you trying to solve?

 

 

 

-----

Alex Shapkin

--

Sent from: http://apache-ignite-users.70518.x6.nabble.com/

 

Alex Shapkin