IgniteRDD SQL query stalled or extremly slow on large input argument

classic Classic list List threaded Threaded
2 messages Options
amgad amgad
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

IgniteRDD SQL query stalled or extremly slow on large input argument

I have a query that stalls/hangs over large argument inputs. The same code works well on smaller SQL argument input. The code is as follows:

CODE:
------
subEntries = dataRDD.sql("SELECT v.id,v.sub,v.obj FROM VPRow v JOIN table(id bigint = ?) i ON v.id = i.id",new Object[] {subKeyEntries.toArray()});
LOG.debug("Reading : "+subEntries.count());

Please note that the Ignite documentation mentions that the input argument can be of any size - "Here you can provide object array (Object[]) of any length as a parameter". The parameter passed to the query in the stalling case was of size 23641 of long values.

My spring configuration file is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd">

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="cacheConfiguration">
            <list>
                <bean class="org.apache.ignite.configuration.CacheConfiguration">
                   
                    <property name="name" value="dataRDD"/>
                   
                    <property name="cacheMode" value="PARTITIONED"/>
                   
                    <property name="indexedTypes">
                        <list>
                            <value>java.lang.Long</value>
                            <value>sample.VPRow</value>
                        </list>
                    </property>
                    <property name="backups" value="0"/>
                </bean>
            </list>
        </property>
       
        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                        <property name="addresses">
                            <list>
                                <value>[IP1]</value>
                                <value>[...]</value>
                                <value>[IP5]</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
</beans>

The VPRow class is defined as follows

public class VPRow implements Serializable {

    @QuerySqlField
    private long id;
    @QuerySqlField
    private String sub;
    @QuerySqlField
    private String obj;

    public VPRow(long id,String sub, String obj) {
        this.id = id;
        this.sub = sub;
        this.obj = obj;
    }
    ...
}
dkarachentsev dkarachentsev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: IgniteRDD SQL query stalled or extremly slow on large input argument

Hi,

Could you please attach thread dumps from all nodes?

Thanks!
-Dmitry.

Loading...