Ignite singel instance

classic Classic list List threaded Threaded
17 messages Options
Siew Wai Yow Siew Wai Yow
Reply | Threaded
Open this post in threaded view
|

Ignite singel instance

Hi,

We are doing POC on ignite performance using ODBC driver but the performance is always capped at around 1700 TPS which is too slow. It is local ignite service. All tuning tips from Ignite page has been applied, no bottleneck from CPU and Memory. At the moment not turn on persistence yet, it will be worse if turn on. This POC is very crucial to our product roadmap. 

Any tips? Thank you.

Test case,
1 x insert --> 49 x select -->49 x update --> 1 x delete
repeat for 50000 times.

    const char* insert_sql = "insert into CDRTEST values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
    const char* update_sql = "update CDRTEST set value01=?,value02=?,value03=?,value04=?,value05=?,value21=?,value22=?,value23=?,value24=?,value25=? where key01=?";
    const char* delete_sql = "delete from CDRTEST where key01=?";
    const char* select_sql = "select value01 from CDRTEST where key01=?";

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE TABLE IF NOT EXISTS CDRTEST ( "
  "key01 VARCHAR PRIMARY KEY, "
  "value01 LONG, "
  "value02 LONG, "
  "value03 LONG, "
  "value04 LONG, "
  "value05 LONG, "
  "value06 LONG, "
  "value07 LONG, "
  "value08 LONG, "
  "value09 LONG, "
  "value10 LONG, "
  "value11 LONG, "
  "value12 LONG, "
  "value13 LONG, "
  "value14 LONG, "
  "value15 LONG, "
  "value16 LONG, "
  "value17 LONG, "
  "value18 LONG, "
  "value19 LONG, "
  "value20 LONG, "  
  "value21 VARCHAR, "
  "value22 VARCHAR, "
  "value23 VARCHAR, "
  "value24 VARCHAR, "
  "value25 VARCHAR, "
  "value26 VARCHAR, "
  "value27 VARCHAR, "
  "value28 VARCHAR, "
  "value29 VARCHAR, "
  "value30 VARCHAR, "    
  "value31 VARCHAR, "
  "value32 VARCHAR, "
  "value33 VARCHAR, "
  "value34 VARCHAR, "
  "value35 VARCHAR, "
  "value36 VARCHAR, "
  "value37 VARCHAR, "
  "value38 VARCHAR, "
  "value39 VARCHAR) "
  "WITH \"template=partitioned,atomicity=TRANSACTIONAL_SNAPSHOT,WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC\"")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create table", hstmt, SQL_HANDLE_STMT);

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE INDEX key01t_idx ON CDRTEST(key01) INLINE_SIZE 64")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create index", hstmt, SQL_HANDLE_STMT);


Below are configuration we used,

<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd">

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="binaryConfiguration">
            <bean class="org.apache.ignite.configuration.BinaryConfiguration">
                <property name="compactFooter" value="false"/>

                <property name="idMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                        <property name="lowerCase" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>

  <property name="clientConnectorConfiguration">
    <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration">
      <!--property name="host" value="127.0.0.1"/-->
      <!--property name="port" value="10800"/-->
      <!--property name="portRange" value="5"/-->
      <property name="maxOpenCursorsPerConnection" value="512"/>
      <property name="socketSendBufferSize" value="65536"/>
      <property name="socketReceiveBufferSize" value="131072"/>
      <!--property name="threadPoolSize" value="4"/-->
    </bean>
  </property>

        <property name="dataStorageConfiguration">
           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
              <!-- Enabling Apache Ignite Persistent Store. -->
              <property name="defaultDataRegionConfiguration">
                 <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                     <property name="persistenceEnabled" value="false"/>
                 </bean>
              </property>
              <property name="walPath" value="/ignite-wal"/>

              <property name="walArchivePath" value="/ignite-wal"/>
           </bean>
        </property>

    </bean>
</beans>

Regards,
Yow
ezhuravlev ezhuravlev
Reply | Threaded
Open this post in threaded view
|

Re: Ignite singel instance

Hi,

Do you use only one ODBC client? Can you start one more ODBC client and check the performance?

Thanks,
Evgenii

сб, 2 нояб. 2019 г. в 16:47, Siew Wai Yow <[hidden email]>:
Hi,

We are doing POC on ignite performance using ODBC driver but the performance is always capped at around 1700 TPS which is too slow. It is local ignite service. All tuning tips from Ignite page has been applied, no bottleneck from CPU and Memory. At the moment not turn on persistence yet, it will be worse if turn on. This POC is very crucial to our product roadmap. 

Any tips? Thank you.

Test case,
1 x insert --> 49 x select -->49 x update --> 1 x delete
repeat for 50000 times.

    const char* insert_sql = "insert into CDRTEST values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
    const char* update_sql = "update CDRTEST set value01=?,value02=?,value03=?,value04=?,value05=?,value21=?,value22=?,value23=?,value24=?,value25=? where key01=?";
    const char* delete_sql = "delete from CDRTEST where key01=?";
    const char* select_sql = "select value01 from CDRTEST where key01=?";

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE TABLE IF NOT EXISTS CDRTEST ( "
  "key01 VARCHAR PRIMARY KEY, "
  "value01 LONG, "
  "value02 LONG, "
  "value03 LONG, "
  "value04 LONG, "
  "value05 LONG, "
  "value06 LONG, "
  "value07 LONG, "
  "value08 LONG, "
  "value09 LONG, "
  "value10 LONG, "
  "value11 LONG, "
  "value12 LONG, "
  "value13 LONG, "
  "value14 LONG, "
  "value15 LONG, "
  "value16 LONG, "
  "value17 LONG, "
  "value18 LONG, "
  "value19 LONG, "
  "value20 LONG, "  
  "value21 VARCHAR, "
  "value22 VARCHAR, "
  "value23 VARCHAR, "
  "value24 VARCHAR, "
  "value25 VARCHAR, "
  "value26 VARCHAR, "
  "value27 VARCHAR, "
  "value28 VARCHAR, "
  "value29 VARCHAR, "
  "value30 VARCHAR, "    
  "value31 VARCHAR, "
  "value32 VARCHAR, "
  "value33 VARCHAR, "
  "value34 VARCHAR, "
  "value35 VARCHAR, "
  "value36 VARCHAR, "
  "value37 VARCHAR, "
  "value38 VARCHAR, "
  "value39 VARCHAR) "
  "WITH \"template=partitioned,atomicity=TRANSACTIONAL_SNAPSHOT,WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC\"")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create table", hstmt, SQL_HANDLE_STMT);

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE INDEX key01t_idx ON CDRTEST(key01) INLINE_SIZE 64")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create index", hstmt, SQL_HANDLE_STMT);


Below are configuration we used,

<?xml version="1.0" encoding="UTF-8"?>

       xsi:schemaLocation="http://www.springframework.org/schema/beans

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="binaryConfiguration">
            <bean class="org.apache.ignite.configuration.BinaryConfiguration">
                <property name="compactFooter" value="false"/>

                <property name="idMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                        <property name="lowerCase" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>

  <property name="clientConnectorConfiguration">
    <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration">
      <!--property name="host" value="127.0.0.1"/-->
      <!--property name="port" value="10800"/-->
      <!--property name="portRange" value="5"/-->
      <property name="maxOpenCursorsPerConnection" value="512"/>
      <property name="socketSendBufferSize" value="65536"/>
      <property name="socketReceiveBufferSize" value="131072"/>
      <!--property name="threadPoolSize" value="4"/-->
    </bean>
  </property>

        <property name="dataStorageConfiguration">
           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
              <!-- Enabling Apache Ignite Persistent Store. -->
              <property name="defaultDataRegionConfiguration">
                 <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                     <property name="persistenceEnabled" value="false"/>
                 </bean>
              </property>
              <property name="walPath" value="/ignite-wal"/>

              <property name="walArchivePath" value="/ignite-wal"/>
           </bean>
        </property>

    </bean>
</beans>

Regards,
Yow
Stanislav Lukyanov Stanislav Lukyanov
Reply | Threaded
Open this post in threaded view
|

Re: Ignite singel instance

First, 1700 TPS given your transaction structure is 170000 simple operations per second, which is quite substantial - especially if you're doing that from a single thread / single ODBC client.

Second, note that TRANSACTIONAL_SNAPSHOT is in beta and is not ready for production use. There are no claims about performance or stability of that cache mode.

Third, you don't need the index on key01 - it is created automatically because it is a primary key. But you do need to set INLINE SIZE of the default index. Run your Ignite server with system property or environment varialy IGNITE_MAX_INDEX_PAYLOAD_SIZE=64.

Finally, the operations you're doing don't look like something to be done with SQL. Consider using key-value API in Ignite C++ instead - https://apacheignite-cpp.readme.io/docs.

Stan

On Sat, Nov 2, 2019 at 8:15 PM Evgenii Zhuravlev <[hidden email]> wrote:
Hi,

Do you use only one ODBC client? Can you start one more ODBC client and check the performance?

Thanks,
Evgenii

сб, 2 нояб. 2019 г. в 16:47, Siew Wai Yow <[hidden email]>:
Hi,

We are doing POC on ignite performance using ODBC driver but the performance is always capped at around 1700 TPS which is too slow. It is local ignite service. All tuning tips from Ignite page has been applied, no bottleneck from CPU and Memory. At the moment not turn on persistence yet, it will be worse if turn on. This POC is very crucial to our product roadmap. 

Any tips? Thank you.

Test case,
1 x insert --> 49 x select -->49 x update --> 1 x delete
repeat for 50000 times.

    const char* insert_sql = "insert into CDRTEST values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
    const char* update_sql = "update CDRTEST set value01=?,value02=?,value03=?,value04=?,value05=?,value21=?,value22=?,value23=?,value24=?,value25=? where key01=?";
    const char* delete_sql = "delete from CDRTEST where key01=?";
    const char* select_sql = "select value01 from CDRTEST where key01=?";

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE TABLE IF NOT EXISTS CDRTEST ( "
  "key01 VARCHAR PRIMARY KEY, "
  "value01 LONG, "
  "value02 LONG, "
  "value03 LONG, "
  "value04 LONG, "
  "value05 LONG, "
  "value06 LONG, "
  "value07 LONG, "
  "value08 LONG, "
  "value09 LONG, "
  "value10 LONG, "
  "value11 LONG, "
  "value12 LONG, "
  "value13 LONG, "
  "value14 LONG, "
  "value15 LONG, "
  "value16 LONG, "
  "value17 LONG, "
  "value18 LONG, "
  "value19 LONG, "
  "value20 LONG, "  
  "value21 VARCHAR, "
  "value22 VARCHAR, "
  "value23 VARCHAR, "
  "value24 VARCHAR, "
  "value25 VARCHAR, "
  "value26 VARCHAR, "
  "value27 VARCHAR, "
  "value28 VARCHAR, "
  "value29 VARCHAR, "
  "value30 VARCHAR, "    
  "value31 VARCHAR, "
  "value32 VARCHAR, "
  "value33 VARCHAR, "
  "value34 VARCHAR, "
  "value35 VARCHAR, "
  "value36 VARCHAR, "
  "value37 VARCHAR, "
  "value38 VARCHAR, "
  "value39 VARCHAR) "
  "WITH \"template=partitioned,atomicity=TRANSACTIONAL_SNAPSHOT,WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC\"")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create table", hstmt, SQL_HANDLE_STMT);

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE INDEX key01t_idx ON CDRTEST(key01) INLINE_SIZE 64")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create index", hstmt, SQL_HANDLE_STMT);


Below are configuration we used,

<?xml version="1.0" encoding="UTF-8"?>

       xsi:schemaLocation="http://www.springframework.org/schema/beans

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="binaryConfiguration">
            <bean class="org.apache.ignite.configuration.BinaryConfiguration">
                <property name="compactFooter" value="false"/>

                <property name="idMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                        <property name="lowerCase" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>

  <property name="clientConnectorConfiguration">
    <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration">
      <!--property name="host" value="127.0.0.1"/-->
      <!--property name="port" value="10800"/-->
      <!--property name="portRange" value="5"/-->
      <property name="maxOpenCursorsPerConnection" value="512"/>
      <property name="socketSendBufferSize" value="65536"/>
      <property name="socketReceiveBufferSize" value="131072"/>
      <!--property name="threadPoolSize" value="4"/-->
    </bean>
  </property>

        <property name="dataStorageConfiguration">
           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
              <!-- Enabling Apache Ignite Persistent Store. -->
              <property name="defaultDataRegionConfiguration">
                 <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                     <property name="persistenceEnabled" value="false"/>
                 </bean>
              </property>
              <property name="walPath" value="/ignite-wal"/>

              <property name="walArchivePath" value="/ignite-wal"/>
           </bean>
        </property>

    </bean>
</beans>

Regards,
Yow
Siew Wai Yow Siew Wai Yow
Reply | Threaded
Open this post in threaded view
|

Re: Ignite singel instance

Hi Stan

The result will be the same without transaction, even with atomic mode. So wondering if there is other reason capped the performance. Attach is our POC source code which give around 1700 TPS, with or without transaction, may be you can advise what's wrong with it.

-DIGNITE_MAX_INDEX_PAYLOAD_SIZE=64 is using already.

We choose ODBC because it is more compatible to existing product's API, which will need to create table/cache on the fly without pre-define C-object. Ignite C++ need to have the cache C-object ready before run.

Also in Ignite cluster, how can we connect to cluster, just connect to one of the host? And when host crash we need to change manually connect to another node? Seems ODBC connection string not allow to input multiple host address.

Thanks in advance.

Regards,
Yow



From: Stanislav Lukyanov <[hidden email]>
Sent: Wednesday, November 6, 2019 12:33 AM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
First, 1700 TPS given your transaction structure is 170000 simple operations per second, which is quite substantial - especially if you're doing that from a single thread / single ODBC client.

Second, note that TRANSACTIONAL_SNAPSHOT is in beta and is not ready for production use. There are no claims about performance or stability of that cache mode.

Third, you don't need the index on key01 - it is created automatically because it is a primary key. But you do need to set INLINE SIZE of the default index. Run your Ignite server with system property or environment varialy IGNITE_MAX_INDEX_PAYLOAD_SIZE=64.

Finally, the operations you're doing don't look like something to be done with SQL. Consider using key-value API in Ignite C++ instead - https://apacheignite-cpp.readme.io/docs.

Stan

On Sat, Nov 2, 2019 at 8:15 PM Evgenii Zhuravlev <[hidden email]> wrote:
Hi,

Do you use only one ODBC client? Can you start one more ODBC client and check the performance?

Thanks,
Evgenii

сб, 2 нояб. 2019 г. в 16:47, Siew Wai Yow <[hidden email]>:
Hi,

We are doing POC on ignite performance using ODBC driver but the performance is always capped at around 1700 TPS which is too slow. It is local ignite service. All tuning tips from Ignite page has been applied, no bottleneck from CPU and Memory. At the moment not turn on persistence yet, it will be worse if turn on. This POC is very crucial to our product roadmap. 

Any tips? Thank you.

Test case,
1 x insert --> 49 x select -->49 x update --> 1 x delete
repeat for 50000 times.

    const char* insert_sql = "insert into CDRTEST values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
    const char* update_sql = "update CDRTEST set value01=?,value02=?,value03=?,value04=?,value05=?,value21=?,value22=?,value23=?,value24=?,value25=? where key01=?";
    const char* delete_sql = "delete from CDRTEST where key01=?";
    const char* select_sql = "select value01 from CDRTEST where key01=?";

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE TABLE IF NOT EXISTS CDRTEST ( "
  "key01 VARCHAR PRIMARY KEY, "
  "value01 LONG, "
  "value02 LONG, "
  "value03 LONG, "
  "value04 LONG, "
  "value05 LONG, "
  "value06 LONG, "
  "value07 LONG, "
  "value08 LONG, "
  "value09 LONG, "
  "value10 LONG, "
  "value11 LONG, "
  "value12 LONG, "
  "value13 LONG, "
  "value14 LONG, "
  "value15 LONG, "
  "value16 LONG, "
  "value17 LONG, "
  "value18 LONG, "
  "value19 LONG, "
  "value20 LONG, "  
  "value21 VARCHAR, "
  "value22 VARCHAR, "
  "value23 VARCHAR, "
  "value24 VARCHAR, "
  "value25 VARCHAR, "
  "value26 VARCHAR, "
  "value27 VARCHAR, "
  "value28 VARCHAR, "
  "value29 VARCHAR, "
  "value30 VARCHAR, "    
  "value31 VARCHAR, "
  "value32 VARCHAR, "
  "value33 VARCHAR, "
  "value34 VARCHAR, "
  "value35 VARCHAR, "
  "value36 VARCHAR, "
  "value37 VARCHAR, "
  "value38 VARCHAR, "
  "value39 VARCHAR) "
  "WITH \"template=partitioned,atomicity=TRANSACTIONAL_SNAPSHOT,WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC\"")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create table", hstmt, SQL_HANDLE_STMT);

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE INDEX key01t_idx ON CDRTEST(key01) INLINE_SIZE 64")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create index", hstmt, SQL_HANDLE_STMT);


Below are configuration we used,

<?xml version="1.0" encoding="UTF-8"?>

       xsi:schemaLocation="http://www.springframework.org/schema/beans

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="binaryConfiguration">
            <bean class="org.apache.ignite.configuration.BinaryConfiguration">
                <property name="compactFooter" value="false"/>

                <property name="idMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                        <property name="lowerCase" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>

  <property name="clientConnectorConfiguration">
    <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration">
      <!--property name="host" value="127.0.0.1"/-->
      <!--property name="port" value="10800"/-->
      <!--property name="portRange" value="5"/-->
      <property name="maxOpenCursorsPerConnection" value="512"/>
      <property name="socketSendBufferSize" value="65536"/>
      <property name="socketReceiveBufferSize" value="131072"/>
      <!--property name="threadPoolSize" value="4"/-->
    </bean>
  </property>

        <property name="dataStorageConfiguration">
           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
              <!-- Enabling Apache Ignite Persistent Store. -->
              <property name="defaultDataRegionConfiguration">
                 <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                     <property name="persistenceEnabled" value="false"/>
                 </bean>
              </property>
              <property name="walPath" value="/ignite-wal"/>

              <property name="walArchivePath" value="/ignite-wal"/>
           </bean>
        </property>

    </bean>
</beans>

Regards,
Yow

odbcsample.cc (20K) Download Attachment
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Ignite singel instance

Hello!

Have you tried collecting thread dumps from node under load, to identify any bottlenecks?

What is the CPU usage by Ignite?

Regards,
--
Ilya Kasnacheev


ср, 6 нояб. 2019 г. в 09:08, Siew Wai Yow <[hidden email]>:
Hi Stan

The result will be the same without transaction, even with atomic mode. So wondering if there is other reason capped the performance. Attach is our POC source code which give around 1700 TPS, with or without transaction, may be you can advise what's wrong with it.

-DIGNITE_MAX_INDEX_PAYLOAD_SIZE=64 is using already.

We choose ODBC because it is more compatible to existing product's API, which will need to create table/cache on the fly without pre-define C-object. Ignite C++ need to have the cache C-object ready before run.

Also in Ignite cluster, how can we connect to cluster, just connect to one of the host? And when host crash we need to change manually connect to another node? Seems ODBC connection string not allow to input multiple host address.

Thanks in advance.

Regards,
Yow



From: Stanislav Lukyanov <[hidden email]>
Sent: Wednesday, November 6, 2019 12:33 AM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
First, 1700 TPS given your transaction structure is 170000 simple operations per second, which is quite substantial - especially if you're doing that from a single thread / single ODBC client.

Second, note that TRANSACTIONAL_SNAPSHOT is in beta and is not ready for production use. There are no claims about performance or stability of that cache mode.

Third, you don't need the index on key01 - it is created automatically because it is a primary key. But you do need to set INLINE SIZE of the default index. Run your Ignite server with system property or environment varialy IGNITE_MAX_INDEX_PAYLOAD_SIZE=64.

Finally, the operations you're doing don't look like something to be done with SQL. Consider using key-value API in Ignite C++ instead - https://apacheignite-cpp.readme.io/docs.

Stan

On Sat, Nov 2, 2019 at 8:15 PM Evgenii Zhuravlev <[hidden email]> wrote:
Hi,

Do you use only one ODBC client? Can you start one more ODBC client and check the performance?

Thanks,
Evgenii

сб, 2 нояб. 2019 г. в 16:47, Siew Wai Yow <[hidden email]>:
Hi,

We are doing POC on ignite performance using ODBC driver but the performance is always capped at around 1700 TPS which is too slow. It is local ignite service. All tuning tips from Ignite page has been applied, no bottleneck from CPU and Memory. At the moment not turn on persistence yet, it will be worse if turn on. This POC is very crucial to our product roadmap. 

Any tips? Thank you.

Test case,
1 x insert --> 49 x select -->49 x update --> 1 x delete
repeat for 50000 times.

    const char* insert_sql = "insert into CDRTEST values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
    const char* update_sql = "update CDRTEST set value01=?,value02=?,value03=?,value04=?,value05=?,value21=?,value22=?,value23=?,value24=?,value25=? where key01=?";
    const char* delete_sql = "delete from CDRTEST where key01=?";
    const char* select_sql = "select value01 from CDRTEST where key01=?";

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE TABLE IF NOT EXISTS CDRTEST ( "
  "key01 VARCHAR PRIMARY KEY, "
  "value01 LONG, "
  "value02 LONG, "
  "value03 LONG, "
  "value04 LONG, "
  "value05 LONG, "
  "value06 LONG, "
  "value07 LONG, "
  "value08 LONG, "
  "value09 LONG, "
  "value10 LONG, "
  "value11 LONG, "
  "value12 LONG, "
  "value13 LONG, "
  "value14 LONG, "
  "value15 LONG, "
  "value16 LONG, "
  "value17 LONG, "
  "value18 LONG, "
  "value19 LONG, "
  "value20 LONG, "  
  "value21 VARCHAR, "
  "value22 VARCHAR, "
  "value23 VARCHAR, "
  "value24 VARCHAR, "
  "value25 VARCHAR, "
  "value26 VARCHAR, "
  "value27 VARCHAR, "
  "value28 VARCHAR, "
  "value29 VARCHAR, "
  "value30 VARCHAR, "    
  "value31 VARCHAR, "
  "value32 VARCHAR, "
  "value33 VARCHAR, "
  "value34 VARCHAR, "
  "value35 VARCHAR, "
  "value36 VARCHAR, "
  "value37 VARCHAR, "
  "value38 VARCHAR, "
  "value39 VARCHAR) "
  "WITH \"template=partitioned,atomicity=TRANSACTIONAL_SNAPSHOT,WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC\"")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create table", hstmt, SQL_HANDLE_STMT);

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE INDEX key01t_idx ON CDRTEST(key01) INLINE_SIZE 64")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create index", hstmt, SQL_HANDLE_STMT);


Below are configuration we used,

<?xml version="1.0" encoding="UTF-8"?>

       xsi:schemaLocation="http://www.springframework.org/schema/beans

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="binaryConfiguration">
            <bean class="org.apache.ignite.configuration.BinaryConfiguration">
                <property name="compactFooter" value="false"/>

                <property name="idMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                        <property name="lowerCase" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>

  <property name="clientConnectorConfiguration">
    <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration">
      <!--property name="host" value="127.0.0.1"/-->
      <!--property name="port" value="10800"/-->
      <!--property name="portRange" value="5"/-->
      <property name="maxOpenCursorsPerConnection" value="512"/>
      <property name="socketSendBufferSize" value="65536"/>
      <property name="socketReceiveBufferSize" value="131072"/>
      <!--property name="threadPoolSize" value="4"/-->
    </bean>
  </property>

        <property name="dataStorageConfiguration">
           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
              <!-- Enabling Apache Ignite Persistent Store. -->
              <property name="defaultDataRegionConfiguration">
                 <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                     <property name="persistenceEnabled" value="false"/>
                 </bean>
              </property>
              <property name="walPath" value="/ignite-wal"/>

              <property name="walArchivePath" value="/ignite-wal"/>
           </bean>
        </property>

    </bean>
</beans>

Regards,
Yow
Siew Wai Yow Siew Wai Yow
Reply | Threaded
Open this post in threaded view
|

Re: Ignite singel instance

Hi llya!

CPU usage is very low. I just did another run as shown, but IO is happening in a spiky pattern, any clue? 
RAM and disk is fine. Only Ignite is running in the host. Thanks.







Regards,
Yow


From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 10:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Have you tried collecting thread dumps from node under load, to identify any bottlenecks?

What is the CPU usage by Ignite?

Regards,
--
Ilya Kasnacheev


ср, 6 нояб. 2019 г. в 09:08, Siew Wai Yow <[hidden email]>:
Hi Stan

The result will be the same without transaction, even with atomic mode. So wondering if there is other reason capped the performance. Attach is our POC source code which give around 1700 TPS, with or without transaction, may be you can advise what's wrong with it.

-DIGNITE_MAX_INDEX_PAYLOAD_SIZE=64 is using already.

We choose ODBC because it is more compatible to existing product's API, which will need to create table/cache on the fly without pre-define C-object. Ignite C++ need to have the cache C-object ready before run.

Also in Ignite cluster, how can we connect to cluster, just connect to one of the host? And when host crash we need to change manually connect to another node? Seems ODBC connection string not allow to input multiple host address.

Thanks in advance.

Regards,
Yow



From: Stanislav Lukyanov <[hidden email]>
Sent: Wednesday, November 6, 2019 12:33 AM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
First, 1700 TPS given your transaction structure is 170000 simple operations per second, which is quite substantial - especially if you're doing that from a single thread / single ODBC client.

Second, note that TRANSACTIONAL_SNAPSHOT is in beta and is not ready for production use. There are no claims about performance or stability of that cache mode.

Third, you don't need the index on key01 - it is created automatically because it is a primary key. But you do need to set INLINE SIZE of the default index. Run your Ignite server with system property or environment varialy IGNITE_MAX_INDEX_PAYLOAD_SIZE=64.

Finally, the operations you're doing don't look like something to be done with SQL. Consider using key-value API in Ignite C++ instead - https://apacheignite-cpp.readme.io/docs.

Stan

On Sat, Nov 2, 2019 at 8:15 PM Evgenii Zhuravlev <[hidden email]> wrote:
Hi,

Do you use only one ODBC client? Can you start one more ODBC client and check the performance?

Thanks,
Evgenii

сб, 2 нояб. 2019 г. в 16:47, Siew Wai Yow <[hidden email]>:
Hi,

We are doing POC on ignite performance using ODBC driver but the performance is always capped at around 1700 TPS which is too slow. It is local ignite service. All tuning tips from Ignite page has been applied, no bottleneck from CPU and Memory. At the moment not turn on persistence yet, it will be worse if turn on. This POC is very crucial to our product roadmap. 

Any tips? Thank you.

Test case,
1 x insert --> 49 x select -->49 x update --> 1 x delete
repeat for 50000 times.

    const char* insert_sql = "insert into CDRTEST values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
    const char* update_sql = "update CDRTEST set value01=?,value02=?,value03=?,value04=?,value05=?,value21=?,value22=?,value23=?,value24=?,value25=? where key01=?";
    const char* delete_sql = "delete from CDRTEST where key01=?";
    const char* select_sql = "select value01 from CDRTEST where key01=?";

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE TABLE IF NOT EXISTS CDRTEST ( "
  "key01 VARCHAR PRIMARY KEY, "
  "value01 LONG, "
  "value02 LONG, "
  "value03 LONG, "
  "value04 LONG, "
  "value05 LONG, "
  "value06 LONG, "
  "value07 LONG, "
  "value08 LONG, "
  "value09 LONG, "
  "value10 LONG, "
  "value11 LONG, "
  "value12 LONG, "
  "value13 LONG, "
  "value14 LONG, "
  "value15 LONG, "
  "value16 LONG, "
  "value17 LONG, "
  "value18 LONG, "
  "value19 LONG, "
  "value20 LONG, "  
  "value21 VARCHAR, "
  "value22 VARCHAR, "
  "value23 VARCHAR, "
  "value24 VARCHAR, "
  "value25 VARCHAR, "
  "value26 VARCHAR, "
  "value27 VARCHAR, "
  "value28 VARCHAR, "
  "value29 VARCHAR, "
  "value30 VARCHAR, "    
  "value31 VARCHAR, "
  "value32 VARCHAR, "
  "value33 VARCHAR, "
  "value34 VARCHAR, "
  "value35 VARCHAR, "
  "value36 VARCHAR, "
  "value37 VARCHAR, "
  "value38 VARCHAR, "
  "value39 VARCHAR) "
  "WITH \"template=partitioned,atomicity=TRANSACTIONAL_SNAPSHOT,WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC\"")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create table", hstmt, SQL_HANDLE_STMT);

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE INDEX key01t_idx ON CDRTEST(key01) INLINE_SIZE 64")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create index", hstmt, SQL_HANDLE_STMT);


Below are configuration we used,

<?xml version="1.0" encoding="UTF-8"?>

       xsi:schemaLocation="http://www.springframework.org/schema/beans

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="binaryConfiguration">
            <bean class="org.apache.ignite.configuration.BinaryConfiguration">
                <property name="compactFooter" value="false"/>

                <property name="idMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                        <property name="lowerCase" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>

  <property name="clientConnectorConfiguration">
    <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration">
      <!--property name="host" value="127.0.0.1"/-->
      <!--property name="port" value="10800"/-->
      <!--property name="portRange" value="5"/-->
      <property name="maxOpenCursorsPerConnection" value="512"/>
      <property name="socketSendBufferSize" value="65536"/>
      <property name="socketReceiveBufferSize" value="131072"/>
      <!--property name="threadPoolSize" value="4"/-->
    </bean>
  </property>

        <property name="dataStorageConfiguration">
           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
              <!-- Enabling Apache Ignite Persistent Store. -->
              <property name="defaultDataRegionConfiguration">
                 <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                     <property name="persistenceEnabled" value="false"/>
                 </bean>
              </property>
              <property name="walPath" value="/ignite-wal"/>

              <property name="walArchivePath" value="/ignite-wal"/>
           </bean>
        </property>

    </bean>
</beans>

Regards,
Yow
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Ignite singel instance

Hello!

Is it possible that you ended up saturating the network? Please also consider profiling for bottlenecks as I have suggested.

Regards,
--
Ilya Kasnacheev


чт, 7 нояб. 2019 г. в 17:55, Siew Wai Yow <[hidden email]>:
Hi llya!

CPU usage is very low. I just did another run as shown, but IO is happening in a spiky pattern, any clue? 
RAM and disk is fine. Only Ignite is running in the host. Thanks.







Regards,
Yow


From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 10:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Have you tried collecting thread dumps from node under load, to identify any bottlenecks?

What is the CPU usage by Ignite?

Regards,
--
Ilya Kasnacheev


ср, 6 нояб. 2019 г. в 09:08, Siew Wai Yow <[hidden email]>:
Hi Stan

The result will be the same without transaction, even with atomic mode. So wondering if there is other reason capped the performance. Attach is our POC source code which give around 1700 TPS, with or without transaction, may be you can advise what's wrong with it.

-DIGNITE_MAX_INDEX_PAYLOAD_SIZE=64 is using already.

We choose ODBC because it is more compatible to existing product's API, which will need to create table/cache on the fly without pre-define C-object. Ignite C++ need to have the cache C-object ready before run.

Also in Ignite cluster, how can we connect to cluster, just connect to one of the host? And when host crash we need to change manually connect to another node? Seems ODBC connection string not allow to input multiple host address.

Thanks in advance.

Regards,
Yow



From: Stanislav Lukyanov <[hidden email]>
Sent: Wednesday, November 6, 2019 12:33 AM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
First, 1700 TPS given your transaction structure is 170000 simple operations per second, which is quite substantial - especially if you're doing that from a single thread / single ODBC client.

Second, note that TRANSACTIONAL_SNAPSHOT is in beta and is not ready for production use. There are no claims about performance or stability of that cache mode.

Third, you don't need the index on key01 - it is created automatically because it is a primary key. But you do need to set INLINE SIZE of the default index. Run your Ignite server with system property or environment varialy IGNITE_MAX_INDEX_PAYLOAD_SIZE=64.

Finally, the operations you're doing don't look like something to be done with SQL. Consider using key-value API in Ignite C++ instead - https://apacheignite-cpp.readme.io/docs.

Stan

On Sat, Nov 2, 2019 at 8:15 PM Evgenii Zhuravlev <[hidden email]> wrote:
Hi,

Do you use only one ODBC client? Can you start one more ODBC client and check the performance?

Thanks,
Evgenii

сб, 2 нояб. 2019 г. в 16:47, Siew Wai Yow <[hidden email]>:
Hi,

We are doing POC on ignite performance using ODBC driver but the performance is always capped at around 1700 TPS which is too slow. It is local ignite service. All tuning tips from Ignite page has been applied, no bottleneck from CPU and Memory. At the moment not turn on persistence yet, it will be worse if turn on. This POC is very crucial to our product roadmap. 

Any tips? Thank you.

Test case,
1 x insert --> 49 x select -->49 x update --> 1 x delete
repeat for 50000 times.

    const char* insert_sql = "insert into CDRTEST values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
    const char* update_sql = "update CDRTEST set value01=?,value02=?,value03=?,value04=?,value05=?,value21=?,value22=?,value23=?,value24=?,value25=? where key01=?";
    const char* delete_sql = "delete from CDRTEST where key01=?";
    const char* select_sql = "select value01 from CDRTEST where key01=?";

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE TABLE IF NOT EXISTS CDRTEST ( "
  "key01 VARCHAR PRIMARY KEY, "
  "value01 LONG, "
  "value02 LONG, "
  "value03 LONG, "
  "value04 LONG, "
  "value05 LONG, "
  "value06 LONG, "
  "value07 LONG, "
  "value08 LONG, "
  "value09 LONG, "
  "value10 LONG, "
  "value11 LONG, "
  "value12 LONG, "
  "value13 LONG, "
  "value14 LONG, "
  "value15 LONG, "
  "value16 LONG, "
  "value17 LONG, "
  "value18 LONG, "
  "value19 LONG, "
  "value20 LONG, "  
  "value21 VARCHAR, "
  "value22 VARCHAR, "
  "value23 VARCHAR, "
  "value24 VARCHAR, "
  "value25 VARCHAR, "
  "value26 VARCHAR, "
  "value27 VARCHAR, "
  "value28 VARCHAR, "
  "value29 VARCHAR, "
  "value30 VARCHAR, "    
  "value31 VARCHAR, "
  "value32 VARCHAR, "
  "value33 VARCHAR, "
  "value34 VARCHAR, "
  "value35 VARCHAR, "
  "value36 VARCHAR, "
  "value37 VARCHAR, "
  "value38 VARCHAR, "
  "value39 VARCHAR) "
  "WITH \"template=partitioned,atomicity=TRANSACTIONAL_SNAPSHOT,WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC\"")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create table", hstmt, SQL_HANDLE_STMT);

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE INDEX key01t_idx ON CDRTEST(key01) INLINE_SIZE 64")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create index", hstmt, SQL_HANDLE_STMT);


Below are configuration we used,

<?xml version="1.0" encoding="UTF-8"?>

       xsi:schemaLocation="http://www.springframework.org/schema/beans

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="binaryConfiguration">
            <bean class="org.apache.ignite.configuration.BinaryConfiguration">
                <property name="compactFooter" value="false"/>

                <property name="idMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                        <property name="lowerCase" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>

  <property name="clientConnectorConfiguration">
    <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration">
      <!--property name="host" value="127.0.0.1"/-->
      <!--property name="port" value="10800"/-->
      <!--property name="portRange" value="5"/-->
      <property name="maxOpenCursorsPerConnection" value="512"/>
      <property name="socketSendBufferSize" value="65536"/>
      <property name="socketReceiveBufferSize" value="131072"/>
      <!--property name="threadPoolSize" value="4"/-->
    </bean>
  </property>

        <property name="dataStorageConfiguration">
           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
              <!-- Enabling Apache Ignite Persistent Store. -->
              <property name="defaultDataRegionConfiguration">
                 <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                     <property name="persistenceEnabled" value="false"/>
                 </bean>
              </property>
              <property name="walPath" value="/ignite-wal"/>

              <property name="walArchivePath" value="/ignite-wal"/>
           </bean>
        </property>

    </bean>
</beans>

Regards,
Yow
Siew Wai Yow Siew Wai Yow
Reply | Threaded
Open this post in threaded view
|

Re: Ignite singel instance

Hi!

NetworkIO is just around 1.2MB/s as shown so that is not a problem. We have profiled and did not found any bottleneck from the environment/resources. We also check the application profiling and most of the time consumed by sql execution which looks normal to me. Any advice?



From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 11:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Is it possible that you ended up saturating the network? Please also consider profiling for bottlenecks as I have suggested.

Regards,
--
Ilya Kasnacheev


чт, 7 нояб. 2019 г. в 17:55, Siew Wai Yow <[hidden email]>:
Hi llya!

CPU usage is very low. I just did another run as shown, but IO is happening in a spiky pattern, any clue? 
RAM and disk is fine. Only Ignite is running in the host. Thanks.







Regards,
Yow


From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 10:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Have you tried collecting thread dumps from node under load, to identify any bottlenecks?

What is the CPU usage by Ignite?

Regards,
--
Ilya Kasnacheev


ср, 6 нояб. 2019 г. в 09:08, Siew Wai Yow <[hidden email]>:
Hi Stan

The result will be the same without transaction, even with atomic mode. So wondering if there is other reason capped the performance. Attach is our POC source code which give around 1700 TPS, with or without transaction, may be you can advise what's wrong with it.

-DIGNITE_MAX_INDEX_PAYLOAD_SIZE=64 is using already.

We choose ODBC because it is more compatible to existing product's API, which will need to create table/cache on the fly without pre-define C-object. Ignite C++ need to have the cache C-object ready before run.

Also in Ignite cluster, how can we connect to cluster, just connect to one of the host? And when host crash we need to change manually connect to another node? Seems ODBC connection string not allow to input multiple host address.

Thanks in advance.

Regards,
Yow



From: Stanislav Lukyanov <[hidden email]>
Sent: Wednesday, November 6, 2019 12:33 AM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
First, 1700 TPS given your transaction structure is 170000 simple operations per second, which is quite substantial - especially if you're doing that from a single thread / single ODBC client.

Second, note that TRANSACTIONAL_SNAPSHOT is in beta and is not ready for production use. There are no claims about performance or stability of that cache mode.

Third, you don't need the index on key01 - it is created automatically because it is a primary key. But you do need to set INLINE SIZE of the default index. Run your Ignite server with system property or environment varialy IGNITE_MAX_INDEX_PAYLOAD_SIZE=64.

Finally, the operations you're doing don't look like something to be done with SQL. Consider using key-value API in Ignite C++ instead - https://apacheignite-cpp.readme.io/docs.

Stan

On Sat, Nov 2, 2019 at 8:15 PM Evgenii Zhuravlev <[hidden email]> wrote:
Hi,

Do you use only one ODBC client? Can you start one more ODBC client and check the performance?

Thanks,
Evgenii

сб, 2 нояб. 2019 г. в 16:47, Siew Wai Yow <[hidden email]>:
Hi,

We are doing POC on ignite performance using ODBC driver but the performance is always capped at around 1700 TPS which is too slow. It is local ignite service. All tuning tips from Ignite page has been applied, no bottleneck from CPU and Memory. At the moment not turn on persistence yet, it will be worse if turn on. This POC is very crucial to our product roadmap. 

Any tips? Thank you.

Test case,
1 x insert --> 49 x select -->49 x update --> 1 x delete
repeat for 50000 times.

    const char* insert_sql = "insert into CDRTEST values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
    const char* update_sql = "update CDRTEST set value01=?,value02=?,value03=?,value04=?,value05=?,value21=?,value22=?,value23=?,value24=?,value25=? where key01=?";
    const char* delete_sql = "delete from CDRTEST where key01=?";
    const char* select_sql = "select value01 from CDRTEST where key01=?";

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE TABLE IF NOT EXISTS CDRTEST ( "
  "key01 VARCHAR PRIMARY KEY, "
  "value01 LONG, "
  "value02 LONG, "
  "value03 LONG, "
  "value04 LONG, "
  "value05 LONG, "
  "value06 LONG, "
  "value07 LONG, "
  "value08 LONG, "
  "value09 LONG, "
  "value10 LONG, "
  "value11 LONG, "
  "value12 LONG, "
  "value13 LONG, "
  "value14 LONG, "
  "value15 LONG, "
  "value16 LONG, "
  "value17 LONG, "
  "value18 LONG, "
  "value19 LONG, "
  "value20 LONG, "  
  "value21 VARCHAR, "
  "value22 VARCHAR, "
  "value23 VARCHAR, "
  "value24 VARCHAR, "
  "value25 VARCHAR, "
  "value26 VARCHAR, "
  "value27 VARCHAR, "
  "value28 VARCHAR, "
  "value29 VARCHAR, "
  "value30 VARCHAR, "    
  "value31 VARCHAR, "
  "value32 VARCHAR, "
  "value33 VARCHAR, "
  "value34 VARCHAR, "
  "value35 VARCHAR, "
  "value36 VARCHAR, "
  "value37 VARCHAR, "
  "value38 VARCHAR, "
  "value39 VARCHAR) "
  "WITH \"template=partitioned,atomicity=TRANSACTIONAL_SNAPSHOT,WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC\"")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create table", hstmt, SQL_HANDLE_STMT);

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE INDEX key01t_idx ON CDRTEST(key01) INLINE_SIZE 64")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create index", hstmt, SQL_HANDLE_STMT);


Below are configuration we used,

<?xml version="1.0" encoding="UTF-8"?>

       xsi:schemaLocation="http://www.springframework.org/schema/beans

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="binaryConfiguration">
            <bean class="org.apache.ignite.configuration.BinaryConfiguration">
                <property name="compactFooter" value="false"/>

                <property name="idMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                        <property name="lowerCase" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>

  <property name="clientConnectorConfiguration">
    <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration">
      <!--property name="host" value="127.0.0.1"/-->
      <!--property name="port" value="10800"/-->
      <!--property name="portRange" value="5"/-->
      <property name="maxOpenCursorsPerConnection" value="512"/>
      <property name="socketSendBufferSize" value="65536"/>
      <property name="socketReceiveBufferSize" value="131072"/>
      <!--property name="threadPoolSize" value="4"/-->
    </bean>
  </property>

        <property name="dataStorageConfiguration">
           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
              <!-- Enabling Apache Ignite Persistent Store. -->
              <property name="defaultDataRegionConfiguration">
                 <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                     <property name="persistenceEnabled" value="false"/>
                 </bean>
              </property>
              <property name="walPath" value="/ignite-wal"/>

              <property name="walArchivePath" value="/ignite-wal"/>
           </bean>
        </property>

    </bean>
</beans>

Regards,
Yow
Ivan Pavlukhin Ivan Pavlukhin
Reply | Threaded
Open this post in threaded view
|

Re: Ignite singel instance

Hi Yow,

Out of curiosity. What TPS will be good for your cases?

пт, 8 нояб. 2019 г. в 03:34, Siew Wai Yow <[hidden email]>:
Hi!

NetworkIO is just around 1.2MB/s as shown so that is not a problem. We have profiled and did not found any bottleneck from the environment/resources. We also check the application profiling and most of the time consumed by sql execution which looks normal to me. Any advice?



From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 11:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Is it possible that you ended up saturating the network? Please also consider profiling for bottlenecks as I have suggested.

Regards,
--
Ilya Kasnacheev


чт, 7 нояб. 2019 г. в 17:55, Siew Wai Yow <[hidden email]>:
Hi llya!

CPU usage is very low. I just did another run as shown, but IO is happening in a spiky pattern, any clue? 
RAM and disk is fine. Only Ignite is running in the host. Thanks.







Regards,
Yow


From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 10:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Have you tried collecting thread dumps from node under load, to identify any bottlenecks?

What is the CPU usage by Ignite?

Regards,
--
Ilya Kasnacheev


ср, 6 нояб. 2019 г. в 09:08, Siew Wai Yow <[hidden email]>:
Hi Stan

The result will be the same without transaction, even with atomic mode. So wondering if there is other reason capped the performance. Attach is our POC source code which give around 1700 TPS, with or without transaction, may be you can advise what's wrong with it.

-DIGNITE_MAX_INDEX_PAYLOAD_SIZE=64 is using already.

We choose ODBC because it is more compatible to existing product's API, which will need to create table/cache on the fly without pre-define C-object. Ignite C++ need to have the cache C-object ready before run.

Also in Ignite cluster, how can we connect to cluster, just connect to one of the host? And when host crash we need to change manually connect to another node? Seems ODBC connection string not allow to input multiple host address.

Thanks in advance.

Regards,
Yow



From: Stanislav Lukyanov <[hidden email]>
Sent: Wednesday, November 6, 2019 12:33 AM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
First, 1700 TPS given your transaction structure is 170000 simple operations per second, which is quite substantial - especially if you're doing that from a single thread / single ODBC client.

Second, note that TRANSACTIONAL_SNAPSHOT is in beta and is not ready for production use. There are no claims about performance or stability of that cache mode.

Third, you don't need the index on key01 - it is created automatically because it is a primary key. But you do need to set INLINE SIZE of the default index. Run your Ignite server with system property or environment varialy IGNITE_MAX_INDEX_PAYLOAD_SIZE=64.

Finally, the operations you're doing don't look like something to be done with SQL. Consider using key-value API in Ignite C++ instead - https://apacheignite-cpp.readme.io/docs.

Stan

On Sat, Nov 2, 2019 at 8:15 PM Evgenii Zhuravlev <[hidden email]> wrote:
Hi,

Do you use only one ODBC client? Can you start one more ODBC client and check the performance?

Thanks,
Evgenii

сб, 2 нояб. 2019 г. в 16:47, Siew Wai Yow <[hidden email]>:
Hi,

We are doing POC on ignite performance using ODBC driver but the performance is always capped at around 1700 TPS which is too slow. It is local ignite service. All tuning tips from Ignite page has been applied, no bottleneck from CPU and Memory. At the moment not turn on persistence yet, it will be worse if turn on. This POC is very crucial to our product roadmap. 

Any tips? Thank you.

Test case,
1 x insert --> 49 x select -->49 x update --> 1 x delete
repeat for 50000 times.

    const char* insert_sql = "insert into CDRTEST values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
    const char* update_sql = "update CDRTEST set value01=?,value02=?,value03=?,value04=?,value05=?,value21=?,value22=?,value23=?,value24=?,value25=? where key01=?";
    const char* delete_sql = "delete from CDRTEST where key01=?";
    const char* select_sql = "select value01 from CDRTEST where key01=?";

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE TABLE IF NOT EXISTS CDRTEST ( "
  "key01 VARCHAR PRIMARY KEY, "
  "value01 LONG, "
  "value02 LONG, "
  "value03 LONG, "
  "value04 LONG, "
  "value05 LONG, "
  "value06 LONG, "
  "value07 LONG, "
  "value08 LONG, "
  "value09 LONG, "
  "value10 LONG, "
  "value11 LONG, "
  "value12 LONG, "
  "value13 LONG, "
  "value14 LONG, "
  "value15 LONG, "
  "value16 LONG, "
  "value17 LONG, "
  "value18 LONG, "
  "value19 LONG, "
  "value20 LONG, "  
  "value21 VARCHAR, "
  "value22 VARCHAR, "
  "value23 VARCHAR, "
  "value24 VARCHAR, "
  "value25 VARCHAR, "
  "value26 VARCHAR, "
  "value27 VARCHAR, "
  "value28 VARCHAR, "
  "value29 VARCHAR, "
  "value30 VARCHAR, "    
  "value31 VARCHAR, "
  "value32 VARCHAR, "
  "value33 VARCHAR, "
  "value34 VARCHAR, "
  "value35 VARCHAR, "
  "value36 VARCHAR, "
  "value37 VARCHAR, "
  "value38 VARCHAR, "
  "value39 VARCHAR) "
  "WITH \"template=partitioned,atomicity=TRANSACTIONAL_SNAPSHOT,WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC\"")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create table", hstmt, SQL_HANDLE_STMT);

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE INDEX key01t_idx ON CDRTEST(key01) INLINE_SIZE 64")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create index", hstmt, SQL_HANDLE_STMT);


Below are configuration we used,

<?xml version="1.0" encoding="UTF-8"?>

       xsi:schemaLocation="http://www.springframework.org/schema/beans

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="binaryConfiguration">
            <bean class="org.apache.ignite.configuration.BinaryConfiguration">
                <property name="compactFooter" value="false"/>

                <property name="idMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                        <property name="lowerCase" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>

  <property name="clientConnectorConfiguration">
    <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration">
      <!--property name="host" value="127.0.0.1"/-->
      <!--property name="port" value="10800"/-->
      <!--property name="portRange" value="5"/-->
      <property name="maxOpenCursorsPerConnection" value="512"/>
      <property name="socketSendBufferSize" value="65536"/>
      <property name="socketReceiveBufferSize" value="131072"/>
      <!--property name="threadPoolSize" value="4"/-->
    </bean>
  </property>

        <property name="dataStorageConfiguration">
           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
              <!-- Enabling Apache Ignite Persistent Store. -->
              <property name="defaultDataRegionConfiguration">
                 <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                     <property name="persistenceEnabled" value="false"/>
                 </bean>
              </property>
              <property name="walPath" value="/ignite-wal"/>

              <property name="walArchivePath" value="/ignite-wal"/>
           </bean>
        </property>

    </bean>
</beans>

Regards,
Yow


--
Best regards,
Ivan Pavlukhin
Siew Wai Yow Siew Wai Yow
Reply | Threaded
Open this post in threaded view
|

Re: Ignite singel instance

Hi Ivan,

5k TPS the minimum and it would be great to have 10k TPS. Do you think this is feasible? Or Ignite expect user to run with multiple instances?

At the moment only 1.7k TPS in local server without persistence. Same test in a 3-nodes-cluster with persistence enabled, the same application run only 400 TPS which is quite disappointing. Any tips? Could it be ODBC driver slowness?

Regards,
Yow


From: Ivan Pavlukhin <[hidden email]>
Sent: Friday, November 8, 2019 4:18 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hi Yow,

Out of curiosity. What TPS will be good for your cases?

пт, 8 нояб. 2019 г. в 03:34, Siew Wai Yow <[hidden email]>:
Hi!

NetworkIO is just around 1.2MB/s as shown so that is not a problem. We have profiled and did not found any bottleneck from the environment/resources. We also check the application profiling and most of the time consumed by sql execution which looks normal to me. Any advice?



From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 11:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Is it possible that you ended up saturating the network? Please also consider profiling for bottlenecks as I have suggested.

Regards,
--
Ilya Kasnacheev


чт, 7 нояб. 2019 г. в 17:55, Siew Wai Yow <[hidden email]>:
Hi llya!

CPU usage is very low. I just did another run as shown, but IO is happening in a spiky pattern, any clue? 
RAM and disk is fine. Only Ignite is running in the host. Thanks.







Regards,
Yow


From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 10:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Have you tried collecting thread dumps from node under load, to identify any bottlenecks?

What is the CPU usage by Ignite?

Regards,
--
Ilya Kasnacheev


ср, 6 нояб. 2019 г. в 09:08, Siew Wai Yow <[hidden email]>:
Hi Stan

The result will be the same without transaction, even with atomic mode. So wondering if there is other reason capped the performance. Attach is our POC source code which give around 1700 TPS, with or without transaction, may be you can advise what's wrong with it.

-DIGNITE_MAX_INDEX_PAYLOAD_SIZE=64 is using already.

We choose ODBC because it is more compatible to existing product's API, which will need to create table/cache on the fly without pre-define C-object. Ignite C++ need to have the cache C-object ready before run.

Also in Ignite cluster, how can we connect to cluster, just connect to one of the host? And when host crash we need to change manually connect to another node? Seems ODBC connection string not allow to input multiple host address.

Thanks in advance.

Regards,
Yow



From: Stanislav Lukyanov <[hidden email]>
Sent: Wednesday, November 6, 2019 12:33 AM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
First, 1700 TPS given your transaction structure is 170000 simple operations per second, which is quite substantial - especially if you're doing that from a single thread / single ODBC client.

Second, note that TRANSACTIONAL_SNAPSHOT is in beta and is not ready for production use. There are no claims about performance or stability of that cache mode.

Third, you don't need the index on key01 - it is created automatically because it is a primary key. But you do need to set INLINE SIZE of the default index. Run your Ignite server with system property or environment varialy IGNITE_MAX_INDEX_PAYLOAD_SIZE=64.

Finally, the operations you're doing don't look like something to be done with SQL. Consider using key-value API in Ignite C++ instead - https://apacheignite-cpp.readme.io/docs.

Stan

On Sat, Nov 2, 2019 at 8:15 PM Evgenii Zhuravlev <[hidden email]> wrote:
Hi,

Do you use only one ODBC client? Can you start one more ODBC client and check the performance?

Thanks,
Evgenii

сб, 2 нояб. 2019 г. в 16:47, Siew Wai Yow <[hidden email]>:
Hi,

We are doing POC on ignite performance using ODBC driver but the performance is always capped at around 1700 TPS which is too slow. It is local ignite service. All tuning tips from Ignite page has been applied, no bottleneck from CPU and Memory. At the moment not turn on persistence yet, it will be worse if turn on. This POC is very crucial to our product roadmap. 

Any tips? Thank you.

Test case,
1 x insert --> 49 x select -->49 x update --> 1 x delete
repeat for 50000 times.

    const char* insert_sql = "insert into CDRTEST values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
    const char* update_sql = "update CDRTEST set value01=?,value02=?,value03=?,value04=?,value05=?,value21=?,value22=?,value23=?,value24=?,value25=? where key01=?";
    const char* delete_sql = "delete from CDRTEST where key01=?";
    const char* select_sql = "select value01 from CDRTEST where key01=?";

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE TABLE IF NOT EXISTS CDRTEST ( "
  "key01 VARCHAR PRIMARY KEY, "
  "value01 LONG, "
  "value02 LONG, "
  "value03 LONG, "
  "value04 LONG, "
  "value05 LONG, "
  "value06 LONG, "
  "value07 LONG, "
  "value08 LONG, "
  "value09 LONG, "
  "value10 LONG, "
  "value11 LONG, "
  "value12 LONG, "
  "value13 LONG, "
  "value14 LONG, "
  "value15 LONG, "
  "value16 LONG, "
  "value17 LONG, "
  "value18 LONG, "
  "value19 LONG, "
  "value20 LONG, "  
  "value21 VARCHAR, "
  "value22 VARCHAR, "
  "value23 VARCHAR, "
  "value24 VARCHAR, "
  "value25 VARCHAR, "
  "value26 VARCHAR, "
  "value27 VARCHAR, "
  "value28 VARCHAR, "
  "value29 VARCHAR, "
  "value30 VARCHAR, "    
  "value31 VARCHAR, "
  "value32 VARCHAR, "
  "value33 VARCHAR, "
  "value34 VARCHAR, "
  "value35 VARCHAR, "
  "value36 VARCHAR, "
  "value37 VARCHAR, "
  "value38 VARCHAR, "
  "value39 VARCHAR) "
  "WITH \"template=partitioned,atomicity=TRANSACTIONAL_SNAPSHOT,WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC\"")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create table", hstmt, SQL_HANDLE_STMT);

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE INDEX key01t_idx ON CDRTEST(key01) INLINE_SIZE 64")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create index", hstmt, SQL_HANDLE_STMT);


Below are configuration we used,

<?xml version="1.0" encoding="UTF-8"?>

       xsi:schemaLocation="http://www.springframework.org/schema/beans

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="binaryConfiguration">
            <bean class="org.apache.ignite.configuration.BinaryConfiguration">
                <property name="compactFooter" value="false"/>

                <property name="idMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                        <property name="lowerCase" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>

  <property name="clientConnectorConfiguration">
    <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration">
      <!--property name="host" value="127.0.0.1"/-->
      <!--property name="port" value="10800"/-->
      <!--property name="portRange" value="5"/-->
      <property name="maxOpenCursorsPerConnection" value="512"/>
      <property name="socketSendBufferSize" value="65536"/>
      <property name="socketReceiveBufferSize" value="131072"/>
      <!--property name="threadPoolSize" value="4"/-->
    </bean>
  </property>

        <property name="dataStorageConfiguration">
           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
              <!-- Enabling Apache Ignite Persistent Store. -->
              <property name="defaultDataRegionConfiguration">
                 <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                     <property name="persistenceEnabled" value="false"/>
                 </bean>
              </property>
              <property name="walPath" value="/ignite-wal"/>

              <property name="walArchivePath" value="/ignite-wal"/>
           </bean>
        </property>

    </bean>
</beans>

Regards,
Yow


--
Best regards,
Ivan Pavlukhin
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Ignite singel instance

Hello!

I was meaning that you should benchmark the Java side (Ignite server) and not ODBC client.

What you should be figuring out is which thread pools are busy and what's they up to.

Regards,
--
Ilya Kasnacheev


пт, 8 нояб. 2019 г. в 11:36, Siew Wai Yow <[hidden email]>:
Hi Ivan,

5k TPS the minimum and it would be great to have 10k TPS. Do you think this is feasible? Or Ignite expect user to run with multiple instances?

At the moment only 1.7k TPS in local server without persistence. Same test in a 3-nodes-cluster with persistence enabled, the same application run only 400 TPS which is quite disappointing. Any tips? Could it be ODBC driver slowness?

Regards,
Yow


From: Ivan Pavlukhin <[hidden email]>
Sent: Friday, November 8, 2019 4:18 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hi Yow,

Out of curiosity. What TPS will be good for your cases?

пт, 8 нояб. 2019 г. в 03:34, Siew Wai Yow <[hidden email]>:
Hi!

NetworkIO is just around 1.2MB/s as shown so that is not a problem. We have profiled and did not found any bottleneck from the environment/resources. We also check the application profiling and most of the time consumed by sql execution which looks normal to me. Any advice?



From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 11:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Is it possible that you ended up saturating the network? Please also consider profiling for bottlenecks as I have suggested.

Regards,
--
Ilya Kasnacheev


чт, 7 нояб. 2019 г. в 17:55, Siew Wai Yow <[hidden email]>:
Hi llya!

CPU usage is very low. I just did another run as shown, but IO is happening in a spiky pattern, any clue? 
RAM and disk is fine. Only Ignite is running in the host. Thanks.







Regards,
Yow


From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 10:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Have you tried collecting thread dumps from node under load, to identify any bottlenecks?

What is the CPU usage by Ignite?

Regards,
--
Ilya Kasnacheev


ср, 6 нояб. 2019 г. в 09:08, Siew Wai Yow <[hidden email]>:
Hi Stan

The result will be the same without transaction, even with atomic mode. So wondering if there is other reason capped the performance. Attach is our POC source code which give around 1700 TPS, with or without transaction, may be you can advise what's wrong with it.

-DIGNITE_MAX_INDEX_PAYLOAD_SIZE=64 is using already.

We choose ODBC because it is more compatible to existing product's API, which will need to create table/cache on the fly without pre-define C-object. Ignite C++ need to have the cache C-object ready before run.

Also in Ignite cluster, how can we connect to cluster, just connect to one of the host? And when host crash we need to change manually connect to another node? Seems ODBC connection string not allow to input multiple host address.

Thanks in advance.

Regards,
Yow



From: Stanislav Lukyanov <[hidden email]>
Sent: Wednesday, November 6, 2019 12:33 AM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
First, 1700 TPS given your transaction structure is 170000 simple operations per second, which is quite substantial - especially if you're doing that from a single thread / single ODBC client.

Second, note that TRANSACTIONAL_SNAPSHOT is in beta and is not ready for production use. There are no claims about performance or stability of that cache mode.

Third, you don't need the index on key01 - it is created automatically because it is a primary key. But you do need to set INLINE SIZE of the default index. Run your Ignite server with system property or environment varialy IGNITE_MAX_INDEX_PAYLOAD_SIZE=64.

Finally, the operations you're doing don't look like something to be done with SQL. Consider using key-value API in Ignite C++ instead - https://apacheignite-cpp.readme.io/docs.

Stan

On Sat, Nov 2, 2019 at 8:15 PM Evgenii Zhuravlev <[hidden email]> wrote:
Hi,

Do you use only one ODBC client? Can you start one more ODBC client and check the performance?

Thanks,
Evgenii

сб, 2 нояб. 2019 г. в 16:47, Siew Wai Yow <[hidden email]>:
Hi,

We are doing POC on ignite performance using ODBC driver but the performance is always capped at around 1700 TPS which is too slow. It is local ignite service. All tuning tips from Ignite page has been applied, no bottleneck from CPU and Memory. At the moment not turn on persistence yet, it will be worse if turn on. This POC is very crucial to our product roadmap. 

Any tips? Thank you.

Test case,
1 x insert --> 49 x select -->49 x update --> 1 x delete
repeat for 50000 times.

    const char* insert_sql = "insert into CDRTEST values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
    const char* update_sql = "update CDRTEST set value01=?,value02=?,value03=?,value04=?,value05=?,value21=?,value22=?,value23=?,value24=?,value25=? where key01=?";
    const char* delete_sql = "delete from CDRTEST where key01=?";
    const char* select_sql = "select value01 from CDRTEST where key01=?";

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE TABLE IF NOT EXISTS CDRTEST ( "
  "key01 VARCHAR PRIMARY KEY, "
  "value01 LONG, "
  "value02 LONG, "
  "value03 LONG, "
  "value04 LONG, "
  "value05 LONG, "
  "value06 LONG, "
  "value07 LONG, "
  "value08 LONG, "
  "value09 LONG, "
  "value10 LONG, "
  "value11 LONG, "
  "value12 LONG, "
  "value13 LONG, "
  "value14 LONG, "
  "value15 LONG, "
  "value16 LONG, "
  "value17 LONG, "
  "value18 LONG, "
  "value19 LONG, "
  "value20 LONG, "  
  "value21 VARCHAR, "
  "value22 VARCHAR, "
  "value23 VARCHAR, "
  "value24 VARCHAR, "
  "value25 VARCHAR, "
  "value26 VARCHAR, "
  "value27 VARCHAR, "
  "value28 VARCHAR, "
  "value29 VARCHAR, "
  "value30 VARCHAR, "    
  "value31 VARCHAR, "
  "value32 VARCHAR, "
  "value33 VARCHAR, "
  "value34 VARCHAR, "
  "value35 VARCHAR, "
  "value36 VARCHAR, "
  "value37 VARCHAR, "
  "value38 VARCHAR, "
  "value39 VARCHAR) "
  "WITH \"template=partitioned,atomicity=TRANSACTIONAL_SNAPSHOT,WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC\"")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create table", hstmt, SQL_HANDLE_STMT);

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE INDEX key01t_idx ON CDRTEST(key01) INLINE_SIZE 64")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create index", hstmt, SQL_HANDLE_STMT);


Below are configuration we used,

<?xml version="1.0" encoding="UTF-8"?>

       xsi:schemaLocation="http://www.springframework.org/schema/beans

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="binaryConfiguration">
            <bean class="org.apache.ignite.configuration.BinaryConfiguration">
                <property name="compactFooter" value="false"/>

                <property name="idMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                        <property name="lowerCase" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>

  <property name="clientConnectorConfiguration">
    <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration">
      <!--property name="host" value="127.0.0.1"/-->
      <!--property name="port" value="10800"/-->
      <!--property name="portRange" value="5"/-->
      <property name="maxOpenCursorsPerConnection" value="512"/>
      <property name="socketSendBufferSize" value="65536"/>
      <property name="socketReceiveBufferSize" value="131072"/>
      <!--property name="threadPoolSize" value="4"/-->
    </bean>
  </property>

        <property name="dataStorageConfiguration">
           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
              <!-- Enabling Apache Ignite Persistent Store. -->
              <property name="defaultDataRegionConfiguration">
                 <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                     <property name="persistenceEnabled" value="false"/>
                 </bean>
              </property>
              <property name="walPath" value="/ignite-wal"/>

              <property name="walArchivePath" value="/ignite-wal"/>
           </bean>
        </property>

    </bean>
</beans>

Regards,
Yow


--
Best regards,
Ivan Pavlukhin
Siew Wai Yow Siew Wai Yow
Reply | Threaded
Open this post in threaded view
|

Re: Ignite singel instance

Hi,

Ignite server hotspot during the run, seems like there are locks in Ignite consume most of the time, any idea?



Backtrace,







threadump(please refer to attachment for complete dump),

"client-connector-#157" - Thread t@194
   java.lang.Thread.State: WAITING
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <46ac352c> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None




Thanks!

Regards,
Yow





From: Ilya Kasnacheev <[hidden email]>
Sent: Friday, November 8, 2019 5:21 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

I was meaning that you should benchmark the Java side (Ignite server) and not ODBC client.

What you should be figuring out is which thread pools are busy and what's they up to.

Regards,
--
Ilya Kasnacheev


пт, 8 нояб. 2019 г. в 11:36, Siew Wai Yow <[hidden email]>:
Hi Ivan,

5k TPS the minimum and it would be great to have 10k TPS. Do you think this is feasible? Or Ignite expect user to run with multiple instances?

At the moment only 1.7k TPS in local server without persistence. Same test in a 3-nodes-cluster with persistence enabled, the same application run only 400 TPS which is quite disappointing. Any tips? Could it be ODBC driver slowness?

Regards,
Yow


From: Ivan Pavlukhin <[hidden email]>
Sent: Friday, November 8, 2019 4:18 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hi Yow,

Out of curiosity. What TPS will be good for your cases?

пт, 8 нояб. 2019 г. в 03:34, Siew Wai Yow <[hidden email]>:
Hi!

NetworkIO is just around 1.2MB/s as shown so that is not a problem. We have profiled and did not found any bottleneck from the environment/resources. We also check the application profiling and most of the time consumed by sql execution which looks normal to me. Any advice?



From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 11:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Is it possible that you ended up saturating the network? Please also consider profiling for bottlenecks as I have suggested.

Regards,
--
Ilya Kasnacheev


чт, 7 нояб. 2019 г. в 17:55, Siew Wai Yow <[hidden email]>:
Hi llya!

CPU usage is very low. I just did another run as shown, but IO is happening in a spiky pattern, any clue? 
RAM and disk is fine. Only Ignite is running in the host. Thanks.







Regards,
Yow


From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 10:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Have you tried collecting thread dumps from node under load, to identify any bottlenecks?

What is the CPU usage by Ignite?

Regards,
--
Ilya Kasnacheev


ср, 6 нояб. 2019 г. в 09:08, Siew Wai Yow <[hidden email]>:
Hi Stan

The result will be the same without transaction, even with atomic mode. So wondering if there is other reason capped the performance. Attach is our POC source code which give around 1700 TPS, with or without transaction, may be you can advise what's wrong with it.

-DIGNITE_MAX_INDEX_PAYLOAD_SIZE=64 is using already.

We choose ODBC because it is more compatible to existing product's API, which will need to create table/cache on the fly without pre-define C-object. Ignite C++ need to have the cache C-object ready before run.

Also in Ignite cluster, how can we connect to cluster, just connect to one of the host? And when host crash we need to change manually connect to another node? Seems ODBC connection string not allow to input multiple host address.

Thanks in advance.

Regards,
Yow



From: Stanislav Lukyanov <[hidden email]>
Sent: Wednesday, November 6, 2019 12:33 AM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
First, 1700 TPS given your transaction structure is 170000 simple operations per second, which is quite substantial - especially if you're doing that from a single thread / single ODBC client.

Second, note that TRANSACTIONAL_SNAPSHOT is in beta and is not ready for production use. There are no claims about performance or stability of that cache mode.

Third, you don't need the index on key01 - it is created automatically because it is a primary key. But you do need to set INLINE SIZE of the default index. Run your Ignite server with system property or environment varialy IGNITE_MAX_INDEX_PAYLOAD_SIZE=64.

Finally, the operations you're doing don't look like something to be done with SQL. Consider using key-value API in Ignite C++ instead - https://apacheignite-cpp.readme.io/docs.

Stan

On Sat, Nov 2, 2019 at 8:15 PM Evgenii Zhuravlev <[hidden email]> wrote:
Hi,

Do you use only one ODBC client? Can you start one more ODBC client and check the performance?

Thanks,
Evgenii

сб, 2 нояб. 2019 г. в 16:47, Siew Wai Yow <[hidden email]>:
Hi,

We are doing POC on ignite performance using ODBC driver but the performance is always capped at around 1700 TPS which is too slow. It is local ignite service. All tuning tips from Ignite page has been applied, no bottleneck from CPU and Memory. At the moment not turn on persistence yet, it will be worse if turn on. This POC is very crucial to our product roadmap. 

Any tips? Thank you.

Test case,
1 x insert --> 49 x select -->49 x update --> 1 x delete
repeat for 50000 times.

    const char* insert_sql = "insert into CDRTEST values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
    const char* update_sql = "update CDRTEST set value01=?,value02=?,value03=?,value04=?,value05=?,value21=?,value22=?,value23=?,value24=?,value25=? where key01=?";
    const char* delete_sql = "delete from CDRTEST where key01=?";
    const char* select_sql = "select value01 from CDRTEST where key01=?";

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE TABLE IF NOT EXISTS CDRTEST ( "
  "key01 VARCHAR PRIMARY KEY, "
  "value01 LONG, "
  "value02 LONG, "
  "value03 LONG, "
  "value04 LONG, "
  "value05 LONG, "
  "value06 LONG, "
  "value07 LONG, "
  "value08 LONG, "
  "value09 LONG, "
  "value10 LONG, "
  "value11 LONG, "
  "value12 LONG, "
  "value13 LONG, "
  "value14 LONG, "
  "value15 LONG, "
  "value16 LONG, "
  "value17 LONG, "
  "value18 LONG, "
  "value19 LONG, "
  "value20 LONG, "  
  "value21 VARCHAR, "
  "value22 VARCHAR, "
  "value23 VARCHAR, "
  "value24 VARCHAR, "
  "value25 VARCHAR, "
  "value26 VARCHAR, "
  "value27 VARCHAR, "
  "value28 VARCHAR, "
  "value29 VARCHAR, "
  "value30 VARCHAR, "    
  "value31 VARCHAR, "
  "value32 VARCHAR, "
  "value33 VARCHAR, "
  "value34 VARCHAR, "
  "value35 VARCHAR, "
  "value36 VARCHAR, "
  "value37 VARCHAR, "
  "value38 VARCHAR, "
  "value39 VARCHAR) "
  "WITH \"template=partitioned,atomicity=TRANSACTIONAL_SNAPSHOT,WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC\"")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create table", hstmt, SQL_HANDLE_STMT);

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE INDEX key01t_idx ON CDRTEST(key01) INLINE_SIZE 64")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create index", hstmt, SQL_HANDLE_STMT);


Below are configuration we used,

<?xml version="1.0" encoding="UTF-8"?>

       xsi:schemaLocation="http://www.springframework.org/schema/beans

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="binaryConfiguration">
            <bean class="org.apache.ignite.configuration.BinaryConfiguration">
                <property name="compactFooter" value="false"/>

                <property name="idMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                        <property name="lowerCase" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>

  <property name="clientConnectorConfiguration">
    <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration">
      <!--property name="host" value="127.0.0.1"/-->
      <!--property name="port" value="10800"/-->
      <!--property name="portRange" value="5"/-->
      <property name="maxOpenCursorsPerConnection" value="512"/>
      <property name="socketSendBufferSize" value="65536"/>
      <property name="socketReceiveBufferSize" value="131072"/>
      <!--property name="threadPoolSize" value="4"/-->
    </bean>
  </property>

        <property name="dataStorageConfiguration">
           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
              <!-- Enabling Apache Ignite Persistent Store. -->
              <property name="defaultDataRegionConfiguration">
                 <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                     <property name="persistenceEnabled" value="false"/>
                 </bean>
              </property>
              <property name="walPath" value="/ignite-wal"/>

              <property name="walArchivePath" value="/ignite-wal"/>
           </bean>
        </property>

    </bean>
</beans>

Regards,
Yow


--
Best regards,
Ivan Pavlukhin

threaddump-1573207804944.tdump (162K) Download Attachment
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Ignite singel instance

Hello!

Of course you may expect that every multi-threaded application will be waiting on locks 99% of thread-time. This is why using some kind of smart tooling, such as Java Flight Recorder/VisualVM is recommended.

Here, you can see that you only use a single connector thread. Consider having more than one ODBC connection at the time to increase throughput.

If you had more than one thread dump, maybe there would be some usage patterns to hint at bottlenecks.

Regards,
--
Ilya Kasnacheev


пт, 8 нояб. 2019 г. в 13:21, Siew Wai Yow <[hidden email]>:
Hi,

Ignite server hotspot during the run, seems like there are locks in Ignite consume most of the time, any idea?



Backtrace,







threadump(please refer to attachment for complete dump),

"client-connector-#157" - Thread t@194
   java.lang.Thread.State: WAITING
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <46ac352c> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None




Thanks!

Regards,
Yow





From: Ilya Kasnacheev <[hidden email]>
Sent: Friday, November 8, 2019 5:21 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

I was meaning that you should benchmark the Java side (Ignite server) and not ODBC client.

What you should be figuring out is which thread pools are busy and what's they up to.

Regards,
--
Ilya Kasnacheev


пт, 8 нояб. 2019 г. в 11:36, Siew Wai Yow <[hidden email]>:
Hi Ivan,

5k TPS the minimum and it would be great to have 10k TPS. Do you think this is feasible? Or Ignite expect user to run with multiple instances?

At the moment only 1.7k TPS in local server without persistence. Same test in a 3-nodes-cluster with persistence enabled, the same application run only 400 TPS which is quite disappointing. Any tips? Could it be ODBC driver slowness?

Regards,
Yow


From: Ivan Pavlukhin <[hidden email]>
Sent: Friday, November 8, 2019 4:18 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hi Yow,

Out of curiosity. What TPS will be good for your cases?

пт, 8 нояб. 2019 г. в 03:34, Siew Wai Yow <[hidden email]>:
Hi!

NetworkIO is just around 1.2MB/s as shown so that is not a problem. We have profiled and did not found any bottleneck from the environment/resources. We also check the application profiling and most of the time consumed by sql execution which looks normal to me. Any advice?



From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 11:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Is it possible that you ended up saturating the network? Please also consider profiling for bottlenecks as I have suggested.

Regards,
--
Ilya Kasnacheev


чт, 7 нояб. 2019 г. в 17:55, Siew Wai Yow <[hidden email]>:
Hi llya!

CPU usage is very low. I just did another run as shown, but IO is happening in a spiky pattern, any clue? 
RAM and disk is fine. Only Ignite is running in the host. Thanks.







Regards,
Yow


From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 10:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Have you tried collecting thread dumps from node under load, to identify any bottlenecks?

What is the CPU usage by Ignite?

Regards,
--
Ilya Kasnacheev


ср, 6 нояб. 2019 г. в 09:08, Siew Wai Yow <[hidden email]>:
Hi Stan

The result will be the same without transaction, even with atomic mode. So wondering if there is other reason capped the performance. Attach is our POC source code which give around 1700 TPS, with or without transaction, may be you can advise what's wrong with it.

-DIGNITE_MAX_INDEX_PAYLOAD_SIZE=64 is using already.

We choose ODBC because it is more compatible to existing product's API, which will need to create table/cache on the fly without pre-define C-object. Ignite C++ need to have the cache C-object ready before run.

Also in Ignite cluster, how can we connect to cluster, just connect to one of the host? And when host crash we need to change manually connect to another node? Seems ODBC connection string not allow to input multiple host address.

Thanks in advance.

Regards,
Yow



From: Stanislav Lukyanov <[hidden email]>
Sent: Wednesday, November 6, 2019 12:33 AM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
First, 1700 TPS given your transaction structure is 170000 simple operations per second, which is quite substantial - especially if you're doing that from a single thread / single ODBC client.

Second, note that TRANSACTIONAL_SNAPSHOT is in beta and is not ready for production use. There are no claims about performance or stability of that cache mode.

Third, you don't need the index on key01 - it is created automatically because it is a primary key. But you do need to set INLINE SIZE of the default index. Run your Ignite server with system property or environment varialy IGNITE_MAX_INDEX_PAYLOAD_SIZE=64.

Finally, the operations you're doing don't look like something to be done with SQL. Consider using key-value API in Ignite C++ instead - https://apacheignite-cpp.readme.io/docs.

Stan

On Sat, Nov 2, 2019 at 8:15 PM Evgenii Zhuravlev <[hidden email]> wrote:
Hi,

Do you use only one ODBC client? Can you start one more ODBC client and check the performance?

Thanks,
Evgenii

сб, 2 нояб. 2019 г. в 16:47, Siew Wai Yow <[hidden email]>:
Hi,

We are doing POC on ignite performance using ODBC driver but the performance is always capped at around 1700 TPS which is too slow. It is local ignite service. All tuning tips from Ignite page has been applied, no bottleneck from CPU and Memory. At the moment not turn on persistence yet, it will be worse if turn on. This POC is very crucial to our product roadmap. 

Any tips? Thank you.

Test case,
1 x insert --> 49 x select -->49 x update --> 1 x delete
repeat for 50000 times.

    const char* insert_sql = "insert into CDRTEST values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
    const char* update_sql = "update CDRTEST set value01=?,value02=?,value03=?,value04=?,value05=?,value21=?,value22=?,value23=?,value24=?,value25=? where key01=?";
    const char* delete_sql = "delete from CDRTEST where key01=?";
    const char* select_sql = "select value01 from CDRTEST where key01=?";

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE TABLE IF NOT EXISTS CDRTEST ( "
  "key01 VARCHAR PRIMARY KEY, "
  "value01 LONG, "
  "value02 LONG, "
  "value03 LONG, "
  "value04 LONG, "
  "value05 LONG, "
  "value06 LONG, "
  "value07 LONG, "
  "value08 LONG, "
  "value09 LONG, "
  "value10 LONG, "
  "value11 LONG, "
  "value12 LONG, "
  "value13 LONG, "
  "value14 LONG, "
  "value15 LONG, "
  "value16 LONG, "
  "value17 LONG, "
  "value18 LONG, "
  "value19 LONG, "
  "value20 LONG, "  
  "value21 VARCHAR, "
  "value22 VARCHAR, "
  "value23 VARCHAR, "
  "value24 VARCHAR, "
  "value25 VARCHAR, "
  "value26 VARCHAR, "
  "value27 VARCHAR, "
  "value28 VARCHAR, "
  "value29 VARCHAR, "
  "value30 VARCHAR, "    
  "value31 VARCHAR, "
  "value32 VARCHAR, "
  "value33 VARCHAR, "
  "value34 VARCHAR, "
  "value35 VARCHAR, "
  "value36 VARCHAR, "
  "value37 VARCHAR, "
  "value38 VARCHAR, "
  "value39 VARCHAR) "
  "WITH \"template=partitioned,atomicity=TRANSACTIONAL_SNAPSHOT,WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC\"")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create table", hstmt, SQL_HANDLE_STMT);

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE INDEX key01t_idx ON CDRTEST(key01) INLINE_SIZE 64")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create index", hstmt, SQL_HANDLE_STMT);


Below are configuration we used,

<?xml version="1.0" encoding="UTF-8"?>

       xsi:schemaLocation="http://www.springframework.org/schema/beans

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="binaryConfiguration">
            <bean class="org.apache.ignite.configuration.BinaryConfiguration">
                <property name="compactFooter" value="false"/>

                <property name="idMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                        <property name="lowerCase" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>

  <property name="clientConnectorConfiguration">
    <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration">
      <!--property name="host" value="127.0.0.1"/-->
      <!--property name="port" value="10800"/-->
      <!--property name="portRange" value="5"/-->
      <property name="maxOpenCursorsPerConnection" value="512"/>
      <property name="socketSendBufferSize" value="65536"/>
      <property name="socketReceiveBufferSize" value="131072"/>
      <!--property name="threadPoolSize" value="4"/-->
    </bean>
  </property>

        <property name="dataStorageConfiguration">
           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
              <!-- Enabling Apache Ignite Persistent Store. -->
              <property name="defaultDataRegionConfiguration">
                 <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                     <property name="persistenceEnabled" value="false"/>
                 </bean>
              </property>
              <property name="walPath" value="/ignite-wal"/>

              <property name="walArchivePath" value="/ignite-wal"/>
           </bean>
        </property>

    </bean>
</beans>

Regards,
Yow


--
Best regards,
Ivan Pavlukhin
Siew Wai Yow Siew Wai Yow
Reply | Threaded
Open this post in threaded view
|

Re: Ignite singel instance

Hi!

Looks like scaling is a must to boost performance. We will try that.

Another question, with ODBC driver the connection string will only accept single address. In case of Ignite cluster, what is the correct way to connect? If node1 crash do I need to switch connection manually in code?

Thanks.

Regards,
Yow




From: Ilya Kasnacheev <[hidden email]>
Sent: Friday, November 8, 2019 6:53 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Of course you may expect that every multi-threaded application will be waiting on locks 99% of thread-time. This is why using some kind of smart tooling, such as Java Flight Recorder/VisualVM is recommended.

Here, you can see that you only use a single connector thread. Consider having more than one ODBC connection at the time to increase throughput.

If you had more than one thread dump, maybe there would be some usage patterns to hint at bottlenecks.

Regards,
--
Ilya Kasnacheev


пт, 8 нояб. 2019 г. в 13:21, Siew Wai Yow <[hidden email]>:
Hi,

Ignite server hotspot during the run, seems like there are locks in Ignite consume most of the time, any idea?



Backtrace,







threadump(please refer to attachment for complete dump),

"client-connector-#157" - Thread t@194
   java.lang.Thread.State: WAITING
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <46ac352c> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None




Thanks!

Regards,
Yow





From: Ilya Kasnacheev <[hidden email]>
Sent: Friday, November 8, 2019 5:21 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

I was meaning that you should benchmark the Java side (Ignite server) and not ODBC client.

What you should be figuring out is which thread pools are busy and what's they up to.

Regards,
--
Ilya Kasnacheev


пт, 8 нояб. 2019 г. в 11:36, Siew Wai Yow <[hidden email]>:
Hi Ivan,

5k TPS the minimum and it would be great to have 10k TPS. Do you think this is feasible? Or Ignite expect user to run with multiple instances?

At the moment only 1.7k TPS in local server without persistence. Same test in a 3-nodes-cluster with persistence enabled, the same application run only 400 TPS which is quite disappointing. Any tips? Could it be ODBC driver slowness?

Regards,
Yow


From: Ivan Pavlukhin <[hidden email]>
Sent: Friday, November 8, 2019 4:18 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hi Yow,

Out of curiosity. What TPS will be good for your cases?

пт, 8 нояб. 2019 г. в 03:34, Siew Wai Yow <[hidden email]>:
Hi!

NetworkIO is just around 1.2MB/s as shown so that is not a problem. We have profiled and did not found any bottleneck from the environment/resources. We also check the application profiling and most of the time consumed by sql execution which looks normal to me. Any advice?



From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 11:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Is it possible that you ended up saturating the network? Please also consider profiling for bottlenecks as I have suggested.

Regards,
--
Ilya Kasnacheev


чт, 7 нояб. 2019 г. в 17:55, Siew Wai Yow <[hidden email]>:
Hi llya!

CPU usage is very low. I just did another run as shown, but IO is happening in a spiky pattern, any clue? 
RAM and disk is fine. Only Ignite is running in the host. Thanks.







Regards,
Yow


From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 10:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Have you tried collecting thread dumps from node under load, to identify any bottlenecks?

What is the CPU usage by Ignite?

Regards,
--
Ilya Kasnacheev


ср, 6 нояб. 2019 г. в 09:08, Siew Wai Yow <[hidden email]>:
Hi Stan

The result will be the same without transaction, even with atomic mode. So wondering if there is other reason capped the performance. Attach is our POC source code which give around 1700 TPS, with or without transaction, may be you can advise what's wrong with it.

-DIGNITE_MAX_INDEX_PAYLOAD_SIZE=64 is using already.

We choose ODBC because it is more compatible to existing product's API, which will need to create table/cache on the fly without pre-define C-object. Ignite C++ need to have the cache C-object ready before run.

Also in Ignite cluster, how can we connect to cluster, just connect to one of the host? And when host crash we need to change manually connect to another node? Seems ODBC connection string not allow to input multiple host address.

Thanks in advance.

Regards,
Yow



From: Stanislav Lukyanov <[hidden email]>
Sent: Wednesday, November 6, 2019 12:33 AM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
First, 1700 TPS given your transaction structure is 170000 simple operations per second, which is quite substantial - especially if you're doing that from a single thread / single ODBC client.

Second, note that TRANSACTIONAL_SNAPSHOT is in beta and is not ready for production use. There are no claims about performance or stability of that cache mode.

Third, you don't need the index on key01 - it is created automatically because it is a primary key. But you do need to set INLINE SIZE of the default index. Run your Ignite server with system property or environment varialy IGNITE_MAX_INDEX_PAYLOAD_SIZE=64.

Finally, the operations you're doing don't look like something to be done with SQL. Consider using key-value API in Ignite C++ instead - https://apacheignite-cpp.readme.io/docs.

Stan

On Sat, Nov 2, 2019 at 8:15 PM Evgenii Zhuravlev <[hidden email]> wrote:
Hi,

Do you use only one ODBC client? Can you start one more ODBC client and check the performance?

Thanks,
Evgenii

сб, 2 нояб. 2019 г. в 16:47, Siew Wai Yow <[hidden email]>:
Hi,

We are doing POC on ignite performance using ODBC driver but the performance is always capped at around 1700 TPS which is too slow. It is local ignite service. All tuning tips from Ignite page has been applied, no bottleneck from CPU and Memory. At the moment not turn on persistence yet, it will be worse if turn on. This POC is very crucial to our product roadmap. 

Any tips? Thank you.

Test case,
1 x insert --> 49 x select -->49 x update --> 1 x delete
repeat for 50000 times.

    const char* insert_sql = "insert into CDRTEST values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
    const char* update_sql = "update CDRTEST set value01=?,value02=?,value03=?,value04=?,value05=?,value21=?,value22=?,value23=?,value24=?,value25=? where key01=?";
    const char* delete_sql = "delete from CDRTEST where key01=?";
    const char* select_sql = "select value01 from CDRTEST where key01=?";

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE TABLE IF NOT EXISTS CDRTEST ( "
  "key01 VARCHAR PRIMARY KEY, "
  "value01 LONG, "
  "value02 LONG, "
  "value03 LONG, "
  "value04 LONG, "
  "value05 LONG, "
  "value06 LONG, "
  "value07 LONG, "
  "value08 LONG, "
  "value09 LONG, "
  "value10 LONG, "
  "value11 LONG, "
  "value12 LONG, "
  "value13 LONG, "
  "value14 LONG, "
  "value15 LONG, "
  "value16 LONG, "
  "value17 LONG, "
  "value18 LONG, "
  "value19 LONG, "
  "value20 LONG, "  
  "value21 VARCHAR, "
  "value22 VARCHAR, "
  "value23 VARCHAR, "
  "value24 VARCHAR, "
  "value25 VARCHAR, "
  "value26 VARCHAR, "
  "value27 VARCHAR, "
  "value28 VARCHAR, "
  "value29 VARCHAR, "
  "value30 VARCHAR, "    
  "value31 VARCHAR, "
  "value32 VARCHAR, "
  "value33 VARCHAR, "
  "value34 VARCHAR, "
  "value35 VARCHAR, "
  "value36 VARCHAR, "
  "value37 VARCHAR, "
  "value38 VARCHAR, "
  "value39 VARCHAR) "
  "WITH \"template=partitioned,atomicity=TRANSACTIONAL_SNAPSHOT,WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC\"")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create table", hstmt, SQL_HANDLE_STMT);

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE INDEX key01t_idx ON CDRTEST(key01) INLINE_SIZE 64")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create index", hstmt, SQL_HANDLE_STMT);


Below are configuration we used,

<?xml version="1.0" encoding="UTF-8"?>

       xsi:schemaLocation="http://www.springframework.org/schema/beans

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="binaryConfiguration">
            <bean class="org.apache.ignite.configuration.BinaryConfiguration">
                <property name="compactFooter" value="false"/>

                <property name="idMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                        <property name="lowerCase" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>

  <property name="clientConnectorConfiguration">
    <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration">
      <!--property name="host" value="127.0.0.1"/-->
      <!--property name="port" value="10800"/-->
      <!--property name="portRange" value="5"/-->
      <property name="maxOpenCursorsPerConnection" value="512"/>
      <property name="socketSendBufferSize" value="65536"/>
      <property name="socketReceiveBufferSize" value="131072"/>
      <!--property name="threadPoolSize" value="4"/-->
    </bean>
  </property>

        <property name="dataStorageConfiguration">
           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
              <!-- Enabling Apache Ignite Persistent Store. -->
              <property name="defaultDataRegionConfiguration">
                 <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                     <property name="persistenceEnabled" value="false"/>
                 </bean>
              </property>
              <property name="walPath" value="/ignite-wal"/>

              <property name="walArchivePath" value="/ignite-wal"/>
           </bean>
        </property>

    </bean>
</beans>

Regards,
Yow


--
Best regards,
Ivan Pavlukhin
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Ignite singel instance

Hello!

I think that ODBC supports supplying more than one address since 2.5 in comma-separated format of:

Connect("DRIVER={Apache Ignite};ADDRESS=127.0.0.1:4242,127.0.0.1:11109..11115,127.0.0.1;SCHEMA=cache");

(example taken from tests)

Regards,
--
Ilya Kasnacheev


пт, 8 нояб. 2019 г. в 17:55, Siew Wai Yow <[hidden email]>:
Hi!

Looks like scaling is a must to boost performance. We will try that.

Another question, with ODBC driver the connection string will only accept single address. In case of Ignite cluster, what is the correct way to connect? If node1 crash do I need to switch connection manually in code?

Thanks.

Regards,
Yow




From: Ilya Kasnacheev <[hidden email]>
Sent: Friday, November 8, 2019 6:53 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Of course you may expect that every multi-threaded application will be waiting on locks 99% of thread-time. This is why using some kind of smart tooling, such as Java Flight Recorder/VisualVM is recommended.

Here, you can see that you only use a single connector thread. Consider having more than one ODBC connection at the time to increase throughput.

If you had more than one thread dump, maybe there would be some usage patterns to hint at bottlenecks.

Regards,
--
Ilya Kasnacheev


пт, 8 нояб. 2019 г. в 13:21, Siew Wai Yow <[hidden email]>:
Hi,

Ignite server hotspot during the run, seems like there are locks in Ignite consume most of the time, any idea?



Backtrace,







threadump(please refer to attachment for complete dump),

"client-connector-#157" - Thread t@194
   java.lang.Thread.State: WAITING
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <46ac352c> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None




Thanks!

Regards,
Yow





From: Ilya Kasnacheev <[hidden email]>
Sent: Friday, November 8, 2019 5:21 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

I was meaning that you should benchmark the Java side (Ignite server) and not ODBC client.

What you should be figuring out is which thread pools are busy and what's they up to.

Regards,
--
Ilya Kasnacheev


пт, 8 нояб. 2019 г. в 11:36, Siew Wai Yow <[hidden email]>:
Hi Ivan,

5k TPS the minimum and it would be great to have 10k TPS. Do you think this is feasible? Or Ignite expect user to run with multiple instances?

At the moment only 1.7k TPS in local server without persistence. Same test in a 3-nodes-cluster with persistence enabled, the same application run only 400 TPS which is quite disappointing. Any tips? Could it be ODBC driver slowness?

Regards,
Yow


From: Ivan Pavlukhin <[hidden email]>
Sent: Friday, November 8, 2019 4:18 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hi Yow,

Out of curiosity. What TPS will be good for your cases?

пт, 8 нояб. 2019 г. в 03:34, Siew Wai Yow <[hidden email]>:
Hi!

NetworkIO is just around 1.2MB/s as shown so that is not a problem. We have profiled and did not found any bottleneck from the environment/resources. We also check the application profiling and most of the time consumed by sql execution which looks normal to me. Any advice?



From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 11:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Is it possible that you ended up saturating the network? Please also consider profiling for bottlenecks as I have suggested.

Regards,
--
Ilya Kasnacheev


чт, 7 нояб. 2019 г. в 17:55, Siew Wai Yow <[hidden email]>:
Hi llya!

CPU usage is very low. I just did another run as shown, but IO is happening in a spiky pattern, any clue? 
RAM and disk is fine. Only Ignite is running in the host. Thanks.







Regards,
Yow


From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 10:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Have you tried collecting thread dumps from node under load, to identify any bottlenecks?

What is the CPU usage by Ignite?

Regards,
--
Ilya Kasnacheev


ср, 6 нояб. 2019 г. в 09:08, Siew Wai Yow <[hidden email]>:
Hi Stan

The result will be the same without transaction, even with atomic mode. So wondering if there is other reason capped the performance. Attach is our POC source code which give around 1700 TPS, with or without transaction, may be you can advise what's wrong with it.

-DIGNITE_MAX_INDEX_PAYLOAD_SIZE=64 is using already.

We choose ODBC because it is more compatible to existing product's API, which will need to create table/cache on the fly without pre-define C-object. Ignite C++ need to have the cache C-object ready before run.

Also in Ignite cluster, how can we connect to cluster, just connect to one of the host? And when host crash we need to change manually connect to another node? Seems ODBC connection string not allow to input multiple host address.

Thanks in advance.

Regards,
Yow



From: Stanislav Lukyanov <[hidden email]>
Sent: Wednesday, November 6, 2019 12:33 AM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
First, 1700 TPS given your transaction structure is 170000 simple operations per second, which is quite substantial - especially if you're doing that from a single thread / single ODBC client.

Second, note that TRANSACTIONAL_SNAPSHOT is in beta and is not ready for production use. There are no claims about performance or stability of that cache mode.

Third, you don't need the index on key01 - it is created automatically because it is a primary key. But you do need to set INLINE SIZE of the default index. Run your Ignite server with system property or environment varialy IGNITE_MAX_INDEX_PAYLOAD_SIZE=64.

Finally, the operations you're doing don't look like something to be done with SQL. Consider using key-value API in Ignite C++ instead - https://apacheignite-cpp.readme.io/docs.

Stan

On Sat, Nov 2, 2019 at 8:15 PM Evgenii Zhuravlev <[hidden email]> wrote:
Hi,

Do you use only one ODBC client? Can you start one more ODBC client and check the performance?

Thanks,
Evgenii

сб, 2 нояб. 2019 г. в 16:47, Siew Wai Yow <[hidden email]>:
Hi,

We are doing POC on ignite performance using ODBC driver but the performance is always capped at around 1700 TPS which is too slow. It is local ignite service. All tuning tips from Ignite page has been applied, no bottleneck from CPU and Memory. At the moment not turn on persistence yet, it will be worse if turn on. This POC is very crucial to our product roadmap. 

Any tips? Thank you.

Test case,
1 x insert --> 49 x select -->49 x update --> 1 x delete
repeat for 50000 times.

    const char* insert_sql = "insert into CDRTEST values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
    const char* update_sql = "update CDRTEST set value01=?,value02=?,value03=?,value04=?,value05=?,value21=?,value22=?,value23=?,value24=?,value25=? where key01=?";
    const char* delete_sql = "delete from CDRTEST where key01=?";
    const char* select_sql = "select value01 from CDRTEST where key01=?";

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE TABLE IF NOT EXISTS CDRTEST ( "
  "key01 VARCHAR PRIMARY KEY, "
  "value01 LONG, "
  "value02 LONG, "
  "value03 LONG, "
  "value04 LONG, "
  "value05 LONG, "
  "value06 LONG, "
  "value07 LONG, "
  "value08 LONG, "
  "value09 LONG, "
  "value10 LONG, "
  "value11 LONG, "
  "value12 LONG, "
  "value13 LONG, "
  "value14 LONG, "
  "value15 LONG, "
  "value16 LONG, "
  "value17 LONG, "
  "value18 LONG, "
  "value19 LONG, "
  "value20 LONG, "  
  "value21 VARCHAR, "
  "value22 VARCHAR, "
  "value23 VARCHAR, "
  "value24 VARCHAR, "
  "value25 VARCHAR, "
  "value26 VARCHAR, "
  "value27 VARCHAR, "
  "value28 VARCHAR, "
  "value29 VARCHAR, "
  "value30 VARCHAR, "    
  "value31 VARCHAR, "
  "value32 VARCHAR, "
  "value33 VARCHAR, "
  "value34 VARCHAR, "
  "value35 VARCHAR, "
  "value36 VARCHAR, "
  "value37 VARCHAR, "
  "value38 VARCHAR, "
  "value39 VARCHAR) "
  "WITH \"template=partitioned,atomicity=TRANSACTIONAL_SNAPSHOT,WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC\"")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create table", hstmt, SQL_HANDLE_STMT);

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE INDEX key01t_idx ON CDRTEST(key01) INLINE_SIZE 64")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create index", hstmt, SQL_HANDLE_STMT);


Below are configuration we used,

<?xml version="1.0" encoding="UTF-8"?>

       xsi:schemaLocation="http://www.springframework.org/schema/beans

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="binaryConfiguration">
            <bean class="org.apache.ignite.configuration.BinaryConfiguration">
                <property name="compactFooter" value="false"/>

                <property name="idMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                        <property name="lowerCase" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>

  <property name="clientConnectorConfiguration">
    <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration">
      <!--property name="host" value="127.0.0.1"/-->
      <!--property name="port" value="10800"/-->
      <!--property name="portRange" value="5"/-->
      <property name="maxOpenCursorsPerConnection" value="512"/>
      <property name="socketSendBufferSize" value="65536"/>
      <property name="socketReceiveBufferSize" value="131072"/>
      <!--property name="threadPoolSize" value="4"/-->
    </bean>
  </property>

        <property name="dataStorageConfiguration">
           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
              <!-- Enabling Apache Ignite Persistent Store. -->
              <property name="defaultDataRegionConfiguration">
                 <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                     <property name="persistenceEnabled" value="false"/>
                 </bean>
              </property>
              <property name="walPath" value="/ignite-wal"/>

              <property name="walArchivePath" value="/ignite-wal"/>
           </bean>
        </property>

    </bean>
</beans>

Regards,
Yow


--
Best regards,
Ivan Pavlukhin
Siew Wai Yow Siew Wai Yow
Reply | Threaded
Open this post in threaded view
|

Re: Ignite singel instance

Hi!

I see, thank you!

What about in case of container? Let's say we have 2 replica of Ignite node, both of them using default port 10800 but same edge node IP. How can i ensure it can reach both replica?

==> v1/Pod(related)
NAME                             READY  STATUS   RESTARTS  AGE
dr-ignite-visor-ff4bf9878-h6h4q  1/1    Running  0         6h20m
ignite-cluster-dr-ignite-0       3/3    Running  0         6h20m
ignite-cluster-dr-ignite-1       3/3    Running  0         6h20m

==> v1/Service
NAME            TYPE      CLUSTER-IP      EXTERNAL-IP  PORT(S)                                                         AGE
ignite-service  NodePort  10.254.136.245  <none>       8080:30364/TCP,10800:32442/TCP,11211:31063/TCP,49113:32042/TCP  6h20m


In ODBC we are connecting using <edge node ip>:32442, do you think this is the right way?

Thanks.

Regards,
Yow




From: Ilya Kasnacheev <[hidden email]>
Sent: Friday, November 8, 2019 11:02 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

I think that ODBC supports supplying more than one address since 2.5 in comma-separated format of:

Connect("DRIVER={Apache Ignite};ADDRESS=127.0.0.1:4242,127.0.0.1:11109..11115,127.0.0.1;SCHEMA=cache");

(example taken from tests)

Regards,
--
Ilya Kasnacheev


пт, 8 нояб. 2019 г. в 17:55, Siew Wai Yow <[hidden email]>:
Hi!

Looks like scaling is a must to boost performance. We will try that.

Another question, with ODBC driver the connection string will only accept single address. In case of Ignite cluster, what is the correct way to connect? If node1 crash do I need to switch connection manually in code?

Thanks.

Regards,
Yow




From: Ilya Kasnacheev <[hidden email]>
Sent: Friday, November 8, 2019 6:53 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Of course you may expect that every multi-threaded application will be waiting on locks 99% of thread-time. This is why using some kind of smart tooling, such as Java Flight Recorder/VisualVM is recommended.

Here, you can see that you only use a single connector thread. Consider having more than one ODBC connection at the time to increase throughput.

If you had more than one thread dump, maybe there would be some usage patterns to hint at bottlenecks.

Regards,
--
Ilya Kasnacheev


пт, 8 нояб. 2019 г. в 13:21, Siew Wai Yow <[hidden email]>:
Hi,

Ignite server hotspot during the run, seems like there are locks in Ignite consume most of the time, any idea?



Backtrace,







threadump(please refer to attachment for complete dump),

"client-connector-#157" - Thread t@194
   java.lang.Thread.State: WAITING
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <46ac352c> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None




Thanks!

Regards,
Yow





From: Ilya Kasnacheev <[hidden email]>
Sent: Friday, November 8, 2019 5:21 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

I was meaning that you should benchmark the Java side (Ignite server) and not ODBC client.

What you should be figuring out is which thread pools are busy and what's they up to.

Regards,
--
Ilya Kasnacheev


пт, 8 нояб. 2019 г. в 11:36, Siew Wai Yow <[hidden email]>:
Hi Ivan,

5k TPS the minimum and it would be great to have 10k TPS. Do you think this is feasible? Or Ignite expect user to run with multiple instances?

At the moment only 1.7k TPS in local server without persistence. Same test in a 3-nodes-cluster with persistence enabled, the same application run only 400 TPS which is quite disappointing. Any tips? Could it be ODBC driver slowness?

Regards,
Yow


From: Ivan Pavlukhin <[hidden email]>
Sent: Friday, November 8, 2019 4:18 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hi Yow,

Out of curiosity. What TPS will be good for your cases?

пт, 8 нояб. 2019 г. в 03:34, Siew Wai Yow <[hidden email]>:
Hi!

NetworkIO is just around 1.2MB/s as shown so that is not a problem. We have profiled and did not found any bottleneck from the environment/resources. We also check the application profiling and most of the time consumed by sql execution which looks normal to me. Any advice?



From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 11:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Is it possible that you ended up saturating the network? Please also consider profiling for bottlenecks as I have suggested.

Regards,
--
Ilya Kasnacheev


чт, 7 нояб. 2019 г. в 17:55, Siew Wai Yow <[hidden email]>:
Hi llya!

CPU usage is very low. I just did another run as shown, but IO is happening in a spiky pattern, any clue? 
RAM and disk is fine. Only Ignite is running in the host. Thanks.







Regards,
Yow


From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 10:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Have you tried collecting thread dumps from node under load, to identify any bottlenecks?

What is the CPU usage by Ignite?

Regards,
--
Ilya Kasnacheev


ср, 6 нояб. 2019 г. в 09:08, Siew Wai Yow <[hidden email]>:
Hi Stan

The result will be the same without transaction, even with atomic mode. So wondering if there is other reason capped the performance. Attach is our POC source code which give around 1700 TPS, with or without transaction, may be you can advise what's wrong with it.

-DIGNITE_MAX_INDEX_PAYLOAD_SIZE=64 is using already.

We choose ODBC because it is more compatible to existing product's API, which will need to create table/cache on the fly without pre-define C-object. Ignite C++ need to have the cache C-object ready before run.

Also in Ignite cluster, how can we connect to cluster, just connect to one of the host? And when host crash we need to change manually connect to another node? Seems ODBC connection string not allow to input multiple host address.

Thanks in advance.

Regards,
Yow



From: Stanislav Lukyanov <[hidden email]>
Sent: Wednesday, November 6, 2019 12:33 AM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
First, 1700 TPS given your transaction structure is 170000 simple operations per second, which is quite substantial - especially if you're doing that from a single thread / single ODBC client.

Second, note that TRANSACTIONAL_SNAPSHOT is in beta and is not ready for production use. There are no claims about performance or stability of that cache mode.

Third, you don't need the index on key01 - it is created automatically because it is a primary key. But you do need to set INLINE SIZE of the default index. Run your Ignite server with system property or environment varialy IGNITE_MAX_INDEX_PAYLOAD_SIZE=64.

Finally, the operations you're doing don't look like something to be done with SQL. Consider using key-value API in Ignite C++ instead - https://apacheignite-cpp.readme.io/docs.

Stan

On Sat, Nov 2, 2019 at 8:15 PM Evgenii Zhuravlev <[hidden email]> wrote:
Hi,

Do you use only one ODBC client? Can you start one more ODBC client and check the performance?

Thanks,
Evgenii

сб, 2 нояб. 2019 г. в 16:47, Siew Wai Yow <[hidden email]>:
Hi,

We are doing POC on ignite performance using ODBC driver but the performance is always capped at around 1700 TPS which is too slow. It is local ignite service. All tuning tips from Ignite page has been applied, no bottleneck from CPU and Memory. At the moment not turn on persistence yet, it will be worse if turn on. This POC is very crucial to our product roadmap. 

Any tips? Thank you.

Test case,
1 x insert --> 49 x select -->49 x update --> 1 x delete
repeat for 50000 times.

    const char* insert_sql = "insert into CDRTEST values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
    const char* update_sql = "update CDRTEST set value01=?,value02=?,value03=?,value04=?,value05=?,value21=?,value22=?,value23=?,value24=?,value25=? where key01=?";
    const char* delete_sql = "delete from CDRTEST where key01=?";
    const char* select_sql = "select value01 from CDRTEST where key01=?";

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE TABLE IF NOT EXISTS CDRTEST ( "
  "key01 VARCHAR PRIMARY KEY, "
  "value01 LONG, "
  "value02 LONG, "
  "value03 LONG, "
  "value04 LONG, "
  "value05 LONG, "
  "value06 LONG, "
  "value07 LONG, "
  "value08 LONG, "
  "value09 LONG, "
  "value10 LONG, "
  "value11 LONG, "
  "value12 LONG, "
  "value13 LONG, "
  "value14 LONG, "
  "value15 LONG, "
  "value16 LONG, "
  "value17 LONG, "
  "value18 LONG, "
  "value19 LONG, "
  "value20 LONG, "  
  "value21 VARCHAR, "
  "value22 VARCHAR, "
  "value23 VARCHAR, "
  "value24 VARCHAR, "
  "value25 VARCHAR, "
  "value26 VARCHAR, "
  "value27 VARCHAR, "
  "value28 VARCHAR, "
  "value29 VARCHAR, "
  "value30 VARCHAR, "    
  "value31 VARCHAR, "
  "value32 VARCHAR, "
  "value33 VARCHAR, "
  "value34 VARCHAR, "
  "value35 VARCHAR, "
  "value36 VARCHAR, "
  "value37 VARCHAR, "
  "value38 VARCHAR, "
  "value39 VARCHAR) "
  "WITH \"template=partitioned,atomicity=TRANSACTIONAL_SNAPSHOT,WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC\"")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create table", hstmt, SQL_HANDLE_STMT);

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE INDEX key01t_idx ON CDRTEST(key01) INLINE_SIZE 64")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create index", hstmt, SQL_HANDLE_STMT);


Below are configuration we used,

<?xml version="1.0" encoding="UTF-8"?>

       xsi:schemaLocation="http://www.springframework.org/schema/beans

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="binaryConfiguration">
            <bean class="org.apache.ignite.configuration.BinaryConfiguration">
                <property name="compactFooter" value="false"/>

                <property name="idMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                        <property name="lowerCase" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>

  <property name="clientConnectorConfiguration">
    <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration">
      <!--property name="host" value="127.0.0.1"/-->
      <!--property name="port" value="10800"/-->
      <!--property name="portRange" value="5"/-->
      <property name="maxOpenCursorsPerConnection" value="512"/>
      <property name="socketSendBufferSize" value="65536"/>
      <property name="socketReceiveBufferSize" value="131072"/>
      <!--property name="threadPoolSize" value="4"/-->
    </bean>
  </property>

        <property name="dataStorageConfiguration">
           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
              <!-- Enabling Apache Ignite Persistent Store. -->
              <property name="defaultDataRegionConfiguration">
                 <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                     <property name="persistenceEnabled" value="false"/>
                 </bean>
              </property>
              <property name="walPath" value="/ignite-wal"/>

              <property name="walArchivePath" value="/ignite-wal"/>
           </bean>
        </property>

    </bean>
</beans>

Regards,
Yow


--
Best regards,
Ivan Pavlukhin
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Ignite singel instance

Hello!

I think you've got to have two edge ports. Maybe containerization will do failover for you, tho.

Regards,
--
Ilya Kasnacheev


пт, 8 нояб. 2019 г. в 18:24, Siew Wai Yow <[hidden email]>:
Hi!

I see, thank you!

What about in case of container? Let's say we have 2 replica of Ignite node, both of them using default port 10800 but same edge node IP. How can i ensure it can reach both replica?

==> v1/Pod(related)
NAME                             READY  STATUS   RESTARTS  AGE
dr-ignite-visor-ff4bf9878-h6h4q  1/1    Running  0         6h20m
ignite-cluster-dr-ignite-0       3/3    Running  0         6h20m
ignite-cluster-dr-ignite-1       3/3    Running  0         6h20m

==> v1/Service
NAME            TYPE      CLUSTER-IP      EXTERNAL-IP  PORT(S)                                                         AGE
ignite-service  NodePort  10.254.136.245  <none>       8080:30364/TCP,10800:32442/TCP,11211:31063/TCP,49113:32042/TCP  6h20m


In ODBC we are connecting using <edge node ip>:32442, do you think this is the right way?

Thanks.

Regards,
Yow




From: Ilya Kasnacheev <[hidden email]>
Sent: Friday, November 8, 2019 11:02 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

I think that ODBC supports supplying more than one address since 2.5 in comma-separated format of:

Connect("DRIVER={Apache Ignite};ADDRESS=127.0.0.1:4242,127.0.0.1:11109..11115,127.0.0.1;SCHEMA=cache");

(example taken from tests)

Regards,
--
Ilya Kasnacheev


пт, 8 нояб. 2019 г. в 17:55, Siew Wai Yow <[hidden email]>:
Hi!

Looks like scaling is a must to boost performance. We will try that.

Another question, with ODBC driver the connection string will only accept single address. In case of Ignite cluster, what is the correct way to connect? If node1 crash do I need to switch connection manually in code?

Thanks.

Regards,
Yow




From: Ilya Kasnacheev <[hidden email]>
Sent: Friday, November 8, 2019 6:53 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Of course you may expect that every multi-threaded application will be waiting on locks 99% of thread-time. This is why using some kind of smart tooling, such as Java Flight Recorder/VisualVM is recommended.

Here, you can see that you only use a single connector thread. Consider having more than one ODBC connection at the time to increase throughput.

If you had more than one thread dump, maybe there would be some usage patterns to hint at bottlenecks.

Regards,
--
Ilya Kasnacheev


пт, 8 нояб. 2019 г. в 13:21, Siew Wai Yow <[hidden email]>:
Hi,

Ignite server hotspot during the run, seems like there are locks in Ignite consume most of the time, any idea?



Backtrace,







threadump(please refer to attachment for complete dump),

"client-connector-#157" - Thread t@194
   java.lang.Thread.State: WAITING
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <46ac352c> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None




Thanks!

Regards,
Yow





From: Ilya Kasnacheev <[hidden email]>
Sent: Friday, November 8, 2019 5:21 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

I was meaning that you should benchmark the Java side (Ignite server) and not ODBC client.

What you should be figuring out is which thread pools are busy and what's they up to.

Regards,
--
Ilya Kasnacheev


пт, 8 нояб. 2019 г. в 11:36, Siew Wai Yow <[hidden email]>:
Hi Ivan,

5k TPS the minimum and it would be great to have 10k TPS. Do you think this is feasible? Or Ignite expect user to run with multiple instances?

At the moment only 1.7k TPS in local server without persistence. Same test in a 3-nodes-cluster with persistence enabled, the same application run only 400 TPS which is quite disappointing. Any tips? Could it be ODBC driver slowness?

Regards,
Yow


From: Ivan Pavlukhin <[hidden email]>
Sent: Friday, November 8, 2019 4:18 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hi Yow,

Out of curiosity. What TPS will be good for your cases?

пт, 8 нояб. 2019 г. в 03:34, Siew Wai Yow <[hidden email]>:
Hi!

NetworkIO is just around 1.2MB/s as shown so that is not a problem. We have profiled and did not found any bottleneck from the environment/resources. We also check the application profiling and most of the time consumed by sql execution which looks normal to me. Any advice?



From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 11:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Is it possible that you ended up saturating the network? Please also consider profiling for bottlenecks as I have suggested.

Regards,
--
Ilya Kasnacheev


чт, 7 нояб. 2019 г. в 17:55, Siew Wai Yow <[hidden email]>:
Hi llya!

CPU usage is very low. I just did another run as shown, but IO is happening in a spiky pattern, any clue? 
RAM and disk is fine. Only Ignite is running in the host. Thanks.







Regards,
Yow


From: Ilya Kasnacheev <[hidden email]>
Sent: Thursday, November 7, 2019 10:04 PM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
Hello!

Have you tried collecting thread dumps from node under load, to identify any bottlenecks?

What is the CPU usage by Ignite?

Regards,
--
Ilya Kasnacheev


ср, 6 нояб. 2019 г. в 09:08, Siew Wai Yow <[hidden email]>:
Hi Stan

The result will be the same without transaction, even with atomic mode. So wondering if there is other reason capped the performance. Attach is our POC source code which give around 1700 TPS, with or without transaction, may be you can advise what's wrong with it.

-DIGNITE_MAX_INDEX_PAYLOAD_SIZE=64 is using already.

We choose ODBC because it is more compatible to existing product's API, which will need to create table/cache on the fly without pre-define C-object. Ignite C++ need to have the cache C-object ready before run.

Also in Ignite cluster, how can we connect to cluster, just connect to one of the host? And when host crash we need to change manually connect to another node? Seems ODBC connection string not allow to input multiple host address.

Thanks in advance.

Regards,
Yow



From: Stanislav Lukyanov <[hidden email]>
Sent: Wednesday, November 6, 2019 12:33 AM
To: [hidden email] <[hidden email]>
Subject: Re: Ignite singel instance
 
First, 1700 TPS given your transaction structure is 170000 simple operations per second, which is quite substantial - especially if you're doing that from a single thread / single ODBC client.

Second, note that TRANSACTIONAL_SNAPSHOT is in beta and is not ready for production use. There are no claims about performance or stability of that cache mode.

Third, you don't need the index on key01 - it is created automatically because it is a primary key. But you do need to set INLINE SIZE of the default index. Run your Ignite server with system property or environment varialy IGNITE_MAX_INDEX_PAYLOAD_SIZE=64.

Finally, the operations you're doing don't look like something to be done with SQL. Consider using key-value API in Ignite C++ instead - https://apacheignite-cpp.readme.io/docs.

Stan

On Sat, Nov 2, 2019 at 8:15 PM Evgenii Zhuravlev <[hidden email]> wrote:
Hi,

Do you use only one ODBC client? Can you start one more ODBC client and check the performance?

Thanks,
Evgenii

сб, 2 нояб. 2019 г. в 16:47, Siew Wai Yow <[hidden email]>:
Hi,

We are doing POC on ignite performance using ODBC driver but the performance is always capped at around 1700 TPS which is too slow. It is local ignite service. All tuning tips from Ignite page has been applied, no bottleneck from CPU and Memory. At the moment not turn on persistence yet, it will be worse if turn on. This POC is very crucial to our product roadmap. 

Any tips? Thank you.

Test case,
1 x insert --> 49 x select -->49 x update --> 1 x delete
repeat for 50000 times.

    const char* insert_sql = "insert into CDRTEST values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
    const char* update_sql = "update CDRTEST set value01=?,value02=?,value03=?,value04=?,value05=?,value21=?,value22=?,value23=?,value24=?,value25=? where key01=?";
    const char* delete_sql = "delete from CDRTEST where key01=?";
    const char* select_sql = "select value01 from CDRTEST where key01=?";

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE TABLE IF NOT EXISTS CDRTEST ( "
  "key01 VARCHAR PRIMARY KEY, "
  "value01 LONG, "
  "value02 LONG, "
  "value03 LONG, "
  "value04 LONG, "
  "value05 LONG, "
  "value06 LONG, "
  "value07 LONG, "
  "value08 LONG, "
  "value09 LONG, "
  "value10 LONG, "
  "value11 LONG, "
  "value12 LONG, "
  "value13 LONG, "
  "value14 LONG, "
  "value15 LONG, "
  "value16 LONG, "
  "value17 LONG, "
  "value18 LONG, "
  "value19 LONG, "
  "value20 LONG, "  
  "value21 VARCHAR, "
  "value22 VARCHAR, "
  "value23 VARCHAR, "
  "value24 VARCHAR, "
  "value25 VARCHAR, "
  "value26 VARCHAR, "
  "value27 VARCHAR, "
  "value28 VARCHAR, "
  "value29 VARCHAR, "
  "value30 VARCHAR, "    
  "value31 VARCHAR, "
  "value32 VARCHAR, "
  "value33 VARCHAR, "
  "value34 VARCHAR, "
  "value35 VARCHAR, "
  "value36 VARCHAR, "
  "value37 VARCHAR, "
  "value38 VARCHAR, "
  "value39 VARCHAR) "
  "WITH \"template=partitioned,atomicity=TRANSACTIONAL_SNAPSHOT,WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC\"")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create table", hstmt, SQL_HANDLE_STMT);

    retcode = SQLExecDirect(hstmt, reinterpret_cast<SQLCHAR*>(const_cast<char*>("CREATE INDEX key01t_idx ON CDRTEST(key01) INLINE_SIZE 64")), SQL_NTS);
            CHECK_ERROR(retcode, "Fail to create index", hstmt, SQL_HANDLE_STMT);


Below are configuration we used,

<?xml version="1.0" encoding="UTF-8"?>

       xsi:schemaLocation="http://www.springframework.org/schema/beans

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="binaryConfiguration">
            <bean class="org.apache.ignite.configuration.BinaryConfiguration">
                <property name="compactFooter" value="false"/>

                <property name="idMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                        <property name="lowerCase" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>

  <property name="clientConnectorConfiguration">
    <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration">
      <!--property name="host" value="127.0.0.1"/-->
      <!--property name="port" value="10800"/-->
      <!--property name="portRange" value="5"/-->
      <property name="maxOpenCursorsPerConnection" value="512"/>
      <property name="socketSendBufferSize" value="65536"/>
      <property name="socketReceiveBufferSize" value="131072"/>
      <!--property name="threadPoolSize" value="4"/-->
    </bean>
  </property>

        <property name="dataStorageConfiguration">
           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
              <!-- Enabling Apache Ignite Persistent Store. -->
              <property name="defaultDataRegionConfiguration">
                 <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                     <property name="persistenceEnabled" value="false"/>
                 </bean>
              </property>
              <property name="walPath" value="/ignite-wal"/>

              <property name="walArchivePath" value="/ignite-wal"/>
           </bean>
        </property>

    </bean>
</beans>

Regards,
Yow


--
Best regards,
Ivan Pavlukhin