Ignite 3rd party persistency DataSourceBean Config in Java

classic Classic list List threaded Threaded
9 messages Options
marble.zhong@coinflex.com marble.zhong@coinflex.com
Reply | Threaded
Open this post in threaded view
|

Ignite 3rd party persistency DataSourceBean Config in Java

Hi Experts,

From the ignite 3rd party rdbms persistency doc, it shows the example with
xml config, like below,

<bean id="dsMySQL_Test" class="com.mysql.cj.jdbc.MysqlDataSource">
  <property name="URL" value="jdbc:mysql://[host]:[port]/[database]"/>
  <property name="user" value="YOUR_USER_NAME"/>
  <property name="password" value="YOUR_PASSWORD"/>
</bean>

Can help an example how to do at the java code? Use which bean? thanks,  
And btw, as currently, ignite dialet not support the postgresql, how can we
use the postgresql to do the 3rd party persistency?  thanks a lot.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
p.srikanta p.srikanta
Reply | Threaded
Open this post in threaded view
|

Re: Ignite 3rd party persistency DataSourceBean Config in Java

 Hi,

I'm using the "org.postgresql.ds.PGPoolingDataSource" dialect within the datasource bean for Postgresql db.

The configuration of the datasource is independent of Ignite and can be done as per the framework you are using. 

Here is an example, if you are using Spring Boot --> https://docs.spring.io/spring-boot/docs/2.1.13.RELEASE/reference/html/howto-data-access.html#howto-configure-a-datasource

An example via the programmatic configuration approach can be found here --> https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/datagrid/store/jdbc/CacheJdbcStoreExample.java


Regards,

Srikanta

On 2020/08/14 07:03:32, "[hidden email]" <[hidden email]> wrote:
> Hi Experts, >
>
> From the ignite 3rd party rdbms persistency doc, it shows the example with>
> xml config, like below, >
>
> <bean id="dsMySQL_Test" class="com.mysql.cj.jdbc.MysqlDataSource">>
>   <property name="URL" value="jdbc:mysql://[host]:[port]/[database]"/>>
>   <property name="user" value="YOUR_USER_NAME"/>>
>   <property name="password" value="YOUR_PASSWORD"/>>
> </bean>>
>
> Can help an example how to do at the java code? Use which bean? thanks,  >
> And btw, as currently, ignite dialet not support the postgresql, how can we>
> use the postgresql to do the 3rd party persistency?  thanks a lot.>
>
>
>
>
>
> -->
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/>

marble.zhong@coinflex.com marble.zhong@coinflex.com
Reply | Threaded
Open this post in threaded view
|

Re: Ignite 3rd party persistency DataSourceBean Config in Java

In reply to this post by marble.zhong@coinflex.com
Need suggestions, thanks a lot.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
marble.zhong@coinflex.com marble.zhong@coinflex.com
Reply | Threaded
Open this post in threaded view
|

Re: Ignite 3rd party persistency DataSourceBean Config in Java

In reply to this post by p.srikanta
Thanks Srikanta,

I'm using below code, with CacheJdbcPojoStoreFactory, for the dsMySQL_Test
dataSourceBean, I try configured in the default-config.xml, but it will show
'IgniteException: Failed to load bean in application context
[beanName=dsMySQL_Test, ', so I am seeking which object can use to config
the DataSourceBean,

CacheJdbcPojoStoreFactory cacheJdbcPojoStoreFactory = new
CacheJdbcPojoStoreFactory<>();

        cacheJdbcPojoStoreFactory.setDataSourceBean("dsMySQL_Test");
       
        cacheJdbcPojoStoreFactory.setDialect(new MySQLDialect());
        JdbcType jdbcType = new JdbcType();
        jdbcType.setCacheName(Student.class.getSimpleName());

thanks.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Ignite 3rd party persistency DataSourceBean Config in Java

Hello!

In this case, you should be using setDataSourceFactory() instead of setDataSourceBean().

Regards,
--
Ilya Kasnacheev


пн, 17 авг. 2020 г. в 10:21, [hidden email] <[hidden email]>:
Thanks Srikanta,

I'm using below code, with CacheJdbcPojoStoreFactory, for the dsMySQL_Test
dataSourceBean, I try configured in the default-config.xml, but it will show
'IgniteException: Failed to load bean in application context
[beanName=dsMySQL_Test, ', so I am seeking which object can use to config
the DataSourceBean,

CacheJdbcPojoStoreFactory cacheJdbcPojoStoreFactory = new
CacheJdbcPojoStoreFactory<>();

        cacheJdbcPojoStoreFactory.setDataSourceBean("dsMySQL_Test");

        cacheJdbcPojoStoreFactory.setDialect(new MySQLDialect());
        JdbcType jdbcType = new JdbcType();
        jdbcType.setCacheName(Student.class.getSimpleName());

thanks.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
marble.zhong@coinflex.com marble.zhong@coinflex.com
Reply | Threaded
Open this post in threaded view
|

Re: Ignite 3rd party persistency DataSourceBean Config in Java

Thanks a lot Kasnacheev,

I have used the below mysqlDataSource, but still met the exception, any
ideas? thanks a lot.

'org.apache.ignite.IgniteException: Failed to initialize cache store (data
source is not provided).'

public void setJdbcStoreConfig(CacheConfiguration config) {
        System.out.println("start to connect to mysql ...");
        CacheJdbcPojoStoreFactory cacheJdbcPojoStoreFactory = new
CacheJdbcPojoStoreFactory<>();

        MysqlDataSource mysqlDataSource = new MysqlDataSource();
       
mysqlDataSource.setUrl("jdbc:mysql://127.0.0.1:3306/userdb?useUnicode=true&characterEncoding=utf8");
        mysqlDataSource.setUser("user-a");
        mysqlDataSource.setPassword("admin");

        cacheJdbcPojoStoreFactory.setDialect(new MySQLDialect());
        cacheJdbcPojoStoreFactory.setDataSource(mysqlDataSource);
        JdbcType jdbcType = new JdbcType();
        jdbcType.setCacheName(Balance.class.getSimpleName());
        jdbcType.setDatabaseTable("balance");
        cacheJdbcPojoStoreFactory.setTypes(jdbcType);
        config.setCacheStoreFactory(cacheJdbcPojoStoreFactory);
        config.setWriteThrough(true);
        config.setReadThrough(true);

        QueryEntity queryEntity = new QueryEntity();
        queryEntity.setKeyType("java.lang.String");
        queryEntity.setValueType("com.marble.common.ignite.entity.Balance");
        queryEntity.setKeyFieldName("id");

        LinkedHashMap<String, String> map = new LinkedHashMap<>();
        map.put("account_id", "java.lang.Long");
        map.put("instrument_id", "java.lang.String");
        map.put("quantity", "java.math.BigDecimal");
        map.put("reserved", "java.math.BigDecimal");
        map.put("last_updated", "java.sql.Timestamp");
        map.put("last_Updated_Event", "java.lang.String");
        map.put("trade_type", "java.lang.String");

        queryEntity.setFields(map);
        Collection<QueryEntity> queryEntities = new
ArrayList<QueryEntity>();
        queryEntities.add(queryEntity);

        config.setQueryEntities(queryEntities);
    }



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
marble.zhong@coinflex.com marble.zhong@coinflex.com
Reply | Threaded
Open this post in threaded view
|

Re: Ignite 3rd party persistency DataSourceBean Config in Java

hi Thanks a lot.

Correct the setDataSourceFactory, met ignite nullPointerException below,
appreciate for the suggestions, thanks.

javax.cache.integration.CacheLoaderException: java.lang.NullPointerException
        at
org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.loadCache(GridCacheStoreManagerAdapter.java:545)
        at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.localLoadCache(GridDhtCacheAdapter.java:636)
        at
org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.localLoadCache(GridCacheProxyImpl.java:226)
        at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$LoadCacheJob.localExecute(GridCacheAdapter.java:6052)
        at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$LoadCacheJobV2.localExecute(GridCacheAdapter.java:6101)
        at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$TopologyVersionAwareJob.execute(GridCacheAdapter.java:6735)
        at
org.apache.ignite.compute.ComputeJobAdapter.call(ComputeJobAdapter.java:131)
        at
org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2.execute(GridClosureProcessor.java:1855)
        at
org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:596)
        at
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7005)
        at
org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:590)
        at
org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:519)
        at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
        at
org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1293)
        at
org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:2121)
        at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1847)
        at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1472)
        at
org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
        at
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1367)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
marble.zhong@coinflex.com marble.zhong@coinflex.com
Reply | Threaded
Open this post in threaded view
|

Re: Ignite 3rd party persistency DataSourceBean Config in Java

More Exceptions here, thanks

06:29:58,723 [mgmt-#66] ERROR
org.apache.ignite.internal.processors.task.GridTaskWorker  - Failed to
obtain remote job result policy for result from ComputeTask.result(..)
method (will fail the whole task): GridJobResultImpl [job=C2
[c=LoadCacheJobV2 [keepBinary=false]], sib=GridJobSiblingImpl
[sesId=04564002471-a65841ce-1b5c-43a2-af82-ff14f930c049,
jobId=14564002471-a65841ce-1b5c-43a2-af82-ff14f930c049,
nodeId=43aaf007-81a1-4ade-b2b2-d3ba2d57b962, isJobDone=false],
jobCtx=GridJobContextImpl
[jobId=14564002471-a65841ce-1b5c-43a2-af82-ff14f930c049, timeoutObj=null,
attrs=HashMap {}], node=TcpDiscoveryNode
[id=43aaf007-81a1-4ade-b2b2-d3ba2d57b962,
consistentId=0:0:0:0:0:0:0:1%lo0,127.0.0.1,192.168.50.115:47500,
addrs=ArrayList [0:0:0:0:0:0:0:1%lo0, 127.0.0.1, 192.168.50.115],
sockAddrs=HashSet [/0:0:0:0:0:0:0:1%lo0:47500, /127.0.0.1:47500,
Marbles-MBP.lan/192.168.50.115:47500], discPort=47500, order=1, intOrder=1,
lastExchangeTime=1598264995506, loc=false, ver=2.8.1#20200521-sha1:86422096,
isClient=false], ex=class o.a.i.IgniteException:
java.lang.NullPointerException, hasRes=true, isCancelled=false,
isOccupied=true]
class org.apache.ignite.IgniteException: Remote job threw user exception
(override or implement ComputeTask.result(..) method if you would like to
have automatic failover for this exception): java.lang.NullPointerException
        at
org.apache.ignite.compute.ComputeTaskAdapter.result(ComputeTaskAdapter.java:102)
        at
org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1062)
        at
org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1055)
        at
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7037)
        at
org.apache.ignite.internal.processors.task.GridTaskWorker.result(GridTaskWorker.java:1055)
        at
org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:862)
        at
org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1146)
        at
org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1370)
        at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1847)
        at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1472)
        at
org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
        at
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1367)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Ignite 3rd party persistency DataSourceBean Config in Java

Hello!

I think that you have skipped some "caused by" clauses here. Can you please the full exception trace?

Thanks,
--
Ilya Kasnacheev


пн, 24 авг. 2020 г. в 13:41, [hidden email] <[hidden email]>:
More Exceptions here, thanks

06:29:58,723 [mgmt-#66] ERROR
org.apache.ignite.internal.processors.task.GridTaskWorker  - Failed to
obtain remote job result policy for result from ComputeTask.result(..)
method (will fail the whole task): GridJobResultImpl [job=C2
[c=LoadCacheJobV2 [keepBinary=false]], sib=GridJobSiblingImpl
[sesId=04564002471-a65841ce-1b5c-43a2-af82-ff14f930c049,
jobId=14564002471-a65841ce-1b5c-43a2-af82-ff14f930c049,
nodeId=43aaf007-81a1-4ade-b2b2-d3ba2d57b962, isJobDone=false],
jobCtx=GridJobContextImpl
[jobId=14564002471-a65841ce-1b5c-43a2-af82-ff14f930c049, timeoutObj=null,
attrs=HashMap {}], node=TcpDiscoveryNode
[id=43aaf007-81a1-4ade-b2b2-d3ba2d57b962,
consistentId=0:0:0:0:0:0:0:1%lo0,127.0.0.1,192.168.50.115:47500,
addrs=ArrayList [0:0:0:0:0:0:0:1%lo0, 127.0.0.1, 192.168.50.115],
sockAddrs=HashSet [/0:0:0:0:0:0:0:1%lo0:47500, /127.0.0.1:47500,
Marbles-MBP.lan/192.168.50.115:47500], discPort=47500, order=1, intOrder=1,
lastExchangeTime=1598264995506, loc=false, ver=2.8.1#20200521-sha1:86422096,
isClient=false], ex=class o.a.i.IgniteException:
java.lang.NullPointerException, hasRes=true, isCancelled=false,
isOccupied=true]
class org.apache.ignite.IgniteException: Remote job threw user exception
(override or implement ComputeTask.result(..) method if you would like to
have automatic failover for this exception): java.lang.NullPointerException
        at
org.apache.ignite.compute.ComputeTaskAdapter.result(ComputeTaskAdapter.java:102)
        at
org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1062)
        at
org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1055)
        at
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7037)
        at
org.apache.ignite.internal.processors.task.GridTaskWorker.result(GridTaskWorker.java:1055)
        at
org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:862)
        at
org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1146)
        at
org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1370)
        at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1847)
        at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1472)
        at
org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
        at
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1367)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/