Scheduling Cache Refresh

classic Classic list List threaded Threaded
17 messages Options
nithin91 nithin91
Reply | Threaded
Open this post in threaded view
|

Scheduling Cache Refresh

Hi ,

We are using Ignite as a caching layer and loading the data from Oracle to
Ignite using JDBC POJO method
ignite.cache("CacheName").loadCache(null,"java.lang.String","CustomSql").

Now we want to schedule the refresh in such a way that whenever the job that
updates the Oracle Table has completed corresponding cache refresh job has
to be triggered.

We thought about going with following approach


    Build a Rest API using Spring Boot that accepts cache name as the query
parameter post which
    corresponding cache would be refreshed by calling the JDBC POJO method
   ignite.cache("CacheName").loadCache(null,"java.lang.String","CustomSql").


    Once the job that updates the Oracle Table has completed,a HTTP Get
Request would be triggered with
    the Cache Name as Query Parameter.

 *  But this apprach isn't working because JDBC POJO method
   ignite.cache("CacheName").loadCache(null,"java.lang.String","CustomSql")
is not available from Spring
   Boot.*

   Can anyone help me with best possible approach to schedule the cache
refresh.







--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
akorensh akorensh
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling Cache Refresh

 *  But this apprach isn't working because JDBC POJO method
   ignite.cache("CacheName").loadCache(null,"java.lang.String","CustomSql")
is not available from Spring
   Boot.*


ignite.cache("cacheName").loadCache is located here:
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteCache.html

It does not conflict w/SpringBoot and is compatible with running SpringBoot
in the same JVM.

See:
https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/datagrid/store/spring/CacheSpringStoreExample.java

and:
https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/datagrid/store/jdbc/CacheJdbcStoreExample.java


Send over a reproducer project where it does not work and I'll take a look.

Also take examine the possibility of using having the REST API calling a
compute job which would
execute in a separate
server:https://apacheignite.readme.io/docs/compute-grid

Thanks, Alex




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
nithin91 nithin91
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling Cache Refresh

This post was updated on .
Hi

Attached the src folder and pom.xml of Sample Spring Boot  Application that i am running in my local
machine to connect to ignite cluster deployed in Unix as a client node and
refreshes the cache when ever a GET Request is made.

Facing the following error.Can you please help me in resolving the errors

*org.apache.ignite.IgniteException: Spring application context resource is
not injected.*

I am facing this error while trying to refresh the cache using the following
method

ignite.cache("NumberandDateFormatCache").loadCache(null,"java.lang.Long",
            "select a.*,row_number() over(order by COUNTRY_CODE) AS ID FROM
\"vwNumberAndDateFormat\" a where rownum<=10 ");

but the application gets executed fine when i try to get the list of caches
using the following method

ignite.cacheNames();


Also can you please let me know is this the right way to instantiate ignite
instance for a building
Rest-API using Spring Boot.

 srcandpom.zip


--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling Cache Refresh

Hello!

Unfortunately I don't see any attachment to your message. Can you please share a link?

Regards,
--
Ilya Kasnacheev


чт, 14 мая 2020 г. в 10:09, nithin91 <[hidden email]>:
Hi

Attached the Sample Spring Boot Application that i am running in my local
machine to connect to ignite cluster deployed in Unix as a client node and
refreshes the cache when ever a GET Request is made.

Facing the following error.Can you please help me in resolving the errors

*org.apache.ignite.IgniteException: Spring application context resource is
not injected.*

I am facing this error while trying to refresh the cache using the following
method

ignite.cache("NumberandDateFormatCache").loadCache(null,"java.lang.Long",
                "select a.*,row_number() over(order by COUNTRY_CODE) AS ID FROM
\"vwNumberAndDateFormat\" a where rownum<=10 ");

but the application gets executed fine when i try to get the list of caches
using the following method

ignite.cacheNames();

Attached the entire project for your reference.Can you please help me in
resolving this issue.

Also can you please let me know is this the right way to instantiate ignite
instance for a building
Rest-API using Spring Boot.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
nithin91 nithin91
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling Cache Refresh

Hi

Attached link is at the end of post.However i am attaching it again here.
srcandpom.zip
<http://apache-ignite-users.70518.x6.nabble.com/file/t2737/srcandpom.zip>  

Please let me know in case of any difficulties while opening the attachment.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
akorensh akorensh
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling Cache Refresh

Hi,  
   loadCache(..) is designed to load the cache from a 3rd party store.

   you are not setting the cache factory in your app.

  See: https://apacheignite.readme.io/docs/3rd-party-store#rdbms-integration 

  example:
https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/datagrid/store/jdbc/CacheJdbcStoreExample.java

  it has:  
cacheCfg.setCacheStoreFactory(FactoryBuilder.factoryOf(CacheJdbcPersonStore.class));
   this tell loadCache(..) where to get the data from.

Thanks, Alex




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
nithin91 nithin91
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling Cache Refresh

Hi

Actually the Bean File which contains  the Cache Store Factory Details  and
Cache Configuration Details
is deployed in Linux Server on which the Ignite Instance is running.I am try
to connect to the ignite instance running on the Linux Server from my local
machine.

PFA for the bean file that is deployed. Ignite-Server.xml
<http://apache-ignite-users.70518.x6.nabble.com/file/t2737/Ignite-Server.xml>  





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
akorensh akorensh
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling Cache Refresh

can you share the server logs.
turn on debug for the appropriate package:
org/apache/ignite/cache/store/jdbc
https://apacheignite.readme.io/docs/logging

  What is the behavior you are getting? No logs related to this at all.

you should be getting the following:
   Start loading entries from db using user queries from arguments.

Started load cache using custom statement [cache=" + U.maskName(cacheName) +
                                      ", keyType=" + keyType + ", stmt=" +
stmt + ']'

"Started load cache using custom query [cache=" + U.maskName(cacheName) +
                                  ", keyType=" + keyType + ", query=" + qry
+ ']'


The actual loading occurs here:
https://github.com/apache/ignite/blob/fb2f608005a29e9a558942dd648bac771bc44830/modules/core/src/main/java/org/apache/ignite/cache/store/jdbc/CacheAbstractJdbcStore.java#L680


Thanks, Alex



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
nithin91 nithin91
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling Cache Refresh

Please find below the log info


  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.2.5.RELEASE)

2020-05-15 10:35:32.005  INFO 14684 --- [           main]
org.example.springbootapi.App            : Starting App on HYDNBK155967 with
PID 14684 (C:\Users\ngovind\eclipse-workspace\springbootapi\target\classes
started by ngovind in C:\Users\ngovind\eclipse-workspace\springbootapi)
2020-05-15 10:35:32.008  INFO 14684 --- [           main]
org.example.springbootapi.App            : No active profile set, falling
back to default profiles: default
2020-05-15 10:35:32.564  INFO 14684 --- [           main]
.s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA
repositories in DEFAULT mode.
2020-05-15 10:35:32.586  INFO 14684 --- [           main]
.s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository
scanning in 13ms. Found 0 JPA repository interfaces.
2020-05-15 10:35:33.450  INFO 14684 --- [           main]
o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s):
3000 (http)
2020-05-15 10:35:33.458  INFO 14684 --- [           main]
o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2020-05-15 10:35:33.459  INFO 14684 --- [           main]
org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache
Tomcat/9.0.31]
2020-05-15 10:35:33.696  INFO 14684 --- [           main]
org.apache.jasper.servlet.TldScanner     : At least one JAR was scanned for
TLDs yet contained no TLDs. Enable debug logging for this logger for a
complete list of JARs that were scanned but no TLDs were found in them.
Skipping unneeded JARs during scanning can improve startup time and JSP
compilation time.
2020-05-15 10:35:33.699  INFO 14684 --- [           main]
o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded
WebApplicationContext
2020-05-15 10:35:33.699  INFO 14684 --- [           main]
o.s.web.context.ContextLoader            : Root WebApplicationContext:
initialization completed in 1645 ms
2020-05-15 10:35:33.955  INFO 14684 --- [           main]
com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2020-05-15 10:35:34.073  INFO 14684 --- [           main]
com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
2020-05-15 10:35:34.120  INFO 14684 --- [           main]
o.hibernate.jpa.internal.util.LogHelper  : HHH000204: Processing
PersistenceUnitInfo [name: default]
2020-05-15 10:35:34.184  INFO 14684 --- [           main]
org.hibernate.Version                    : HHH000412: Hibernate ORM core
version 5.4.12.Final
2020-05-15 10:35:34.305  INFO 14684 --- [           main]
o.hibernate.annotations.common.Version   : HCANN000001: Hibernate Commons
Annotations {5.1.0.Final}
2020-05-15 10:35:34.406  INFO 14684 --- [           main]
org.hibernate.dialect.Dialect            : HHH000400: Using dialect:
org.hibernate.dialect.H2Dialect
2020-05-15 10:35:34.619  INFO 14684 --- [           main]
o.h.e.t.j.p.i.JtaPlatformInitiator       : HHH000490: Using JtaPlatform
implementation:
[org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2020-05-15 10:35:34.625  INFO 14684 --- [           main]
j.LocalContainerEntityManagerFactoryBean : Initialized JPA
EntityManagerFactory for persistence unit 'default'
2020-05-15 10:35:34.687  WARN 14684 --- [           main]                                        
: Failed to resolve default logging config file:
config/java.util.logging.properties
Console logging handler is not configured.
2020-05-15 10:35:34.690  WARN 14684 --- [           main]
o.apache.ignite.internal.util.typedef.G  : Ignite work directory is not
provided, automatically resolved to:
C:\Users\ngovind\eclipse-workspace\springbootapi\ignite\work
2020-05-15 10:35:34.783  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  :

>>>    __________  ________________  
>>>   /  _/ ___/ |/ /  _/_  __/ __/  
>>>  _/ // (7 7    // /  / / / _/    
>>> /___/\___/_/|_/___/ /_/ /___/  
>>>
>>> ver. 2.7.6#20190911-sha1:21f7ca41
>>> 2019 Copyright(C) Apache Software Foundation
>>>
>>> Ignite documentation: http://ignite.apache.org

2020-05-15 10:35:34.783  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : Config URL: n/a
2020-05-15 10:35:34.798  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : IgniteConfiguration
[igniteInstanceName=null, pubPoolSize=8, svcPoolSize=8, callbackPoolSize=8,
stripedPoolSize=8, sysPoolSize=8, mgmtPoolSize=4, igfsPoolSize=8,
dataStreamerPoolSize=8, utilityCachePoolSize=8,
utilityCacheKeepAliveTime=60000, p2pPoolSize=2, qryPoolSize=8,
igniteHome=null,
igniteWorkDir=C:\Users\ngovind\eclipse-workspace\springbootapi\ignite\work,
mbeanSrv=com.sun.jmx.mbeanserver.JmxMBeanServer@368d5c00,
nodeId=0afce27c-9b6d-402f-807a-36d9c5a506b6, marsh=BinaryMarshaller [],
marshLocJobs=false, daemon=false, p2pEnabled=true, netTimeout=5000,
sndRetryDelay=1000, sndRetryCnt=3, metricsHistSize=10000,
metricsUpdateFreq=2000, metricsExpTime=9223372036854775807,
discoSpi=TcpDiscoverySpi [addrRslvr=null, sockTimeout=0, ackTimeout=0,
marsh=null, reconCnt=10, reconDelay=2000, maxAckTimeout=600000,
forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null],
segPlc=STOP, segResolveAttempts=2, waitForSegOnStart=true,
allResolversPassReq=true, segChkFreq=10000, commSpi=TcpCommunicationSpi
[connectGate=null,
connPlc=org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$FirstConnectionPolicy@39666e42,
enableForcibleNodeKill=false, enableTroubleshootingLog=false, locAddr=null,
locHost=null, locPort=47100, locPortRange=100, shmemPort=-1, directBuf=true,
directSndBuf=false, idleConnTimeout=600000, connTimeout=5000,
maxConnTimeout=600000, reconCnt=10, sockSndBuf=32768, sockRcvBuf=32768,
msgQueueLimit=0, slowClientQueueLimit=0, nioSrvr=null, shmemSrv=null,
usePairedConnections=false, connectionsPerNode=1, tcpNoDelay=true,
filterReachableAddresses=false, ackSndThreshold=32, unackedMsgsBufSize=0,
sockWriteTimeout=2000, boundTcpPort=-1, boundTcpShmemPort=-1,
selectorsCnt=4, selectorSpins=0, addrRslvr=null,
ctxInitLatch=java.util.concurrent.CountDownLatch@25c2a9e3[Count = 1],
stopping=false],
evtSpi=org.apache.ignite.spi.eventstorage.NoopEventStorageSpi@9b23822,
colSpi=NoopCollisionSpi [], deploySpi=LocalDeploymentSpi [],
indexingSpi=org.apache.ignite.spi.indexing.noop.NoopIndexingSpi@32f32623,
addrRslvr=null,
encryptionSpi=org.apache.ignite.spi.encryption.noop.NoopEncryptionSpi@3becc950,
clientMode=true, rebalanceThreadPoolSize=1, txCfg=TransactionConfiguration
[txSerEnabled=false, dfltIsolation=REPEATABLE_READ,
dfltConcurrency=PESSIMISTIC, dfltTxTimeout=0,
txTimeoutOnPartitionMapExchange=0, pessimisticTxLogSize=0,
pessimisticTxLogLinger=10000, tmLookupClsName=null, txManagerFactory=null,
useJtaSync=false], cacheSanityCheckEnabled=true, discoStartupDelay=60000,
deployMode=SHARED, p2pMissedCacheSize=100, locHost=null,
timeSrvPortBase=31100, timeSrvPortRange=100, failureDetectionTimeout=10000,
sysWorkerBlockedTimeout=null, clientFailureDetectionTimeout=30000,
metricsLogFreq=60000, hadoopCfg=null, connectorCfg=ConnectorConfiguration
[jettyPath=null, host=null, port=11211, noDelay=true, directBuf=false,
sndBufSize=32768, rcvBufSize=32768, idleQryCurTimeout=600000,
idleQryCurCheckFreq=60000, sndQueueLimit=0, selectorCnt=4, idleTimeout=7000,
sslEnabled=false, sslClientAuth=false, sslCtxFactory=null, sslFactory=null,
portRange=100, threadPoolSize=8, msgInterceptor=null], odbcCfg=null,
warmupClos=null, atomicCfg=AtomicConfiguration [seqReserveSize=1000,
cacheMode=PARTITIONED, backups=1, aff=null, grpName=null], classLdr=null,
sslCtxFactory=null, platformCfg=null, binaryCfg=null, memCfg=null,
pstCfg=null, dsCfg=null, activeOnStart=true, autoActivation=true,
longQryWarnTimeout=3000, sqlConnCfg=null,
cliConnCfg=ClientConnectorConfiguration [host=null, port=10800,
portRange=100, sockSndBufSize=0, sockRcvBufSize=0, tcpNoDelay=true,
maxOpenCursorsPerConn=128, threadPoolSize=8, idleTimeout=0,
jdbcEnabled=true, odbcEnabled=true, thinCliEnabled=true, sslEnabled=false,
useIgniteSslCtxFactory=true, sslClientAuth=false, sslCtxFactory=null],
mvccVacuumThreadCnt=2, mvccVacuumFreq=5000, authEnabled=false,
failureHnd=null, commFailureRslvr=null]
2020-05-15 10:35:34.799  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : Daemon mode: off
2020-05-15 10:35:34.799  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : OS: Windows 10 10.0 amd64
2020-05-15 10:35:34.799  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : OS user: ngovind
2020-05-15 10:35:34.799  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : PID: 14684
2020-05-15 10:35:34.799  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : Language runtime: Java Platform
API Specification ver. 1.8
2020-05-15 10:35:34.800  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : VM information: Java(TM) SE
Runtime Environment 1.8.0_131-b11 Oracle Corporation Java HotSpot(TM) 64-Bit
Server VM 25.131-b11
2020-05-15 10:35:34.800  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : VM total memory: 3.5GB
2020-05-15 10:35:34.800  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : Remote Management [restart: off,
REST: off, JMX (remote: off)]
2020-05-15 10:35:34.800  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : Logger: JavaLogger [quiet=false,
config=null]
2020-05-15 10:35:34.800  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : IGNITE_HOME=null
2020-05-15 10:35:34.801  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : VM arguments:
[-Dfile.encoding=UTF-8]
2020-05-15 10:35:34.809  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : Configured caches [in 'sysMemPlc'
dataRegion: ['ignite-sys-cache']]
2020-05-15 10:35:34.809  WARN 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : Peer class loading is enabled
(disable it in production for performance and deployment consistency
reasons)
2020-05-15 10:35:34.809  WARN 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : Please set system property
'-Djava.net.preferIPv4Stack=true' to avoid possible problems in mixed
environments.
2020-05-15 10:35:34.810  WARN 14684 --- [        pub-#19]
o.apache.ignite.internal.GridDiagnostic  : This operating system has been
tested less rigorously: Windows 10 10.0 amd64. Our team will appreciate the
feedback if you experience any problems running ignite in this environment.
2020-05-15 10:35:34.811  WARN 14684 --- [        pub-#22]
o.apache.ignite.internal.GridDiagnostic  : Initial heap size is 254MB
(should be no less than 512MB, use -Xms512m -Xmx512m).
2020-05-15 10:35:34.886  INFO 14684 --- [           main]
o.a.i.i.p.plugin.IgnitePluginProcessor   : Configured plugins:
2020-05-15 10:35:34.886  INFO 14684 --- [           main]
o.a.i.i.p.plugin.IgnitePluginProcessor   :   ^-- None
2020-05-15 10:35:34.886  INFO 14684 --- [           main]
o.a.i.i.p.plugin.IgnitePluginProcessor   :
2020-05-15 10:35:34.889  INFO 14684 --- [           main]
o.a.i.i.p.failure.FailureProcessor       : Configured failure handler:
[hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED,
SYSTEM_CRITICAL_OPERATION_TIMEOUT]]]]
2020-05-15 10:35:34.969  INFO 14684 --- [           main]
o.a.i.s.c.tcp.TcpCommunicationSpi        : Successfully bound communication
NIO server to TCP port [port=47100, locHost=0.0.0.0/0.0.0.0, selectorsCnt=4,
selectorSpins=0, pairedConn=false]
2020-05-15 10:35:39.825  WARN 14684 --- [           main]
o.a.i.s.c.tcp.TcpCommunicationSpi        : Message queue limit is set to 0
which may lead to potential OOMEs when running cache operations in
FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and
receiver sides.
2020-05-15 10:35:39.842  WARN 14684 --- [           main]
o.a.i.s.c.noop.NoopCheckpointSpi         : Checkpoints are disabled (to
enable configure any GridCheckpointSpi implementation)
2020-05-15 10:35:39.864  WARN 14684 --- [           main]
o.a.i.i.m.c.GridCollisionManager         : Collision resolution is disabled
(all jobs will be activated upon arrival).
2020-05-15 10:35:39.867  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : Security status
[authentication=off, tls/ssl=off]
2020-05-15 10:35:40.081  INFO 14684 --- [           main]
o.a.i.i.p.c.d.d.t.PartitionsEvictManager : Evict partition permits=2
2020-05-15 10:35:40.137  WARN 14684 --- [           main]
o.a.i.i.p.query.h2.IgniteH2Indexing      : Serialization of Java objects in
H2 was enabled.
2020-05-15 10:35:40.172  INFO 14684 --- [           main]
o.a.i.i.p.odbc.ClientListenerProcessor   : Client connector processor has
started on TCP port 10800
2020-05-15 10:35:40.188  INFO 14684 --- [           main]
o.a.i.i.p.rest.GridRestProcessor         : REST protocols do not start on
client node. To start the protocols on client node set
'-DIGNITE_REST_START_ON_CLIENT=true' system property.
2020-05-15 10:35:41.352  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : Non-loopback local IPs:
10.146.121.167, 192.168.0.5, fe80:0:0:0:2fd2:b78f:3b35:22e5%eth0,
fe80:0:0:0:41c0:729c:8669:75f0%eth5, fe80:0:0:0:49df:b6a1:c48:4a8e%eth2,
fe80:0:0:0:683f:1be7:a09a:cc24%wlan1, fe80:0:0:0:a1a7:988d:9428:5678%wlan2,
fe80:0:0:0:a415:3742:eb6a:6ff9%eth0, fe80:0:0:0:e105:6dcd:3606:9e24%wlan0
2020-05-15 10:35:41.352  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : Enabled local MACs: 00059A3C7A00,
2079186209AD, 2079186209AE, 2079186209B1, 2279186209AD, B00CD1CB4F10
2020-05-15 10:36:06.226  INFO 14684 --- [fier-worker-#36]
o.a.i.i.p.cache.mvcc.MvccProcessorImpl   : Assigned mvcc coordinator
[crd=MvccCoordinator [nodeId=be10cd31-aed7-448a-8fec-60fd72a62313,
crdVer=1589462925826, topVer=AffinityTopologyVersion [topVer=13,
minorTopVer=0]], crdNode=TcpDiscoveryNode
[id=be10cd31-aed7-448a-8fec-60fd72a62313, addrs=[127.0.0.1, 172.30.197.5],
sockAddrs=[/127.0.0.1:47500,
azuswvlnx00687.corp.frk.com/172.30.197.5:47500], discPort=47500, order=1,
intOrder=1, lastExchangeTime=1589519158803, loc=false,
ver=2.7.6#20190911-sha1:21f7ca41, isClient=false]]
2020-05-15 10:36:06.254  INFO 14684 --- [ange-worker-#38]
o.apache.ignite.internal.exchange.time   : Started exchange init
[topVer=AffinityTopologyVersion [topVer=13, minorTopVer=0],
mvccCrd=MvccCoordinator [nodeId=be10cd31-aed7-448a-8fec-60fd72a62313,
crdVer=1589462925826, topVer=AffinityTopologyVersion [topVer=13,
minorTopVer=0]], mvccCrdChange=true, crd=false, evt=NODE_JOINED,
evtNode=0afce27c-9b6d-402f-807a-36d9c5a506b6, customEvt=null,
allowMerge=true]
2020-05-15 10:36:06.254  INFO 14684 --- [ange-worker-#38]
org.apache.ignite.cache.msg              : Components activation performed
in 0 ms.
2020-05-15 10:36:06.366  INFO 14684 --- [ange-worker-#38]
o.a.i.i.p.cache.GridCacheProcessor       : Started cache
[name=ignite-sys-cache, id=-2100569601, memoryPolicyName=sysMemPlc,
mode=REPLICATED, atomicity=TRANSACTIONAL, backups=2147483647, mvcc=false],
encryptionEnabled=false]
2020-05-15 10:36:06.367  INFO 14684 --- [ange-worker-#38]
o.a.i.i.p.cache.GridCacheProcessor       : Starting caches on local join
performed in 113 ms.
2020-05-15 10:36:07.323  INFO 14684 --- [-tcp-comm-0-#24]
o.a.i.s.c.tcp.TcpCommunicationSpi        : Established outgoing
communication connection [locAddr=/10.146.121.167:60942,
rmtAddr=azuswvlnx00687.corp.frk.com/172.30.197.5:47100]
2020-05-15 10:36:07.327  INFO 14684 --- [ange-worker-#38]
.c.d.d.p.GridDhtPartitionsExchangeFuture : Sending Single Message performed
in 955 ms.
2020-05-15 10:36:07.327  INFO 14684 --- [ange-worker-#38]
o.apache.ignite.internal.exchange.time   : Finished exchange init
[topVer=AffinityTopologyVersion [topVer=13, minorTopVer=0], crd=false]
2020-05-15 10:36:07.853  INFO 14684 --- [        sys-#39]
.c.d.d.p.GridDhtPartitionsExchangeFuture : Received full message, will
finish exchange [node=be10cd31-aed7-448a-8fec-60fd72a62313,
resVer=AffinityTopologyVersion [topVer=13, minorTopVer=0]]
2020-05-15 10:36:07.857  INFO 14684 --- [        sys-#39]
o.a.i.i.p.c.CacheAffinitySharedManager   : Affinity initialization on local
join performed in 4 ms.
2020-05-15 10:36:07.857  INFO 14684 --- [        sys-#39]
.c.d.d.p.GridDhtPartitionsExchangeFuture : Affinity changes applied in 4 ms.
2020-05-15 10:36:07.866  INFO 14684 --- [        sys-#39]
.c.d.d.p.GridDhtPartitionsExchangeFuture : Full map updating for 13 groups
performed in 9 ms.
2020-05-15 10:36:07.866  INFO 14684 --- [        sys-#39]
.c.d.d.p.GridDhtPartitionsExchangeFuture : Finish exchange future
[startVer=AffinityTopologyVersion [topVer=13, minorTopVer=0],
resVer=AffinityTopologyVersion [topVer=13, minorTopVer=0], err=null]
2020-05-15 10:36:07.872  INFO 14684 --- [        sys-#39]
o.a.i.i.p.cache.GridCacheProcessor       : Finish proxy initialization,
cacheName=ignite-sys-cache, localNodeId=0afce27c-9b6d-402f-807a-36d9c5a506b6
2020-05-15 10:36:07.873  INFO 14684 --- [        sys-#39]
.c.d.d.p.GridDhtPartitionsExchangeFuture : Completed partition exchange
[localNode=0afce27c-9b6d-402f-807a-36d9c5a506b6,
exchange=GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion
[topVer=13, minorTopVer=0], evt=NODE_JOINED, evtNode=TcpDiscoveryNode
[id=0afce27c-9b6d-402f-807a-36d9c5a506b6, addrs=[0:0:0:0:0:0:0:1,
10.146.121.167, 127.0.0.1, 192.168.0.5], sockAddrs=[/0:0:0:0:0:0:0:1:0,
/127.0.0.1:0, HYDNBK155967.euro.corp.frk.com/10.146.121.167:0,
/192.168.0.5:0], discPort=0, order=13, intOrder=0,
lastExchangeTime=1589519141364, loc=true, ver=2.7.6#20190911-sha1:21f7ca41,
isClient=true], done=true], topVer=AffinityTopologyVersion [topVer=13,
minorTopVer=0], durationFromInit=1617]
2020-05-15 10:36:07.873  INFO 14684 --- [ange-worker-#38]
.i.p.c.GridCachePartitionExchangeManager : Skipping rebalancing (no affinity
changes) [top=AffinityTopologyVersion [topVer=13, minorTopVer=0],
rebTopVer=AffinityTopologyVersion [topVer=-1, minorTopVer=0],
evt=NODE_JOINED, evtNode=0afce27c-9b6d-402f-807a-36d9c5a506b6, client=true]
2020-05-15 10:36:07.897  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : Performance suggestions for grid
(fix if possible)
2020-05-15 10:36:07.897  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : To disable, set
-DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
2020-05-15 10:36:07.897  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  :   ^-- Enable G1 Garbage Collector
(add '-XX:+UseG1GC' to JVM options)
2020-05-15 10:36:07.897  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  :   ^-- Specify JVM heap max size
(add '-Xmx<size>[g|G|m|M|k|K]' to JVM options)
2020-05-15 10:36:07.897  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  :   ^-- Set max direct memory size
if getting 'OOME: Direct buffer memory' (add
'-XX:MaxDirectMemorySize=<size>[g|G|m|M|k|K]' to JVM options)
2020-05-15 10:36:07.897  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  :   ^-- Disable processing of calls
to System.gc() (add '-XX:+DisableExplicitGC' to JVM options)
2020-05-15 10:36:07.897  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : Refer to this page for more
performance suggestions:
https://apacheignite.readme.io/docs/jvm-and-system-tuning
2020-05-15 10:36:07.897  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  :
2020-05-15 10:36:07.897  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  : To start Console Management &
Monitoring run ignitevisorcmd.{sh|bat}
2020-05-15 10:36:07.899  INFO 14684 --- [           main]
org.apache.ignite.internal.IgniteKernal  :

>>> +----------------------------------------------------------------------+
>>> Ignite ver. 2.7.6#20190911-sha1:21f7ca41c4348909e2fd26ccf59b5b2ce1f4474e
>>> +----------------------------------------------------------------------+
>>> OS name: Windows 10 10.0 amd64
>>> CPU(s): 8
>>> Heap: 3.5GB
>>> VM name: 14684@HYDNBK155967
>>> Local node [ID=0AFCE27C-9B6D-402F-807A-36D9C5A506B6, order=13,
>>> clientMode=true]
>>> Local node addresses: [HYDNBK155967.euro.corp.frk.com/0:0:0:0:0:0:0:1,
>>> HYDNBK155967.euro.corp.frk.com/10.146.121.167, /127.0.0.1, /192.168.0.5]
>>> Local ports: TCP:10800 TCP:47100 UDP:47400

2020-05-15 10:36:07.900  INFO 14684 --- [           main]
o.a.i.i.m.d.GridDiscoveryManager         : Topology snapshot [ver=13,
locNode=0afce27c, servers=2, clients=1, state=ACTIVE, CPUs=16,
offheap=9.0GB, heap=9.6GB]
2020-05-15 10:36:07.900  INFO 14684 --- [           main]
o.a.i.i.m.d.GridDiscoveryManager         :   ^-- Baseline [id=0, size=2,
online=2, offline=0]
2020-05-15 10:36:07.943  WARN 14684 --- [           main]
JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is
enabled by default. Therefore, database queries may be performed during view
rendering. Explicitly configure spring.jpa.open-in-view to disable this
warning
2020-05-15 10:36:08.050  INFO 14684 --- [           main]
o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService
'applicationTaskExecutor'
2020-05-15 10:36:08.253  INFO 14684 --- [           main]
o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 3000
(http) with context path ''
2020-05-15 10:36:08.255  INFO 14684 --- [           main]
org.example.springbootapi.App            : Started App in 36.559 seconds
(JVM running for 37.077)
2020-05-15 10:36:14.006  INFO 14684 --- [nio-3000-exec-1]
o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring
DispatcherServlet 'dispatcherServlet'
2020-05-15 10:36:14.006  INFO 14684 --- [nio-3000-exec-1]
o.s.web.servlet.DispatcherServlet        : Initializing Servlet
'dispatcherServlet'
2020-05-15 10:36:14.013  INFO 14684 --- [nio-3000-exec-1]
o.s.web.servlet.DispatcherServlet        : Completed initialization in 7 ms
2020-05-15 10:36:14.108 ERROR 14684 --- [ange-worker-#38]
.i.p.c.GridCachePartitionExchangeManager : Failed to process custom exchange
task: ClientCacheChangeDummyDiscoveryMessage
[reqId=82c4c9bc-860e-4393-a738-0e7e7ff1cc6e, cachesToClose=null,
startCaches=[NumberandDateFormatCache]]

org.apache.ignite.IgniteException: Spring application context resource is
not injected.
        at
org.apache.ignite.cache.store.jdbc.CacheJdbcPojoStoreFactory.create(CacheJdbcPojoStoreFactory.java:171)
~[ignite-core-2.7.6.jar:2.7.6]
        at
org.apache.ignite.cache.store.jdbc.CacheJdbcPojoStoreFactory.create(CacheJdbcPojoStoreFactory.java:100)
~[ignite-core-2.7.6.jar:2.7.6]
        at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.createCache(GridCacheProcessor.java:1565)
~[ignite-core-2.7.6.jar:2.7.6]
        at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:2153)
~[ignite-core-2.7.6.jar:2.7.6]
        at
org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.processClientCacheStartRequests(CacheAffinitySharedManager.java:438)
~[ignite-core-2.7.6.jar:2.7.6]
        at
org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.processClientCachesChanges(CacheAffinitySharedManager.java:637)
~[ignite-core-2.7.6.jar:2.7.6]
        at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.processCustomExchangeTask(GridCacheProcessor.java:391)
~[ignite-core-2.7.6.jar:2.7.6]
        at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.processCustomTask(GridCachePartitionExchangeManager.java:2489)
[ignite-core-2.7.6.jar:2.7.6]
        at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2634)
[ignite-core-2.7.6.jar:2.7.6]
        at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2553)
[ignite-core-2.7.6.jar:2.7.6]
        at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
[ignite-core-2.7.6.jar:2.7.6]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
akorensh akorensh
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling Cache Refresh

org.apache.ignite.IgniteException: Spring application context resource is not
injected.
        at
org.apache.ignite.cache.store.jdbc.CacheJdbcPojoStoreFactory.create(CacheJdbcPojoStoreFactory.java:171)

Take a look at the source:
https://github.com/apache/ignite/blob/feace547f83181853ca6db8626ab9ac6d372eb83/modules/core/src/main/java/org/apache/ignite/cache/store/jdbc/CacheJdbcPojoStoreFactory.java#L171


This means the app was started w/out an application context.

If you used Ignition.start() then this stuation might occur.

Are you using @SpringApplicationContextResource or any such resource.

I noticed that the app you sent over did not use the xml config.

Can you send over the piece of code that starts the app(s).

Can you try w/out spring boot to see that it works.
Use Ignition.start(xml config) --- don't inject via annotation

Then add spring boot.






--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
nithin91 nithin91
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling Cache Refresh

This post was updated on .
Hi

Thanks for the inputs.Issue got resolved after doing the following changes.
Attached you the updated code,client.xml(will be in src/resources folder)
and pom.xml file.
srcandpomupdated.zip
<http://apache-ignite-users.70518.x6.nabble.com/file/t2737/srcandpomupdated.zip

        1. Removed the following dependency from pom.xml
            <dependency>
                        <groupId>org.apache.ignite</groupId>
                        <artifactId>ignite-spring-boot-autoconfigure-ext</artifactId>
                        <version>1.0.0</version>
                </dependency>
        2. Modifying the IgniteConfig.class such that
                changing the method igniteInstance to return Ignite
                instance by calling
                Ignition.start("Ignite-Client.xml").
                Eariler it used to return IgniteConfiguration
                by using the following piece of code.Also earlier
                i haven't used IgniteClient.xml.
               
                IgniteConfiguration cfg = new IgniteConfiguration();

                // The node will be started as a client node.
                cfg.setClientMode(true);

                // Classes of custom Java logic will be transferred over the wire
from this app.
                cfg.setPeerClassLoadingEnabled(true);

                // Setting up an IP Finder to ensure the client can locate the
servers.
                TcpDiscoveryMulticastIpFinder ipFinder = new
TcpDiscoveryMulticastIpFinder();
                ipFinder.setAddresses(Collections.singletonList("Ipaddress"));
                cfg.setDiscoverySpi(new TcpDiscoverySpi().setIpFinder(ipFinder));
                // Starting the node
             
                cfg.setSqlSchemas("PIE");
                        retrun cfg
               

Although this seems to be working.
I have following queries with this approach.Can you please
provide your inputs/suggestions.



                        We are going to expose this SpringBoot
                        App as a RestFul API where we are trying to
                        refresh the Ignite Cache by sending  request to this
                        Application whenever the data in Oracle changes.
                        .
                        As we are trying to expose this APP as a API,
                        which means that this  Ignite Client Node
                        instance launched from Spring boot Application will
                        be active for ever.Now my queries are
                       
                        1. Is it ok to have a client node which
                        is continously in connected state forever.
                        If Ok,then how many such client node can exist.
                        2. Can this Application, be hosted on the  
                        one of the server node with different port.
                        3. Also will client node connect automatically
                        to the cluster if the client node has  lost
                        communication with server because if the client
                        node is down and if there is an API request
                        , then the application will throw an error.
                        4. Is this way of refreshing the cache is efficient.
                        If no, can you please suggest a better way.
                        5. Also can you please provide an example on the possibility
                        of how to make a
                        compute job
                        execute in a separate server which you have suggested earlier
                        https://apacheignite.readme.io/docs/compute-grid
        Also i have few more queries related to Ignite
        Behaviour.It would be really helpful if you can
        provide some inputs.
       
        1. Currently we are using
                ignite.cache("CacheName").loadCache(null) method
                to load  the cache initially and also increamentally as
                this method loads data very fast almost 0.1 million
                in 1 minute. For increamental load also  we are using this method
                by passing an additional argument i.e. Custom
                Sql  to load/update only the records that have created/changed.
                but what is happening is if the key doesn't
                exist it is inserting the data  but if the key is present
                it is not updating its value.
                Why is the value not updated if the key is present.
       
        2. To overcome this, i have to build my own custom
        cache store as mentioned in the following link
        https://apacheignite.readme.io/v1.9/docs/data-loading.
        But the problem with this approach is  i need to use JDBC
        result set to load the data which is very slow as
        we have to insert row by row.
        Is there a better way to update the keys.
       
       
        3. How to load data faster with custom cache store
        as loading the data using jdbc result set is very slow as
        we have to insert data row by row.
                               
                       
                       
                       
                       
                       
                       




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
nithin91 nithin91
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling Cache Refresh

Hi

Can anyone help me by providing inputs to the questions posted in my
previous message.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling Cache Refresh

In reply to this post by nithin91
Hello!

See below inline:

пт, 15 мая 2020 г. в 23:59, nithin91 <[hidden email]>:
Hi

Although this seems to be working.
I have following queries with this approach.Can you please
provide your inputs/suggestions.



                        We are going to expose this SpringBoot
                        App as a RestFul API where we are trying to
                        refresh the Ignite Cache by sending  request to this
                        Application whenever the data in Oracle changes.
                        .
                        As we are trying to expose this APP as a API,
                        which means that this  Ignite Client Node
                        instance launched from Spring boot Application will     
                        be active for ever.Now my queries are

                        1. Is it ok to have a client node which
                        is continously in connected state forever.
                        If Ok,then how many such client node can exist.
Yes, it's OK, you can have dozens of clients connected at all times.

                        2. Can this Application, be hosted on the   
                        one of the server node with different port.
Yes. No additional configuration is necessary.
 
                        3. Also will client node connect automatically
                        to the cluster if the client node has  lost
                        communication with server because if the client
                        node is down and if there is an API request
                        , then the application will throw an error.
Client will try to reconnect automatically. It will throw ClientNode IgniteClientDisconnectedException in the meantime.
 
                        4. Is this way of refreshing the cache is efficient.
                        If no, can you please suggest a better way.
What do you mean by refreshing cache?
 
                        5. Also take provide an example on the possibility
                        of  having the REST API calling a
                        compute job which would
                        execute in a separate server
                        https://apacheignite.readme.io/docs/compute-grid
        Also i have few more queries related to Ignite
        Behaviour.It would be really helpful if you can
        provide some inputs.

        1. Currently we are using
                ignite.cache("CacheName").loadCache(null) method
                to load  the cache initially and also increamentally as
                this method loads data very fast almost 0.1 million
                in 1 minute. For increamental load also  we are using this method
                by passing an additional argument i.e. Custom
                Sql  to load/update only the records that have created/changed.
                but what is happening is if the key doesn't
                exist it is inserting the data  but if the key is present
                it is not updating its value.
                Why is the value not updated if the key is present.
This is by design.
 

        2. To overcome this, i have to build my own custom
        cache store as mentioned in the following link
        https://apacheignite.readme.io/v1.9/docs/data-loading.
        But the problem with this approach is  i need to use JDBC
        result set to load the data which is very slow as
        we have to insert row by row.
        Is there a better way to update the keys.
Consider using DataStreamer.
 


        3. How to load data faster with custom cache store
        as loading the data which is very slow as
        we have to insert row by row.
See above.

Regards,
--
Ilya.
 













--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
nithin91 nithin91
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling Cache Refresh

This post was updated on .
Hi

Thanks For the inputs.

To the question is this way of refreshing the cache is efficient, i mean
refreshing the cache using the following process.Also let me know if there is any other efficient or simpler
approach to refresh the cache.

We have created a REST-API using Spring Boot which refreshes a particular
cache when a GET Request is triggered.

*Sample REST-API Url :*

http://domainname.com/api/v1/refresh?cachename="MyCache".

To refresh the cache, in the Spring Boot Application, we have started a
client node which stays connected to the cluster as long as the Application
forever.

Also can you please let me know whether DataSteamer API provided by Ignite
can be accessed through any of the thin clients like JAVA,Python,JS.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
nithin91 nithin91
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling Cache Refresh

Hi

Can anyone please help me with your inputs



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling Cache Refresh

In reply to this post by nithin91
Hello!

You will need to write your own code for that and expose it via compute tasks or service. Then you can invoke these via REST.

Regards,
--
Ilya Kasnacheev


вт, 19 мая 2020 г. в 20:52, nithin91 <[hidden email]>:
Hi

Thanks For the inputs.

To the question is this way of refreshing the cache is efficient, i mean
refreshing the cache using the following process.

We have created a REST-API using Spring Boot which refreshes a particular
cache when a GET Request is triggered.

*Sample REST-API Url :*

http://domainname.com/api/v1/refresh?cachename="MyCache".

To refresh the cache, in the Spring Boot Application, we have started a
client node which stays connected to the cluster as long as the Application
forever.

Also can you please let me know whether DataSteamer API provided by Ignite
can be accessed through any of the thin clients like JAVA,Python,JS.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
nithin91 nithin91
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling Cache Refresh

Thanks for the inputs.It is really helpful.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/