DataStreamers and Daemon-mode

classic Classic list List threaded Threaded
3 messages Options
cwaters cwaters
Reply | Threaded
Open this post in threaded view
|

DataStreamers and Daemon-mode

Hi all,

In Ignite 1.3.0-incubating I was able to populate a cache on my grid with data via a DataStreamer running in Daemon-mode. From Ignite 1.4 I get a "Cache doesn't exist".

I'm not sure if this is a issue with my approach to data-loading and is therefore a deliberate feature change? My approach so far has been to start-up a number of "data-nodes", then run the data-load via a separate Java process. I'm happy to take pointers on a better way.

I've reproduced the issue in fairly cutdown example code and you can see it at https://github.com/waters-colin/ignite-dataloading
You can observe the different behaviour by changing the $IGNITE_HOME as well as the ignite dependency version in the pom.

The Stack I get on error is:

Exception in thread "main" java.lang.IllegalStateException: Cache doesn't exist: city-cache
    at org.apache.ignite.internal.processors.cache.GridCacheProcessor.cacheConfiguration(GridCacheProcessor.java:3051)
    at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.<init>(DataStreamerImpl.java:240)
    at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.dataStreamer(DataStreamProcessor.java:171)
    at org.apache.ignite.internal.IgniteKernal.dataStreamer(IgniteKernal.java:2756)
    at ignite.example.CityBulkLoader.loadAll(CityBulkLoader.java:28)
    at ignite.example.DataLoadingApp.loadData(DataLoadingApp.java:45)
    at ignite.example.DataLoadingApp.main(DataLoadingApp.java:64)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

I can see that there was a check for isDaemon() added to GridCacheProcessor.start() CacheConfiguration loop ~ line 601 and that when this check is commented that I again can use the DataStreamer ok. Is it expected to be able to use an IgniteDataStreamer from code running with Ignition.setDaemon(true)?

All the best,
Colin.
vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|

Re: DataStreamers and Daemon-mode

Hi Colin,

You should use client mode instead of daemon. See this page for more details: https://apacheignite.readme.io/docs/clients-vs-servers

Daemon node doesn't have access to data, it can be used only for management purposes (e.g., collection metrics). We use this mode in Visor console, for example.

-Val
cwaters cwaters
Reply | Threaded
Open this post in threaded view
|

Re: DataStreamers and Daemon-mode

Thank you Val,

That worked well for me. I appreciate the pointer to the docs.

All the best,
Col.