How to get a instance of IgniteContext in spark-shell?

classic Classic list List threaded Threaded
7 messages Options
yaoqin yaoqin
Reply | Threaded
Open this post in threaded view
|

How to get a instance of IgniteContext in spark-shell?

How to get a instance of IgniteContext in spark-shell

 



 

mike-griggs mike-griggs
Reply | Threaded
Open this post in threaded view
|

Re: How to get a instance of IgniteContext in spark-shell?

You can't pass a file path String to IgniteConfiguration, it has either a default constructor

IgniteConfiguration()

or a constructor that takes another IgniteConfiguration instance (i.e., a it's a copy-constructor)

IgniteConfiguration(IgniteConfiguration config)

In Java I call the static method

Ignition.start("path-to-ignite-config.xml")
alexey.goncharuk alexey.goncharuk
Reply | Threaded
Open this post in threaded view
|

Re: How to get a instance of IgniteContext in spark-shell?

I assume you are using a 1.3 release, here you can either create an instance of IgniteConfiguration manually:

val cfg = new IgniteConfiguration()
// configure Ignite
val ic = new IgniteContext[Integer, Integer](sc, () => cfg)

or pass a path to the Ignite configuration directly to the IgniteContext constructor:
val ic = new IgniteContext[Integer, Integer](sc, "path/to/igntie-config.xml")

2015-08-28 1:52 GMT-07:00 endian675 <[hidden email]>:
You can't pass a file path String to IgniteConfiguration, it has either a
default constructor

IgniteConfiguration()

or a constructor that takes another IgniteConfiguration instance (i.e., a
it's a copy-constructor)

IgniteConfiguration(IgniteConfiguration config)

In Java I call the static method

Ignition.start("path-to-ignite-config.xml")



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-get-a-instance-of-IgniteContext-in-spark-shell-tp1180p1187.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

yaoqin yaoqin
Reply | Threaded
Open this post in threaded view
|

答复: How to get a instance of IgniteContext in spark-shell?

 



 

 

发件人: Alexey Goncharuk [mailto:[hidden email]]
发送时间: 2015829 3:13
收件人: [hidden email]
主题: Re: How to get a instance of IgniteContext in spark-shell

 

I assume you are using a 1.3 release, here you can either create an instance of IgniteConfiguration manually:

 

val cfg = new IgniteConfiguration()

// configure Ignite

val ic = new IgniteContext[Integer, Integer](sc, () => cfg)

 

or pass a path to the Ignite configuration directly to the IgniteContext constructor:

val ic = new IgniteContext[Integer, Integer](sc, "path/to/igntie-config.xml")

 

2015-08-28 1:52 GMT-07:00 endian675 <[hidden email]>:

You can't pass a file path String to IgniteConfiguration, it has either a
default constructor

IgniteConfiguration()

or a constructor that takes another IgniteConfiguration instance (i.e., a
it's a copy-constructor)

IgniteConfiguration(IgniteConfiguration config)

In Java I call the static method

Ignition.start("path-to-ignite-config.xml")



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-get-a-instance-of-IgniteContext-in-spark-shell-tp1180p1187.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

 

alexey.goncharuk alexey.goncharuk
Reply | Threaded
Open this post in threaded view
|

Re: 答复: How to get a instance of IgniteContext in spark-shell?

Spark was not able to locate the spring-related classes because spring integration is not included to the igntie-spark module dependencies. I will fix this in the upcoming 1.4 release.

For now you need to add the ignite-spring module dependency to your spark shell execution arguments.
dsetrakyan dsetrakyan
Reply | Threaded
Open this post in threaded view
|

Re: 答复: How to get a instance of IgniteContext in spark-shell?



On Mon, Aug 31, 2015 at 4:57 PM, alexey.goncharuk <[hidden email]> wrote:
Spark was not able to locate the spring-related classes because spring
integration is not included to the igntie-spark module dependencies. I will
fix this in the upcoming 1.4 release.

For now you need to add the ignite-spring module dependency to your spark
shell execution arguments.

Alexey, can you please provide an example or a link to the documentation describing how to do it?




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-get-a-instance-of-IgniteContext-in-spark-shell-tp1180p1228.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

alexey.goncharuk alexey.goncharuk
Reply | Threaded
Open this post in threaded view
|

Re: 答复: How to get a instance of IgniteContext in spark-shell?


2015-08-31 17:35 GMT-07:00 Dmitriy Setrakyan <[hidden email]>:


On Mon, Aug 31, 2015 at 4:57 PM, alexey.goncharuk <[hidden email]> wrote:
Spark was not able to locate the spring-related classes because spring
integration is not included to the igntie-spark module dependencies. I will
fix this in the upcoming 1.4 release.

For now you need to add the ignite-spring module dependency to your spark
shell execution arguments.

Alexey, can you please provide an example or a link to the documentation describing how to do it?




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-get-a-instance-of-IgniteContext-in-spark-shell-tp1180p1228.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.