Apache Tez support for Ignite

classic Classic list List threaded Threaded
6 messages Options
ravi ravi
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Apache Tez support for Ignite

This post has NOT been accepted by the mailing list yet.
Hi,
 We wanted to use apache hive 2.x LLAP feature with Tez on top of Ignite. Is it supported?. If so can you share some github links with  examples?.
  Similarly hive on spark with ignite will it work?.

Regards
Ravi.P
slava.koptilin slava.koptilin
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Apache Tez support for Ignite

Hi Ravi,

Please properly subscribe to the mailing list so that the community can
receive email notifications for your messages. To subscribe, send empty
email to user-subscribe@ignite.apache.org and follow simple instructions in
the reply.

> Similarly hive on spark with ignite will it work?.
Apache Ignite implements Spark Resilient Distributed Datasets (RDD), and so can be used with Spark Jobs, Workers and applications.
https://apacheignite-fs.readme.io/docs/ignite-for-spark

Thanks!
ravi ravi
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Apache Tez support for Ignite

Hi
 Thanks for the details. I have sent the email to subscribe now and got confirmation back. The reason i have asked the question about Tez and LLAP is apache hive community has deprecated the MR as execution engine and moving towards Tez and LLAP, Will Ignite have equivalent In Memory Tez/LLAP implementation similar to In memory map reduce?.

Regards
Ravi
vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Apache Tez support for Ignite

Ravi,

Have you seen the Hadoop Accelerator? https://apacheignite-fs.readme.io/docs/hadoop-accelerator

It also provides custom implementation of MR engine.

-Val
ravi ravi
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Apache Tez support for Ignite

Hi Val,
 I have seen the hadoop accelerator link you shared.

If i have to choose from the following combination for the performance reason which one should i choose? why? and when should i choose?

1) Hive on MR(Standard Hadoop MR)
2) Hive on Ignite MR
3) Hive on Spark
4) Hive on Tez
5) Hive on Tez+LLAP



Regards
Ravi.P
vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Apache Tez support for Ignite

I doubt there is a general answer, as it will depend on your particular use case. Ignite MR is very easy to install, so I think you should just try it out and compare with other options.

-Val
Loading...