Evaluating Ignite, is this possible

classic Classic list List threaded Threaded
2 messages Options
exabrial exabrial
Reply | Threaded
Open this post in threaded view
|

Evaluating Ignite, is this possible

Hello everyone. We're thinking about using ignite.

We have a need to transform some data in a MySQL database. We have a table
like

Data Table:
id val0_cv_id val1_cv_id val2_cv_id
1  2              2               1
2  2              3               null
the cv_id columns are foreign keys to this table:

Code Values
id value_name
1  feature_z
2  feature_a
3  feature_b


The tool the data science team needs the data in the database to be in the
folowing format:

data_table_row_id has_feature_a has_feature_b has_feature_z
1                          1                    0                    1
2                          1                    1                    0

Essentially, rotating the data 90 degrees about the origin and "binarizing"
it.

Is it possible for Ignite to do this, and then allow third party tools (Like
a SQL client or Metabase or Tableau) to connect to the transformed Ignite
dataset?

I see in the documentation there is a preprocessor that does binarization,
but I'm unsure if the result dataset can be offered as a service.


Thank you!
-Jonathan



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Evaluating Ignite, is this possible

Hello!

I think that you could write a Cache Store loader that did this transformation before passing data to be put in cache.


Regards,
--
Ilya Kasnacheev


вт, 25 июн. 2019 г. в 00:39, exabrial <[hidden email]>:
Hello everyone. We're thinking about using ignite.

We have a need to transform some data in a MySQL database. We have a table
like

Data Table:
id val0_cv_id val1_cv_id val2_cv_id
1  2              2               1
2  2              3               null
the cv_id columns are foreign keys to this table:

Code Values
id value_name
1  feature_z
2  feature_a
3  feature_b


The tool the data science team needs the data in the database to be in the
folowing format:

data_table_row_id has_feature_a has_feature_b has_feature_z
1                          1                    0                    1
2                          1                    1                    0

Essentially, rotating the data 90 degrees about the origin and "binarizing"
it.

Is it possible for Ignite to do this, and then allow third party tools (Like
a SQL client or Metabase or Tableau) to connect to the transformed Ignite
dataset?

I see in the documentation there is a preprocessor that does binarization,
but I'm unsure if the result dataset can be offered as a service.


Thank you!
-Jonathan



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/