[Bug] [Seatunnel Transform] FieldMapper是否支持对象.属性的映射关系

[oldwolf1984]

[Bug] [Seatunnel Transform] FieldMapper是否支持对象.属性的映射关系

[liunaijie]

you can use sql transform and inner query feature.
the query like select id, data.MANTER as new_col ...

[oldwolf1984]

我的源库和目标库都是MongoDB，也可以使用sql么？

[oldwolf1984]

使用sql 确实可以解决了，但是又有了一个新的问题

[liunaijie]

使用sql 确实可以解决了，但是又有了一个新的问题

when you use sql tranfrom.
the field_mapper will ignore. the transform result will same with you query result.

You can change to console sink to verify the result struct.

And you can use multiple tranfroms to generate the result you wanted,
like

transform {
  sql {
       query = ""
       result_table_name = "sql_res1"
   }

   sql {
      query = ""
       source_table_name = "sql_res1"
   }

}

[oldwolf1984]

你好 我按照上面发的写了两个sql标签，执行的时候后台有报错了
024-05-23 15:35:06,093 ERROR [o.a.s.c.s.SeaTunnel ] [main] - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:202)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.engine.common.exception.JobDefineCheckException: The source/transform(MongoDB) is not configured with 'result_table_name' option
at org.apache.seatunnel.engine.core.parse.ConfigParserUtil.checkExistTableId(ConfigParserUtil.java:237)
at org.apache.seatunnel.engine.core.parse.ConfigParserUtil.checkComplexGraph(ConfigParserUtil.java:155)
at org.apache.seatunnel.engine.core.parse.ConfigParserUtil.checkGraph(ConfigParserUtil.java:75)
at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:179)
at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:88)
at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:156)
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:149)
... 2 more
Caused by: org.apache.seatunnel.api.configuration.util.OptionValidationException: ErrorCode:[API-02], ErrorDescription:[Option item validate failed] - The option("result_table_name") is incorrectly configured, please refer to the doc: When result_table_name is not specified, the data processed by this plugin will not be registered as a data set (dataStream/dataset) that can be directly accessed by other plugins, or called a temporary table (table)When result_table_name is specified, the data processed by this plugin will be registered as a data set (dataStream/dataset) that can be directly accessed by other plugins, or called a temporary table (table) . The data set (dataStream/dataset) registered here can be directly accessed by other plugins by specifying source_table_name .
... 9 more

[oldwolf1984]

[liunaijie]

1. the gived config is not complete
   it should be ( still ignore som config) :

source  {
   xxx {
       result_table_name = "source_result"
       ....
   }
}

transform {
  sql {
       query = ""
       source_table_name = "source_result"
       result_table_name = "sql_res1"    
   }

   sql {
      query = ""
       source_table_name = "sql_res1"
       result_table_name = "sql_res2"  // your exception is this config missing parameters
   }

}

sink {

   xxx {
      source_table_name = "sql_res2"
      ......
   }

}

2. in your second query, you want query from sink table, this is not right. you can only query from the source table.

data flow is source -> transform -> sink

and also not support query with multiple table using like join

3. for your case, your can try (i am not sure it can work, just suggestion)
   a. select data.A as data.B from source
   b. using json tranform