substrait-io/substrait-java

substrait-java's Issues

Time type precision check failed

Hi, I noticed that the time type parsed from calcite will have a precision value check before generating substrait plan.

substrait-java/isthmus/src/main/java/io/substrait/isthmus/TypeConverter.java

Lines 67 to 87 in 5045aa3

	case TIME -> {
	if (type.getPrecision() != 6) {
	throw new UnsupportedOperationException(
	"unsupported time precision " + type.getPrecision());
	}
	yield creator.TIME;
	}
	case TIMESTAMP -> {
	if (type.getPrecision() != 6) {
	throw new UnsupportedOperationException(
	"unsupported timestamp precision " + type.getPrecision());
	}
	yield creator.TIMESTAMP;
	}
	case TIMESTAMP_WITH_LOCAL_TIME_ZONE -> {
	if (type.getPrecision() != 6) {
	throw new UnsupportedOperationException(
	"unsupported timestamptz precision " + type.getPrecision());
	}
	yield creator.TIMESTAMP_TZ;
	}

I added some timestamp related sql tests(sql ddl like "CREATE TABLE test(col_date DATE, col_time TIME(6), col_timestamp TIMESTAMP(6));"）and found time type precision value parsed from calcite will not be greater than three, will always make this check fail. Is there a bug in the use of time types?

unsupported timestamp precision 3
java.lang.UnsupportedOperationException: unsupported timestamp precision 3
	at io.substrait.isthmus.TypeConverter.convert(TypeConverter.java:77)
	at io.substrait.isthmus.TypeConverter.convert(TypeConverter.java:109)
	at io.substrait.isthmus.TypeConverter.toNamedStruct(TypeConverter.java:37)
	at io.substrait.isthmus.SubstraitRelVisitor.visit(SubstraitRelVisitor.java:76)
	at io.substrait.isthmus.SubstraitRelVisitor.visit(SubstraitRelVisitor.java:32)
	at io.substrait.isthmus.RelNodeVisitor.reverseAccept(RelNodeVisitor.java:85)
	at io.substrait.isthmus.SubstraitRelVisitor.apply(SubstraitRelVisitor.java:319)
	at io.substrait.isthmus.SubstraitRelVisitor.visit(SubstraitRelVisitor.java:122)
	at io.substrait.isthmus.SubstraitRelVisitor.visit(SubstraitRelVisitor.java:32)
	at io.substrait.isthmus.RelNodeVisitor.reverseAccept(RelNodeVisitor.java:95)
	at io.substrait.isthmus.SubstraitRelVisitor.apply(SubstraitRelVisitor.java:319)
	at io.substrait.isthmus.SubstraitRelVisitor.convert(SubstraitRelVisitor.java:340)
	at io.substrait.isthmus.SubstraitRelVisitor.convert(SubstraitRelVisitor.java:332)
	at io.substrait.isthmus.SqlToSubstrait.lambda$executeInner$0(SqlToSubstrait.java:120)
	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
	at io.substrait.isthmus.SqlToSubstrait.executeInner(SqlToSubstrait.java:113)
	at io.substrait.isthmus.SqlToSubstrait.execute(SqlToSubstrait.java:81)

Update Readme.md with build instructions

Contribution guidelines does state that the project uses JDK 17 and Gradle, however detailed developer environment setup, build and test instructions will be helpful for beginners.

failed query 15 in tpch_smoke.sh

query 15 failed due to comments in .sql file. CommandLine cannot recognize the comments which start from --
https://github.com/substrait-io/substrait-java/blob/main/isthmus/src/test/resources/tpch/queries/15.sql

Can't load substrait-java project in IntlliJ IDEA

hi, folks

When i open substrait-java project in IntlliJ IDEA, output below error message.
How should I solve this problem? Please help, thanks!

A problem occurred configuring root project 'substrait'.
> Could not resolve all files for configuration ':classpath'.
   > Could not find spotless-lib-extra-2.30.0.jar (com.diffplug.spotless:spotless-lib-extra:2.30.0).
     Searched in the following locations:
         https://plugins.gradle.org/m2/com/diffplug/spotless/spotless-lib-extra/2.30.0/spotless-lib-extra-2.30.0.jar
   > Could not find spotless-lib-2.30.0.jar (com.diffplug.spotless:spotless-lib:2.30.0).
     Searched in the following locations:
         https://plugins.gradle.org/m2/com/diffplug/spotless/spotless-lib/2.30.0/spotless-lib-2.30.0.jar

Possible solution:
 - Declare repository providing the artifact, see the documentation at https://docs.gradle.org/current/userguide/declaring_repositories.html

[BUILD] Duplicate automated releases from core repo to substrait-java

[ISTHMUS] [TPCDS] Add support for is null and not is null predicates

For example in q44 has the expression: ss_customer_sk is null

Queries in tpcds having this issue: [40, 44, 49, 75, 76, 77, 78, 80, 93, 97]

How to reproduce:
Add a test to TpcdsQueryNoValidation like this:

@ParameterizedTest
  @ValueSource(ints = {44})
  public void tpcdsQ12(int query) throws Exception {
    testQuery(query);
  }

On fixing, please move the fixed queries from the tpcdsFailure list to the tpcdsSuccess list

[ISTHMUS] [TPCDS] Error in rank call

In q67 has the expression: rank() over (partition by i_category order by sumsales desc)
This is after applying #56

Error Statcktrace:

java.lang.UnsupportedOperationException: RexOver not supported
	at io.substrait.isthmus.expression.RexExpressionConverter.visitOver(RexExpressionConverter.java:76)
	at io.substrait.isthmus.expression.RexExpressionConverter.visitOver(RexExpressionConverter.java:17)
	at org.apache.calcite.rex.RexOver.accept(RexOver.java:122)
	at io.substrait.isthmus.SubstraitRelVisitor.toExpression(SubstraitRelVisitor.java:68)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992)
...

How to reproduce:
Add a test to TpcdsQueryNoValidation like this:

@ParameterizedTest
  @ValueSource(ints = {67})
  public void tpcdsQ12(int query) throws Exception {
    testQuery(query);
  }

On fixing, please move the fixed queries from the tpcdsFailure list to the tpcdsSuccess list

Calcite optimizations in Isthmus/Substrait plans

Based on our experiments with Substrait plans and conversations with the Substrait folks highlighted on the Google Group, it looks like the Substrait plans currently generated by Isthmus are unoptimized.

We're actively looking to use Substrait plans infused with Calcite optimizations for our internal projects. Please keep us updated on when support for the same could be added, so that we can incorporate and experiment with the optimized plans.

Additionally, if there're any other (ideally pythonic) tools/APIs that successfully generate optimized Substrait plans (from any other query optimizers) now or in the future, please let us know and we'd love to utilize them for our projects.

Thank you very much for the continued support!

Reflection-based GenericRoundtripTest for ExpressionCreator methods (and failures)

(disclaimer this is me poking at the project to learn about it)

I wrote a simple reflection based test, that lists all methods in ExpressionCreator and for those using simple types (primitives and things like Dates or BigDecimal etc). I generate a synthetic set of params and invoke the method doing a roundtrip conversion (lifted from LiteralRoundtripTest which was only covering decimal so far). Two questions:

1. Do you agree it makes sense to have something like this in the codebase, to automatically cover (at least a few) cases as they get added to the system? Substrait strikes me as a project that will grow pretty broad and automating high-coverage tests seems worthy. (happy to post code if you agree it makes sense).

2. I saw several of the methods failing on the roundtrip because the "nullability" seems not retained correctly. Is this by design and we should ignore it in the comparison or is it a bug? (example below with timestamp method).

Invoking timestamp ([boolean, long]) with values ([true, -5750464908882337892])

expected: <TimestampLiteral{nullable=true, value=-5750464908882337892}> but was: <TimestampLiteral{nullable=false, value=-5750464908882337892}>
Expected :TimestampLiteral{nullable=true, value=-5750464908882337892}
Actual   :TimestampLiteral{nullable=false, value=-5750464908882337892}

Release build failing

I merged the release patch it looks like the release build is failing.

Additionally, it looks like it is tagging version on each merge (maybe) as opposed to once a week...

Failure here: https://github.com/substrait-io/substrait-java/actions/runs/3544635664/jobs/5952198598

@davisusanibar, can you take a look?

[ISTHMUS] support order by, limit offset clause in substrait to sql conversion

Currently, the converter from substrait rel plan to calcite relnode plan does not support SortRel / FetchRel, and would not produce sql with order by clause, limit offset clause.

This ticket is to track this missing functionality.

[ISTHMUS] Support correlated or non-correlated subquery in substrait plan to sql conversion

The converter from substrait plan to sql currently does not support a subquery, either in the form of correlated or non-correlated subquery.

This ticket is to track this missing functionality.

[ISTHMUS] Add support for set operations

Add support for set operations UNION and MINUS in SQL -> Substrait Proto -> Substrait POJO. This would involve adding implementation of methods in SubstraitRelVisitor, RelCopyOnWriteVisitor, ProtoRelConverter, and RelProtoConverter. Queries such as these should roundtrip between SQL, Substrait Proto, and Substrait POJO:

select p_partkey as partkey from part where p_partkey > cast(100 as bigint) 
union select l_partkey as partkey from lineitem where l_orderkey > cast(100 as bigint);
select p_partkey as partkey from part where p_partkey > cast(100 as bigint) 
minus select l_partkey as partkey from lineitem where l_orderkey > cast(100 as bigint);

failed query q8 q12 in tpch_smoke.sh

Below is error log of q8, q12 is almost same.

Processing tpc-h query, 8
select o_year, sum(case when nation = 'EGYPT' then volume else 0 end) / sum(volume) as mkt_share from ( select extract(year from o.o_orderdate) as o_year, l.l_extendedprice 1.txt 1y.txt count.txt date.txt decimal.txt filter.txt null.txt smoke.sh str.txt tpch_smoke.sh where.txt (1 - l.l_discount) as volume, n2.n_name as nation from "part" p, "supplier" s, "lineitem" l, "orders" o, "customer" c, "nation" n1, "nation" n2, "region" r where p.p_partkey = l.l_partkey and s.s_suppkey = l.l_suppkey and l.l_orderkey = o.o_orderkey and o.o_custkey = c.c_custkey and c.c_nationkey = n1.n_nationkey and n1.n_regionkey = r.r_regionkey and r.r_name = 'MIDDLE EAST' and s.s_nationkey = n2.n_nationkey and o.o_orderdate between date '1995-01-01' and date '1996-12-31' and p.p_type = 'PROMO BRUSHED COPPER' ) as all_nations group by o_year order by o_year
Exception in thread "main" java.lang.ExceptionInInitializerError
        at org.apache.calcite.linq4j.tree.BlockBuilder.<clinit>(BlockBuilder.java:52)
        at org.apache.calcite.rex.RexExecutorImpl.compile(RexExecutorImpl.java:83)
        at org.apache.calcite.rex.RexExecutorImpl.compile(RexExecutorImpl.java:68)
        at org.apache.calcite.rex.RexExecutorImpl.reduce(RexExecutorImpl.java:132)
        at org.apache.calcite.rex.RexSimplify.simplifyCast(RexSimplify.java:2216)
        at org.apache.calcite.rex.RexSimplify.simplify(RexSimplify.java:289)
        at org.apache.calcite.rex.RexSimplify.simplifyList(RexSimplify.java:662)
        at org.apache.calcite.rex.RexSimplify.simplifyComparison(RexSimplify.java:506)
        at org.apache.calcite.rex.RexSimplify.simplifyComparison(RexSimplify.java:498)
        at org.apache.calcite.rex.RexSimplify.simplify(RexSimplify.java:307)
        at org.apache.calcite.rex.RexSimplify.simplifyCase(RexSimplify.java:1138)
        at org.apache.calcite.rex.RexSimplify.simplify(RexSimplify.java:285)
        at org.apache.calcite.rex.RexSimplify.simplifyUnknownAs(RexSimplify.java:248)
        at org.apache.calcite.rex.RexSimplify.simplifyPreservingType(RexSimplify.java:187)
        at org.apache.calcite.rex.RexSimplify.simplifyPreservingType(RexSimplify.java:182)
        at org.apache.calcite.tools.RelBuilder.project_(RelBuilder.java:1947)
        at org.apache.calcite.tools.RelBuilder.project_(RelBuilder.java:1938)
        at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1797)
        at org.apache.calcite.tools.RelBuilder.projectNamed(RelBuilder.java:2088)
        at org.apache.calcite.sql2rel.SqlToRelConverter.createAggImpl(SqlToRelConverter.java:3289)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertAgg(SqlToRelConverter.java:3192)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:738)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:664)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3589)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:589)
        at io.substrait.isthmus.SqlToSubstrait.lambda$sqlToRelNode$1(SqlToSubstrait.java:153)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
        at java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:720)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575)
        at java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
        at java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616)
        at java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:622)
        at java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:627)
        at io.substrait.isthmus.SqlToSubstrait.sqlToRelNode(SqlToSubstrait.java:162)
        at io.substrait.isthmus.SqlToSubstrait.executeInner(SqlToSubstrait.java:112)
        at io.substrait.isthmus.SqlToSubstrait.execute(SqlToSubstrait.java:81)
        at io.substrait.isthmus.PlanEntryPoint.call(PlanEntryPoint.java:41)
        at io.substrait.isthmus.PlanEntryPoint.call(PlanEntryPoint.java:15)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
        at picocli.CommandLine.access$1300(CommandLine.java:145)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
        at picocli.CommandLine.execute(CommandLine.java:2078)
        at io.substrait.isthmus.PlanEntryPoint.main(PlanEntryPoint.java:49)
Caused by: java.lang.RuntimeException: Unknown field 'FALSE' in class class java.lang.Boolean
        at org.apache.calcite.linq4j.tree.Types.getField(Types.java:110)
        at org.apache.calcite.linq4j.tree.Types.getField(Types.java:121)
        at org.apache.calcite.linq4j.tree.Expressions.field(Expressions.java:849)
        at org.apache.calcite.linq4j.tree.OptimizeShuttle.<clinit>(OptimizeShuttle.java:47)
        ... 48 more
Caused by: java.lang.NoSuchFieldException: FALSE
        at java.lang.Class.getField(DynamicHub.java:1103)
        at org.apache.calcite.linq4j.tree.Types.getField(Types.java:108)
        ... 51 more

[ISTHMUS] [TPCDS] Support date arithmetic expressions

For example in q72 has the expression: d3.d_date > d1.d_date + 5

Queries in tpcds having this issue: [72]

How to reproduce:
Add a test to TpcdsQueryNoValidation like this:

@ParameterizedTest
  @ValueSource(ints = {72})
  public void tpcdsQ72(int query) throws Exception {
    testQuery(query);
  }

On fixing, please move the fixed queries from the tpcdsFailure list to the tpcdsSuccess list

Add back task of init substrait submodule to fix build issue.

The commit 83abf91 removed the submodule init task. As such, ./gradlew build would fail due to missing source codes in substrait repo.

 error: package io.substrait.proto does not exist
import io.substrait.proto.JoinRel;

83abf91

Remove preview features

As discussed in the PR when it was on the other project, whether we go with Java 17 minimum or something older, either way we should probably not make use of preview features.

[ISTHMUS] [TPCDS] Support Grouping

For example:

• in q27 has the expression: group by rollup (i_item_id, s_state)
• in q36 additionally has rank() over (partition by grouping(i_category)+grouping(i_class)..)

Queries in tpcds having this issue: [27, 36, 86]

How to reproduce:
Add a test to TpcdsQueryNoValidation like this:

@ParameterizedTest
  @ValueSource(ints = {27})
  public void tpcdsQ27(int query) throws Exception {
    testQuery(query);
  }

On fixing, please move the fixed queries from the tpcdsFailure list to the tpcdsSuccess list

[ISTHMUS] Calcite literal with Sarg fails to generate Substrait plan

Calcite produces RexLiteral with Sarg to handle ORed together column value comparison. This Sarg encapsulates multiple literal values, but our code assumes there will be a single value. For example, tpc-h query 12 fails:

select
  l.l_shipmode,
  sum(case
    when o.o_orderpriority = '1-URGENT'
      or o.o_orderpriority = '2-HIGH'
      then 1
    else 0
  end) as high_line_count,
  sum(case
    when o.o_orderpriority <> '1-URGENT'
      and o.o_orderpriority <> '2-HIGH'
      then 1
    else 0
  end) as low_line_count
from
  "orders" o,
  "lineitem" l
where
  o.o_orderkey = l.l_orderkey
  and l.l_shipmode in ('TRUCK', 'REG AIR')
  and l.l_commitdate < l.l_receiptdate
  and l.l_shipdate < l.l_commitdate
  and l.l_receiptdate >= date '1994-01-01'
  and l.l_receiptdate < date '1994-01-01' + interval '1' year
group by
  l.l_shipmode
order by
  l.l_shipmode

This query ends up with a RexLiteral with these values: Sarg[_ISO-8859-1'1-URGENT ', _ISO-8859-1'2-HIGH '] ending up producing the following exception:

java.lang.UnsupportedOperationException: Unable to handle char type: Sarg[_ISO-8859-1'1-URGENT       ', _ISO-8859-1'2-HIGH         ']
	at io.substrait.isthmus.expression.LiteralConverter.convert(LiteralConverter.java:96)
	at io.substrait.isthmus.expression.RexExpressionConverter.visitLiteral(RexExpressionConverter.java:92)
	at io.substrait.isthmus.expression.RexExpressionConverter.visitLiteral(RexExpressionConverter.java:21)
	at org.apache.calcite.rex.RexLiteral.accept(RexLiteral.java:1217)
	at io.substrait.isthmus.expression.RexExpressionConverter.lambda$visitCall$1(RexExpressionConverter.java:81)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at io.substrait.isthmus.expression.RexExpressionConverter.visitCall(RexExpressionConverter.java:82)
	at io.substrait.isthmus.expression.RexExpressionConverter.visitCall(RexExpressionConverter.java:21)
	at org.apache.calcite.rex.RexCall.accept(RexCall.java:189)
	at io.substrait.isthmus.expression.RexExpressionConverter.lambda$visitCall$0(RexExpressionConverter.java:57)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575)
	at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
	at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616)
	at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:622)
	at java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:627)
	at io.substrait.isthmus.expression.CallConverters.lambda$static$1(CallConverters.java:51)
	at io.substrait.isthmus.expression.CallConverters$SimpleCallConverter.convert(CallConverters.java:83)
	at io.substrait.isthmus.expression.RexExpressionConverter.visitCall(RexExpressionConverter.java:57)
	at io.substrait.isthmus.expression.RexExpressionConverter.visitCall(RexExpressionConverter.java:21)
	at org.apache.calcite.rex.RexCall.accept(RexCall.java:189)
	at io.substrait.isthmus.SubstraitRelVisitor.toExpression(SubstraitRelVisitor.java:76)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575)
	at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
	at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616)
	at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:622)
	at java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:627)
	at io.substrait.isthmus.SubstraitRelVisitor.visit(SubstraitRelVisitor.java:131)
	at io.substrait.isthmus.SubstraitRelVisitor.visit(SubstraitRelVisitor.java:36)
	at io.substrait.isthmus.RelNodeVisitor.reverseAccept(RelNodeVisitor.java:95)
	at io.substrait.isthmus.SubstraitRelVisitor.apply(SubstraitRelVisitor.java:317)
	at io.substrait.isthmus.SubstraitRelVisitor.visit(SubstraitRelVisitor.java:192)
	at io.substrait.isthmus.SubstraitRelVisitor.visit(SubstraitRelVisitor.java:36)
	at io.substrait.isthmus.RelNodeVisitor.reverseAccept(RelNodeVisitor.java:113)
	at io.substrait.isthmus.SubstraitRelVisitor.apply(SubstraitRelVisitor.java:317)
	at io.substrait.isthmus.SubstraitRelVisitor.visit(SubstraitRelVisitor.java:243)
	at io.substrait.isthmus.SubstraitRelVisitor.visit(SubstraitRelVisitor.java:36)
	at io.substrait.isthmus.RelNodeVisitor.reverseAccept(RelNodeVisitor.java:109)
	at io.substrait.isthmus.SubstraitRelVisitor.apply(SubstraitRelVisitor.java:317)
	at io.substrait.isthmus.SubstraitRelVisitor.convert(SubstraitRelVisitor.java:334)
	at io.substrait.isthmus.SubstraitRelVisitor.convert(SubstraitRelVisitor.java:326)
	at io.substrait.isthmus.SqlToSubstrait.lambda$executeInner$0(SqlToSubstrait.java:114)
	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
	at io.substrait.isthmus.SqlToSubstrait.executeInner(SqlToSubstrait.java:107)
	at io.substrait.isthmus.SqlToSubstrait.execute(SqlToSubstrait.java:80)
	at io.substrait.isthmus.TpchQueryNoValidation.tpch(TpchQueryNoValidation.java:18)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:688)
	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:92)
	at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:210)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:206)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:131)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:65)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask$DefaultDynamicTestExecutor.execute(NodeTestTask.java:212)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask$DefaultDynamicTestExecutor.execute(NodeTestTask.java:192)
	at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:139)
	at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.lambda$execute$2(TestTemplateTestDescriptor.java:107)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$15$1.accept(ReferencePipeline.java:541)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:107)
	at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:42)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:75)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at jdk.proxy1/jdk.proxy1.$Proxy2.stop(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)

[ISTHMUS] [TPCDS] Fix Concat scalar function support

For example:

• q5 has the expression: 'web_site' || web_site_id as id
• q66 has the expression: 'DHL' || ',' || 'BARIAN' as ship_carriers
• q80 has expression using ||

Implicit casting is not

Queries in tpcds having this issue: [5, 66, 80, 84]

How to reproduce:
Add a test to TpcdsQueryNoValidation like this:

@ParameterizedTest
  @ValueSource(ints = {5})
  public void tpcdsQ5(int query) throws Exception {
    testQuery(query);
  }

On fixing, please move the fixed queries from the tpcdsFailure list to the tpcdsSuccess list

[ISTHMUS] Add aggregation function MIN/MAX support

Current code does not support aggregation MIN/MAX in sql -> substrait plan conversion. As such, tpc-h q2 and q15 would fail with error like "Unable to find binding for call MIN($0)".

Two parts of places probably need fix:

• substrait spec's function yaml files do not declare MIN/MAX.
• substrait-java code does not have define function mapping between substrait function and Calcite SqlOperator.

Open this ticket to track the missing MIN/MAX support.

Implement ExtendedExpression

substrait-io/substrait#405 added an ExtendedExpression message to represent standalone expressions (expressions that are not part of plans). It would be nice to have this implemented in the Substrait Java library.

[CORE] Create a README for the top-level repository.

[Isthmus] Output substrait plan in binary format

Currently Isthmus only supports outputting the substrait plan in json format. It would be great to have the options of getting the binary output as well.

[ISTHMUS] Cannot upgrade to substrait 0.9.0

Looks like there was some breaking change in more recent substrait release. When attempting to upgrade substrait-java to latest substrait (v 0.9.0), I get the following exception when running the tpch tests:

java.lang.ExceptionInInitializerError
	at io.substrait.isthmus.TpchQueryNoValidation.tpch(TpchQueryNoValidation.java:15)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:688)
	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:92)
	at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:210)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:206)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:131)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:65)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask$DefaultDynamicTestExecutor.execute(NodeTestTask.java:212)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask$DefaultDynamicTestExecutor.execute(NodeTestTask.java:192)
	at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:139)
	at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.lambda$execute$2(TestTemplateTestDescriptor.java:107)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$15$1.accept(ReferencePipeline.java:541)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:107)
	at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:42)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:75)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at jdk.proxy1/jdk.proxy1.$Proxy2.stop(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
Caused by: java.lang.RuntimeException: Failure while parsing /functions_arithmetic.yaml
	at io.substrait.function.SimpleExtension.load(SimpleExtension.java:605)
	at io.substrait.function.SimpleExtension.lambda$load$1(SimpleExtension.java:571)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:720)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575)
	at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
	at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616)
	at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:622)
	at java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:627)
	at io.substrait.function.SimpleExtension.load(SimpleExtension.java:576)
	at io.substrait.function.SimpleExtension.loadDefaults(SimpleExtension.java:558)
	at io.substrait.isthmus.SqlConverterBase.<clinit>(SqlConverterBase.java:76)
	... 136 more
Caused by: com.fasterxml.jackson.databind.exc.ValueInstantiationException: Cannot construct instance of `io.substrait.function.ImmutableSimpleExtension$AggregateFunctionVariant`, problem: Cannot build AggregateFunctionVariant, some of required attributes are not set [intermediate]
 at [Source: (sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream); line: 721, column: 7] (through reference chain: io.substrait.function.ImmutableSimpleExtension$FunctionSignatures$Json["aggregate_functions"]->java.util.ArrayList[5]->io.substrait.function.ImmutableSimpleExtension$AggregateFunction$Json["impls"]->java.util.ArrayList[0])
	at app//com.fasterxml.jackson.databind.exc.ValueInstantiationException.from(ValueInstantiationException.java:47)
	at app//com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:1907)
	at app//com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.wrapAsJsonMappingException(StdValueInstantiator.java:587)
	at app//com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.rewrapCtorProblem(StdValueInstantiator.java:610)
	at app//com.fasterxml.jackson.databind.deser.std.StdValueInstantiator._createUsingDelegate(StdValueInstantiator.java:647)
	at app//com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createUsingDelegate(StdValueInstantiator.java:306)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1397)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:362)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:195)
	at app//com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:355)
	at app//com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244)
	at app//com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28)
	at app//com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:324)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:225)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:197)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1398)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:362)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:195)
	at app//com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:355)
	at app//com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244)
	at app//com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28)
	at app//com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:324)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:225)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:197)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1398)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:362)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:195)
	at app//com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
	at app//com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4593)
	at app//com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3585)
	at app//io.substrait.function.SimpleExtension.load(SimpleExtension.java:586)
	... 149 more
Caused by: java.lang.IllegalStateException: Cannot build AggregateFunctionVariant, some of required attributes are not set [intermediate]
	at io.substrait.function.ImmutableSimpleExtension$AggregateFunctionVariant$Builder.build(ImmutableSimpleExtension.java:3224)
	at io.substrait.function.ImmutableSimpleExtension$AggregateFunctionVariant.fromJson(ImmutableSimpleExtension.java:2950)
	at jdk.internal.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at com.fasterxml.jackson.databind.introspect.AnnotatedMethod.call1(AnnotatedMethod.java:107)
	at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator._createUsingDelegate(StdValueInstantiator.java:631)
	... 177 more

[ISTHMUS] [TPCDS] Add support for reference to Aggregate expr in windowing expressions

For example in q12 has the expression: sum(sum(ws_ext_sales_price)) over (partition by i_class)

Queries in tpcds having this issue: [12, 20, 47, 51, 53, 57, 63, 89, 98]

How to reproduce:
Add a test to TpcdsQueryNoValidation like this:

@ParameterizedTest
  @ValueSource(ints = {12})
  public void tpcdsQ12(int query) throws Exception {
    testQuery(query);
  }

On fixing, please move the fixed queries from the tpcdsFailure list to the tpcdsSuccess list

[ISTHMUS] [TPCDS] Unsupported subquery nested in Project relational operator

See tpcds q9

How to reproduce:
Add a test to TpcdsQueryNoValidation like this:

@ParameterizedTest
  @ValueSource(ints = {9})
  public void tpcdsQ9(int query) throws Exception {
    testQuery(query);
  }

On fixing, please move the fixed queries from the tpcdsFailure list to the tpcdsSuccess list

[ISTHMUS] Null value support in filter

Hi, it seems we don't support null data in filter currently, e.g
table :
CREATE TABLE table_test(col_a BIGINT)
query :
SELECT col_a FROM table_test WHERE col_a IS NOT NULL or
SELECT col_a FROM table_test WHERE col_a != NULL

Any help will be appreciated!

[ISTHMUS] Add support bi-drectional calcite/substrait support for window functions

[ISTHMUS] Issues in string concatenation

Isthmus may run into problems when converting a string concatenation call to substrait plan.

1. It does not support concatenating two string literals

The input to Isthmus is

"SELECT 'foo' || 'bar' FROM test;" -c "CREATE TABLE test(col_1 INTEGER NOT NULL, col_2 VARCHAR(10));"

and it throws an error. I suspect that this is because substrait only have two signatures for concat: concat:vchar_vchar and concat:str_str. In this case, both inputs are fixed char fchar so the signature match fails. Perhaps adding a cast somewhere can fix this?

Full stack trace below:

java.lang.IllegalArgumentException: Unable to convert call ||(char<3>, char<3>).
        at io.substrait.isthmus.expression.RexExpressionConverter.visitCall(RexExpressionConverter.java:78)
        at io.substrait.isthmus.expression.RexExpressionConverter.visitCall(RexExpressionConverter.java:19)
        at org.apache.calcite.rex.RexCall.accept(RexCall.java:189)
        at io.substrait.isthmus.SubstraitRelVisitor.toExpression(SubstraitRelVisitor.java:99)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
        at java.util.Collections$2.tryAdvance(Collections.java:4853)
        at java.util.Collections$2.forEachRemaining(Collections.java:4861)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
        at io.substrait.isthmus.SubstraitRelVisitor.visit(SubstraitRelVisitor.java:156)
        at io.substrait.isthmus.SubstraitRelVisitor.visit(SubstraitRelVisitor.java:61)
        at io.substrait.isthmus.RelNodeVisitor.reverseAccept(RelNodeVisitor.java:95)
        at io.substrait.isthmus.SubstraitRelVisitor.apply(SubstraitRelVisitor.java:351)
        at io.substrait.isthmus.SubstraitRelVisitor.convert(SubstraitRelVisitor.java:374)
        at io.substrait.isthmus.SubstraitRelVisitor.convert(SubstraitRelVisitor.java:366)
        at io.substrait.isthmus.SqlToSubstrait.lambda$executeInner$0(SqlToSubstrait.java:80)
        at java.util.ArrayList.forEach(ArrayList.java:1511)
        at io.substrait.isthmus.SqlToSubstrait.executeInner(SqlToSubstrait.java:73)
        at io.substrait.isthmus.SqlToSubstrait.execute(SqlToSubstrait.java:41)
        at io.substrait.isthmus.PlanEntryPoint.call(PlanEntryPoint.java:59)
        at io.substrait.isthmus.PlanEntryPoint.call(PlanEntryPoint.java:16)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
        at picocli.CommandLine.access$1300(CommandLine.java:145)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
        at picocli.CommandLine.execute(CommandLine.java:2078)
        at io.substrait.isthmus.PlanEntryPoint.main(PlanEntryPoint.java:51)

2. It only recognizes a || b as the concatenation operator, and does not recognize CONCAT(a, b)

The input query is

"SELECT CONCAT('foo', col_2) FROM test;"

This seems to be limited by Apache Calcite. "SELECT 'foo' || col_2 FROM test;" would work just fine.

Full stack trace below:

org.apache.calcite.runtime.CalciteContextException: From line 1, column 8 to line 1, column 27: No match found for function signature CONCAT(<CHARACTER>, <CHARACTER>)
        at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:480)
        at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:505)
        at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:932)
        at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:917)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:5266)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.handleUnresolvedFunction(SqlValidatorImpl.java:1948)
        at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:326)
        at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:231)
        at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:6277)
        at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:6264)
        at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:161)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1862)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1847)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:463)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4409)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3652)
        at org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)
        at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1100)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1071)
        at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:247)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1046)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:752)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:586)
        at io.substrait.isthmus.SqlToSubstrait.lambda$sqlToRelNode$1(SqlToSubstrait.java:110)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
        at java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:720)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
        at io.substrait.isthmus.SqlToSubstrait.sqlToRelNode(SqlToSubstrait.java:119)
        at io.substrait.isthmus.SqlToSubstrait.executeInner(SqlToSubstrait.java:72)
        at io.substrait.isthmus.SqlToSubstrait.execute(SqlToSubstrait.java:41)
        at io.substrait.isthmus.PlanEntryPoint.call(PlanEntryPoint.java:59)
        at io.substrait.isthmus.PlanEntryPoint.call(PlanEntryPoint.java:16)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
        at picocli.CommandLine.access$1300(CommandLine.java:145)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
        at picocli.CommandLine.execute(CommandLine.java:2078)
        at io.substrait.isthmus.PlanEntryPoint.main(PlanEntryPoint.java:51)
Caused by: org.apache.calcite.sql.validate.SqlValidatorException: No match found for function signature CONCAT(<CHARACTER>, <CHARACTER>)
        at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:480)
        at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:505)
        at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:599)
        ... 43 more

[ISTHMUS] [TPCDS] Missing functions: round, stddev_samp, upper, like

For example in q2 has the expression: round(sun_sales1/sun_sales2,2)

Queries in tpcds having this issue: [2, 17, 24, 39, 78, 91]

How to reproduce:
Add a test to TpcdsQueryNoValidation like this:

@ParameterizedTest
  @ValueSource(ints = {2})
  public void tpcdsQ2(int query) throws Exception {
    testQuery(query);
  }

On fixing, please move the fixed queries from the tpcdsFailure list to the tpcdsSuccess list

Add FuncArg type

• Update Scalar and Aggregate FunctionInvocation to be made of FuncArgs
• Update Proto, Calcite converters to build & consume FuncArgs

See Issue #37 for background on this issue.

[ISTHMUS] Function Signatures Include Outdated `opt`

Plans generated with Isthmus currently include opt_ as a prefix in function signatures.

For example

./isthmus -c "CREATE TABLE persons (zip INT)" "SELECT zip + 1 FROM persons"

creates the follow plan (elided for brevity)

{
  "extensionUris": [{
    "extensionUriAnchor": 1,
    "uri": "/functions_arithmetic.yaml"
  }],
  "extensions": [{
    "extensionFunction": {
      "extensionUriReference": 1,
      "functionAnchor": 0,
      "name": "add:opt_i32_i32"
    }
  }],
  ...
}

in which the + operators is named "add:opt_i32_i32"

The current spec for Function Signature Compound Names no longer includes opt.
This was removed as part of substrait-io/substrait#342.

Isthmus should be updated to omit the opt_ prefix to match the spec.

spotlessJava task failures during build

I'm seeing the following task failures when running ./gradlew build.

:isthmus:spotlessJava error

> Task :isthmus:spotlessJava FAILED
Step 'google-java-format' found problem in 'src/main/java/io/substrait/isthmus/TypeConverter.java':
null
java.lang.reflect.InvocationTargetException
	at jdk.internal.reflect.GeneratedMethodAccessor415.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.diffplug.spotless.java.GoogleJavaFormatStep$State.lambda$createFormat$1(GoogleJavaFormatStep.java:177)
	at com.diffplug.spotless.FormatterFunc.apply(FormatterFunc.java:32)
	at com.diffplug.spotless.FormatterStepImpl$Standard.format(FormatterStepImpl.java:82)
	at com.diffplug.spotless.FormatterStep$Strict.format(FormatterStep.java:88)
	at com.diffplug.spotless.Formatter.compute(Formatter.java:230)
	at com.diffplug.spotless.PaddedCell.calculateDirtyState(PaddedCell.java:203)
	at com.diffplug.spotless.PaddedCell.calculateDirtyState(PaddedCell.java:190)
	at com.diffplug.gradle.spotless.SpotlessTaskImpl.processInputFile(SpotlessTaskImpl.java:102)
	at com.diffplug.gradle.spotless.SpotlessTaskImpl.performAction(SpotlessTaskImpl.java:88)
	at jdk.internal.reflect.GeneratedMethodAccessor458.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:104)
	at org.gradle.api.internal.project.taskfactory.IncrementalInputsTaskAction.doExecute(IncrementalInputsTaskAction.java:32)
	at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:51)
	at org.gradle.api.internal.project.taskfactory.AbstractIncrementalTaskAction.execute(AbstractIncrementalTaskAction.java:25)
	at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:29)
	at org.gradle.api.internal.tasks.execution.TaskExecution$3.run(TaskExecution.java:242)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.run(DefaultBuildOperationRunner.java:47)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:68)
	at org.gradle.api.internal.tasks.execution.TaskExecution.executeAction(TaskExecution.java:227)
	at org.gradle.api.internal.tasks.execution.TaskExecution.executeActions(TaskExecution.java:210)
	at org.gradle.api.internal.tasks.execution.TaskExecution.executeWithPreviousOutputFiles(TaskExecution.java:193)
	at org.gradle.api.internal.tasks.execution.TaskExecution.execute(TaskExecution.java:171)
	at org.gradle.internal.execution.steps.ExecuteStep.executeInternal(ExecuteStep.java:89)
	at org.gradle.internal.execution.steps.ExecuteStep.access$000(ExecuteStep.java:40)
	at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:53)
	at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:50)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73)
	at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:50)
	at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:40)
	at org.gradle.internal.execution.steps.RemovePreviousOutputsStep.execute(RemovePreviousOutputsStep.java:68)
	at org.gradle.internal.execution.steps.RemovePreviousOutputsStep.execute(RemovePreviousOutputsStep.java:38)
	at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:48)
	at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:36)
	at org.gradle.internal.execution.steps.CancelExecutionStep.execute(CancelExecutionStep.java:41)
	at org.gradle.internal.execution.steps.TimeoutStep.executeWithoutTimeout(TimeoutStep.java:74)
	at org.gradle.internal.execution.steps.TimeoutStep.execute(TimeoutStep.java:55)
	at org.gradle.internal.execution.steps.CreateOutputsStep.execute(CreateOutputsStep.java:51)
	at org.gradle.internal.execution.steps.CreateOutputsStep.execute(CreateOutputsStep.java:29)
	at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.execute(CaptureStateAfterExecutionStep.java:61)
	at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.execute(CaptureStateAfterExecutionStep.java:42)
	at org.gradle.internal.execution.steps.BroadcastChangingOutputsStep.execute(BroadcastChangingOutputsStep.java:60)
	at org.gradle.internal.execution.steps.BroadcastChangingOutputsStep.execute(BroadcastChangingOutputsStep.java:27)
	at org.gradle.internal.execution.steps.BuildCacheStep.executeWithoutCache(BuildCacheStep.java:180)
	at org.gradle.internal.execution.steps.BuildCacheStep.executeAndStoreInCache(BuildCacheStep.java:155)
	at org.gradle.internal.execution.steps.BuildCacheStep.lambda$executeWithCache$4(BuildCacheStep.java:125)
	at java.base/java.util.Optional.orElseGet(Optional.java:369)
	at org.gradle.internal.execution.steps.BuildCacheStep.lambda$executeWithCache$5(BuildCacheStep.java:125)
	at org.gradle.internal.Try$Success.map(Try.java:164)
	at org.gradle.internal.execution.steps.BuildCacheStep.executeWithCache(BuildCacheStep.java:85)
	at org.gradle.internal.execution.steps.BuildCacheStep.lambda$execute$0(BuildCacheStep.java:74)
	at org.gradle.internal.Either$Left.fold(Either.java:115)
	at org.gradle.internal.execution.caching.CachingState.fold(CachingState.java:59)
	at org.gradle.internal.execution.steps.BuildCacheStep.execute(BuildCacheStep.java:73)
	at org.gradle.internal.execution.steps.BuildCacheStep.execute(BuildCacheStep.java:48)
	at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:36)
	at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:25)
	at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:36)
	at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:22)
	at org.gradle.internal.execution.steps.SkipUpToDateStep.executeBecause(SkipUpToDateStep.java:110)
	at org.gradle.internal.execution.steps.SkipUpToDateStep.lambda$execute$2(SkipUpToDateStep.java:56)
	at java.base/java.util.Optional.orElseGet(Optional.java:369)
	at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:56)
	at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:38)
	at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:73)
	at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:44)
	at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:37)
	at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:27)
	at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:89)
	at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:50)
	at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:114)
	at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:57)
	at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:76)
	at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:50)
	at org.gradle.internal.execution.steps.SkipEmptyWorkStep.executeWithNoEmptySources(SkipEmptyWorkStep.java:249)
	at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:86)
	at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:54)
	at org.gradle.internal.execution.steps.RemoveUntrackedExecutionStateStep.execute(RemoveUntrackedExecutionStateStep.java:32)
	at org.gradle.internal.execution.steps.RemoveUntrackedExecutionStateStep.execute(RemoveUntrackedExecutionStateStep.java:21)
	at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsStartedStep.execute(MarkSnapshottingInputsStartedStep.java:38)
	at org.gradle.internal.execution.steps.LoadPreviousExecutionStateStep.execute(LoadPreviousExecutionStateStep.java:43)
	at org.gradle.internal.execution.steps.LoadPreviousExecutionStateStep.execute(LoadPreviousExecutionStateStep.java:31)
	at org.gradle.internal.execution.steps.AssignWorkspaceStep.lambda$execute$0(AssignWorkspaceStep.java:40)
	at org.gradle.api.internal.tasks.execution.TaskExecution$4.withWorkspace(TaskExecution.java:287)
	at org.gradle.internal.execution.steps.AssignWorkspaceStep.execute(AssignWorkspaceStep.java:40)
	at org.gradle.internal.execution.steps.AssignWorkspaceStep.execute(AssignWorkspaceStep.java:30)
	at org.gradle.internal.execution.steps.IdentityCacheStep.execute(IdentityCacheStep.java:37)
	at org.gradle.internal.execution.steps.IdentityCacheStep.execute(IdentityCacheStep.java:27)
	at org.gradle.internal.execution.steps.IdentifyStep.execute(IdentifyStep.java:44)
	at org.gradle.internal.execution.steps.IdentifyStep.execute(IdentifyStep.java:33)
	at org.gradle.internal.execution.impl.DefaultExecutionEngine$1.execute(DefaultExecutionEngine.java:76)
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:144)
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:133)
	at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:77)
	at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
	at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:51)
	at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
	at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
	at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73)
	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
	at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:74)
	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:333)
	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:320)
	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:313)
	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:299)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.lambda$run$0(DefaultPlanExecutor.java:143)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:227)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:218)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:140)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: com.google.googlejavaformat.java.FormatterException: 43:13: error: illegal start of expression
	at com.google.googlejavaformat.java.FormatterException.fromJavacDiagnostics(FormatterException.java:51)
	at com.google.googlejavaformat.java.Formatter.format(Formatter.java:149)
	at com.google.googlejavaformat.java.Formatter.getFormatReplacements(Formatter.java:271)
	at com.google.googlejavaformat.java.Formatter.formatSource(Formatter.java:247)
	at com.google.googlejavaformat.java.Formatter.formatSource(Formatter.java:213)
	... 142 more

:core:spotlessJava error

> Task :core:spotlessJava FAILED
Step 'google-java-format' found problem in 'src/main/java/io/substrait/relation/files/FileOrFiles.java':
null
java.lang.reflect.InvocationTargetException
	at jdk.internal.reflect.GeneratedMethodAccessor415.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.diffplug.spotless.java.GoogleJavaFormatStep$State.lambda$createFormat$1(GoogleJavaFormatStep.java:177)
	at com.diffplug.spotless.FormatterFunc.apply(FormatterFunc.java:32)
	at com.diffplug.spotless.FormatterStepImpl$Standard.format(FormatterStepImpl.java:82)
	at com.diffplug.spotless.FormatterStep$Strict.format(FormatterStep.java:88)
	at com.diffplug.spotless.Formatter.compute(Formatter.java:230)
	at com.diffplug.spotless.PaddedCell.calculateDirtyState(PaddedCell.java:203)
	at com.diffplug.spotless.PaddedCell.calculateDirtyState(PaddedCell.java:190)
	at com.diffplug.gradle.spotless.SpotlessTaskImpl.processInputFile(SpotlessTaskImpl.java:102)
	at com.diffplug.gradle.spotless.SpotlessTaskImpl.performAction(SpotlessTaskImpl.java:88)
	at jdk.internal.reflect.GeneratedMethodAccessor458.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:104)
	at org.gradle.api.internal.project.taskfactory.IncrementalInputsTaskAction.doExecute(IncrementalInputsTaskAction.java:32)
	at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:51)
	at org.gradle.api.internal.project.taskfactory.AbstractIncrementalTaskAction.execute(AbstractIncrementalTaskAction.java:25)
	at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:29)
	at org.gradle.api.internal.tasks.execution.TaskExecution$3.run(TaskExecution.java:242)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.run(DefaultBuildOperationRunner.java:47)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:68)
	at org.gradle.api.internal.tasks.execution.TaskExecution.executeAction(TaskExecution.java:227)
	at org.gradle.api.internal.tasks.execution.TaskExecution.executeActions(TaskExecution.java:210)
	at org.gradle.api.internal.tasks.execution.TaskExecution.executeWithPreviousOutputFiles(TaskExecution.java:193)
	at org.gradle.api.internal.tasks.execution.TaskExecution.execute(TaskExecution.java:171)
	at org.gradle.internal.execution.steps.ExecuteStep.executeInternal(ExecuteStep.java:89)
	at org.gradle.internal.execution.steps.ExecuteStep.access$000(ExecuteStep.java:40)
	at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:53)
	at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:50)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73)
	at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:50)
	at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:40)
	at org.gradle.internal.execution.steps.RemovePreviousOutputsStep.execute(RemovePreviousOutputsStep.java:68)
	at org.gradle.internal.execution.steps.RemovePreviousOutputsStep.execute(RemovePreviousOutputsStep.java:38)
	at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:48)
	at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:36)
	at org.gradle.internal.execution.steps.CancelExecutionStep.execute(CancelExecutionStep.java:41)
	at org.gradle.internal.execution.steps.TimeoutStep.executeWithoutTimeout(TimeoutStep.java:74)
	at org.gradle.internal.execution.steps.TimeoutStep.execute(TimeoutStep.java:55)
	at org.gradle.internal.execution.steps.CreateOutputsStep.execute(CreateOutputsStep.java:51)
	at org.gradle.internal.execution.steps.CreateOutputsStep.execute(CreateOutputsStep.java:29)
	at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.execute(CaptureStateAfterExecutionStep.java:61)
	at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.execute(CaptureStateAfterExecutionStep.java:42)
	at org.gradle.internal.execution.steps.BroadcastChangingOutputsStep.execute(BroadcastChangingOutputsStep.java:60)
	at org.gradle.internal.execution.steps.BroadcastChangingOutputsStep.execute(BroadcastChangingOutputsStep.java:27)
	at org.gradle.internal.execution.steps.BuildCacheStep.executeWithoutCache(BuildCacheStep.java:180)
	at org.gradle.internal.execution.steps.BuildCacheStep.executeAndStoreInCache(BuildCacheStep.java:155)
	at org.gradle.internal.execution.steps.BuildCacheStep.lambda$executeWithCache$4(BuildCacheStep.java:125)
	at java.base/java.util.Optional.orElseGet(Optional.java:369)
	at org.gradle.internal.execution.steps.BuildCacheStep.lambda$executeWithCache$5(BuildCacheStep.java:125)
	at org.gradle.internal.Try$Success.map(Try.java:164)
	at org.gradle.internal.execution.steps.BuildCacheStep.executeWithCache(BuildCacheStep.java:85)
	at org.gradle.internal.execution.steps.BuildCacheStep.lambda$execute$0(BuildCacheStep.java:74)
	at org.gradle.internal.Either$Left.fold(Either.java:115)
	at org.gradle.internal.execution.caching.CachingState.fold(CachingState.java:59)
	at org.gradle.internal.execution.steps.BuildCacheStep.execute(BuildCacheStep.java:73)
	at org.gradle.internal.execution.steps.BuildCacheStep.execute(BuildCacheStep.java:48)
	at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:36)
	at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:25)
	at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:36)
	at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:22)
	at org.gradle.internal.execution.steps.SkipUpToDateStep.executeBecause(SkipUpToDateStep.java:110)
	at org.gradle.internal.execution.steps.SkipUpToDateStep.lambda$execute$2(SkipUpToDateStep.java:56)
	at java.base/java.util.Optional.orElseGet(Optional.java:369)
	at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:56)
	at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:38)
	at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:73)
	at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:44)
	at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:37)
	at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:27)
	at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:89)
	at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:50)
	at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:114)
	at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:57)
	at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:76)
	at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:50)
	at org.gradle.internal.execution.steps.SkipEmptyWorkStep.executeWithNoEmptySources(SkipEmptyWorkStep.java:249)
	at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:86)
	at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:54)
	at org.gradle.internal.execution.steps.RemoveUntrackedExecutionStateStep.execute(RemoveUntrackedExecutionStateStep.java:32)
	at org.gradle.internal.execution.steps.RemoveUntrackedExecutionStateStep.execute(RemoveUntrackedExecutionStateStep.java:21)
	at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsStartedStep.execute(MarkSnapshottingInputsStartedStep.java:38)
	at org.gradle.internal.execution.steps.LoadPreviousExecutionStateStep.execute(LoadPreviousExecutionStateStep.java:43)
	at org.gradle.internal.execution.steps.LoadPreviousExecutionStateStep.execute(LoadPreviousExecutionStateStep.java:31)
	at org.gradle.internal.execution.steps.AssignWorkspaceStep.lambda$execute$0(AssignWorkspaceStep.java:40)
	at org.gradle.api.internal.tasks.execution.TaskExecution$4.withWorkspace(TaskExecution.java:287)
	at org.gradle.internal.execution.steps.AssignWorkspaceStep.execute(AssignWorkspaceStep.java:40)
	at org.gradle.internal.execution.steps.AssignWorkspaceStep.execute(AssignWorkspaceStep.java:30)
	at org.gradle.internal.execution.steps.IdentityCacheStep.execute(IdentityCacheStep.java:37)
	at org.gradle.internal.execution.steps.IdentityCacheStep.execute(IdentityCacheStep.java:27)
	at org.gradle.internal.execution.steps.IdentifyStep.execute(IdentifyStep.java:44)
	at org.gradle.internal.execution.steps.IdentifyStep.execute(IdentifyStep.java:33)
	at org.gradle.internal.execution.impl.DefaultExecutionEngine$1.execute(DefaultExecutionEngine.java:76)
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:144)
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:133)
	at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:77)
	at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
	at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:51)
	at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
	at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
	at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73)
	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
	at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:74)
	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:333)
	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:320)
	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:313)
	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:299)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.lambda$run$0(DefaultPlanExecutor.java:143)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:227)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:218)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:140)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: com.google.googlejavaformat.java.FormatterException: 35:71: error: ')' expected
	at com.google.googlejavaformat.java.FormatterException.fromJavacDiagnostics(FormatterException.java:51)
	at com.google.googlejavaformat.java.Formatter.format(Formatter.java:149)
	at com.google.googlejavaformat.java.Formatter.getFormatReplacements(Formatter.java:271)
	at com.google.googlejavaformat.java.Formatter.formatSource(Formatter.java:247)
	at com.google.googlejavaformat.java.Formatter.formatSource(Formatter.java:213)
	... 142 more

[CORE] Push core java release artifacts to mvn central

[Isthmus] Convert SQL expression to Substrait `ExtendedExpression`

substrait-io/substrait#405 added an ExtendedExpression message to represent standalone expressions (expressions that are not part of plans). #127 would add support for this in the Substrait Java library. It would also be nice if Isthmus could convert SQL expressions (not whole queries) to Substrait ExtendedExpression messages. This would help to open up more applications for Substrait as an interoperable representation for expressions.

[ISTHMUS] Add friendly tips for the CLI about using -c option to pass DDL SQL when parsing DML SQL

For beginners, if a user only parses a DML SQL via isthmus will meet the calcite exception, like this:

org.apache.calcite.runtime.CalciteContextException: From line 1, column 17 to line 1, column 31: Object 'XXX' not found
        at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:480)
        at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:505)
        at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:932)
        at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:917)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:5266)
        at org.apache.calcite.sql.validate.IdentifierNamespace.resolveImpl(IdentifierNamespace.java:183)
        at org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl(IdentifierNamespace.java:188)
        at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1100)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1071)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:3375)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3639)
        at org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)
        at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1100)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1071)
        at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:247)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1046)
        at org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:752)
        at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:586)
        at io.substrait.isthmus.SqlToSubstrait.getBestExpRelRoot(SqlToSubstrait.java:124)
        at io.substrait.isthmus.SqlToSubstrait.lambda$sqlToRelNode$1(SqlToSubstrait.java:103)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
        at java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:720)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
        at io.substrait.isthmus.SqlToSubstrait.sqlToRelNode(SqlToSubstrait.java:104)
        at io.substrait.isthmus.SqlToSubstrait.executeInner(SqlToSubstrait.java:74)
        at io.substrait.isthmus.SqlToSubstrait.execute(SqlToSubstrait.java:43)
        at io.substrait.isthmus.PlanEntryPoint.call(PlanEntryPoint.java:59)
        at io.substrait.isthmus.PlanEntryPoint.call(PlanEntryPoint.java:16)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
        at picocli.CommandLine.access$1300(CommandLine.java:145)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
        at picocli.CommandLine.execute(CommandLine.java:2078)
        at io.substrait.isthmus.PlanEntryPoint.main(PlanEntryPoint.java:51)
Caused by: org.apache.calcite.sql.validate.SqlValidatorException: Object 'XXX' not found
        at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:480)
        at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:505)
        at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:599)
        ... 40 more

So, IMO, it would be better to give a friendly tip when parsing DML SQL to let the users know it could give the definition via the -c option.

Add python bindings for Isthmus

Adding python bindings for Isthmus would allow us to easily generate Isthmus substrait plans on the fly in the consumer-testing, instead of manually generating the plans and merging them into the repo to be tested against.

[ISTHMUS] Create a README for Isthmus

[Isthmus] What's the difference between tpch_smoke.sh and TpchQueryNoValidation

I noticed that both of them share almost same code path. tpch_smoke.sh can not pass some queries(q7,q8...) while TpchQueryNoValidation can pass all. And tpch_smoke.sh is in CI workflow, the other one is not.

SqlToSubstrait conversion fails on encountering key constraints

SQL create-table statements often contain key constraints, for e.g. TPC-DS table primary key declarations like below:

create table store_sales (
    ss_item_sk                integer               not null,
    ...
    ss_net_profit             decimal(7,2)                  ,
    primary key (ss_item_sk));

These constraint declaration are valid per SQL standard. Currently the SQL to Substrait fails when it encounters such constraints with the following error:

java.lang.ClassCastException: class org.apache.calcite.sql.ddl.SqlDdlNodes$1 cannot be cast to class org.apache.calcite.sql.ddl.SqlColumnDeclaration (org.apache.calcite.sql.ddl.SqlDdlNodes$1 and org.apache.calcite.sql.ddl.SqlColumnDeclaration are in unnamed module of loader 'app')
	at io.substrait.isthmus.SqlConverterBase.parseCreateTable(SqlConverterBase.java:123)
	at io.substrait.isthmus.SqlConverterBase.registerCreateTables(SqlConverterBase.java:86)
	at io.substrait.isthmus.SqlToSubstrait.execute(SqlToSubstrait.java:81)

Instead of expecting users to omit constraints from input table definitions, key constraints should be ignored so that the plan generation does not fail.

[ISTHMUS] Fix handling of the year/month/day Enum of the extract function

In sql -> substrait plan conversion, tpc-q q7, q8, q9 would fail with error: Unable to convert call EXTRACT(string, date?).

Although substrait spec's functions_datetime.yaml has declarations for EXTRACT function, substrait-java's ScalarFunctionConverter would not resolve EXTRACT function call properly.

Part of reason it does not work properly is the handling of EnumArgument. Somehow the code handles EnumArgument as a string argument, and leads to the failure of function resolution.

Open this ticket to track this issue.

[Isthmus] failed queries(7 8 9 12) in tpch_smoke.sh

I verified all tpch queries via tpch_smoke.sh which use graal compiled native code, found that these queries(7 8 9 12) will fail.
BTW, q15 will fail due to comments can not be parsed, remove that will work.
error message:

        .......
        at com.google.protobuf.util.JsonFormat$Printer.print(JsonFormat.java:381)
        at io.substrait.isthmus.PlanEntryPoint.call(PlanEntryPoint.java:42)
        at io.substrait.isthmus.PlanEntryPoint.call(PlanEntryPoint.java:15)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
        at picocli.CommandLine.access$1300(CommandLine.java:145)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
        at picocli.CommandLine.execute(CommandLine.java:2078)
        at io.substrait.isthmus.PlanEntryPoint.main(PlanEntryPoint.java:49)
Caused by: java.lang.NoSuchMethodException: com.google.protobuf.Empty.newBuilder()

Can anyone help verify and fix it? Thanks in advance!

[ISTHMUS] Add TPC-DS smoke tests

Repository Setup & Automation

👋 hi all - having tried out Substrait and Substrait Java I'm really excited about the future of the projects! I'd love to help out if I can.

I'm currently experimenting with converting ZetaSQL parse trees into Substrait plans, but have found that it would be great to get the latest version of the Substrait protos into substrait-java.

I have a bit of experience with Gradle Java projects and saw there were a few open issues around repository automation such as auto-upgrade of the substrait specs and getting releases into Maven Central. Would it be helpful if I put up a PR to set this up? I've had good experiences with Renovate Bot in the past - you can see some of the upgrade PRs it has made to my fork of substrait-java

Semantic release workflow error (node version error)

Error message:

[semantic-release]: node version >=18 is required. Found v16.19.0.

See https://github.com/semantic-release/semantic-release/blob/master/docs/support/node-version.md for more details and solutions.

[Isthmus] Specify failure behavior on cast

The field Cast::failure_behavior has no listed default and yet Isthmus does not specify a value which forces consumers to guess.

Running the repo to get the Substrait plans for all supported TPCH queries

Hi, I've been trying to run the repo to get the substrait plans for the supported TPCH queries. I'm using Ubuntu for Windows.

Initially I tried the following commands:

git clone https://github.com/substrait-io/substrait-java
cd substrait-java/
git submodule update --init --recursive
./gradlew build
./gradlew nativeImage

And got a failed build error with the ./gradlew nativeImage command (error statement attached).

As an alternative, I tried to download the isthmus-ubuntu-latest pre-compiled build artifact and ran

./isthmus-ubuntu-latest --create "cat isthmus/src/test/resources/tpch/schema.sql" "cat isthmus/src/test/resources/tpch/queries/06.sql"

but got a Calcite undefined method issue (error statement attached).

Error statements:
isthmus.txt

Please let me know if I'm missing anything that I need to do to resolve this issue. Alternatively, if anyone already has the substrait plans for all the supported TPCH queries and could directly share them with me, that'll be incredibly helpful.

Looking forward to your response, thank you very much for your support!

[BUILD] Create an automated bot that upgrades core submodule automatically and merges if tests pass

[ISTHMUS] Out-of-spec plans returned by Isthmus

While testing the validator against Isthmus output for TPC-H, I found a number of issues where Isthmus disagrees with the spec as written on the website. The ones I found are:

• The grouping set index column appended by the aggregate relation isn't removed from the output, resulting in excess columns both functionally and in the RelRoot column name list. Failing example: select l_returnflag, count(*) from lineitem group by l_returnflag. The easy solution is just to generate an emit clause for all aggregate relations.
• Decimal literals are generated with 38 rather than 16 bytes. Failing example: select 0.1
• Negative decimal literals are not sign extended all the way to the MSB. Failing example: select -0.1, which yields out-of-bounds value 25.5 for P=2 S=1.
• For whatever reason, some decimal literals are just emitted as zero. Failing example: select 000000.000001.
• The emitted function signatures don't follow the table here for the abbreviated names. For example, decimal is used instead of dec, and any1 is used in place of just any. Failing example: literally anything with a function, such as select 0.1 + 1.0, which generates add:opt_decimal_decimal instead of add:opt_dec_dec.
• Virtual tables receive a schema representing a single column of structs containing the fields. Failing example: select 1 is given the schema STRUCT<STRUCT<i32>>. The type itself is also invalid because of this, because only one field name is attached when two would be needed for that type. It should be NSTRUCT<EXPR$0: i32> here.
• Arguably this is not a spec violation (the spec never mentions how this should be done), but using anchor value 0 is IMO a really bad idea, because on the reference side it's indistinguishable from not referencing anything. For type variations it would actually be an issue, because "no variation" would be indistinguishable from the type variation with anchor 0, and if any optional references are ever added for the other anchor types it would become a problem for them too. It's also confusing when reading a plan where the reference is "optimized out" due to having the default value, and it's easy to forget to add a reference because nothing can check for this error. Simply starting the counter from 1 instead of 0 prevents all this and barely costs anything (specifically, it costs ~0.00000003% of the anchor code space, and a couple bytes in the binary serialization).

I haven't touched anything java for about a decade, so I'm probably not the right person to propose fixes for these things.

Here's a script to easily run Isthmus and generate a validation report for the result. Just run pip install . at https://github.com/jvanstraten/substrait-validator/tree/initial/py to install the validator, and chuck the script in the root of substrait-java. Call the script without arguments to generate reports for all TPC-H queries (it will try them all, but will only generate reports for successful runs), or with a SQL query as cmdline argument for a custom query based on the TPC-H database schema. The reports are written to the tpch-output directory.

tpch.py

#!/usr/bin/env python3
# SPDX-License-Identifier: Apache-2.0

import os
import sys
import shutil
import subprocess
import json
import substrait_validator

def format_html_code_block(text, lang):
    text = text.replace('&', '&amp')
    text = text.replace('<', '&lt;')
    text = text.replace('>', '&gt;')
    text = text.replace('"', '&quot;')
    text = text.replace("'", '&apos;')
    text = text.replace(" ", '&nbsp;')
    text = text.replace("\t", '&nbsp;&nbsp;&nbsp;&nbsp;')
    text = text.replace("\n", '<br/>\n')
    return f'<pre><code class="language-{lang}">{text}</code></pre>'

_EXTRA_HEAD = """
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.5.0/styles/default.min.css">
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.5.0/highlight.min.js"></script>
"""

if __name__ == '__main__':

    # Figure out environment.
    script_path = os.path.dirname(os.path.realpath(__file__))
    output_path = os.path.join(script_path, 'tpch-output')
    repo_path = script_path
    java_path = repo_path
    tpch_path = os.path.join(java_path, 'isthmus', 'src', 'test', 'resources', 'tpch')
    schema_path = os.path.join(tpch_path, 'schema.sql')
    query_path = os.path.join(tpch_path, 'queries')
    isthmus_path = os.path.join(java_path, 'isthmus', 'build', 'graal', 'isthmus')

    # Create output directory.
    if not os.path.isdir(output_path):
        os.makedirs(output_path)

    # Build isthmus if it hasn't already been built.
    if not os.path.isfile(isthmus_path):
        cur = os.curdir
        os.chdir(java_path)
        try:
            subprocess.run(['./gradlew', 'nativeImage'], check=True)
        finally:
            os.chdir(cur)

    # Load TPC-H schema files.
    isthmus_args = [isthmus_path]
    with open(schema_path, 'r') as f:
        for query in filter(bool, map(str.strip, f.read().split(';'))):
            isthmus_args.append('-c')
            isthmus_args.append(query)

    # Function to run Isthmus and the validator on some named query.
    def run_isthmus(name, query):
        try:

            # Convert to Substrait plan with Isthmus.
            result = subprocess.run(isthmus_args + [query], capture_output=True)
            if result.returncode != 0:
                raise Exception(result.stderr.decode('utf-8'))
            plan = result.stdout.decode('utf-8')

            # Convert to HTML validation result with validator.
            html = substrait_validator.plan_to_html(plan)

            # Unpack the HTML a bit so we can add stuff to it.
            html_gen_a, remain = html.split('</head>', maxsplit=1)
            html_gen_b, remain = remain.split('<body>', maxsplit=1)
            html_gen_b = f'</head>{html_gen_b}<body>'
            html_gen_c, html_gen_d = remain.split('</body>', maxsplit=1)
            html_gen_d = f'</body>{html_gen_d}'

            # Add our stuff to it.
            html = []
            html.append(html_gen_a)
            html.append(f'<title>{name}</title>')
            html.append(_EXTRA_HEAD)
            html.append(html_gen_b)
            html.append(f'<h1>{name} with Isthmus</h1>')
            html.append(f'<h2>SQL query</h2>')
            html.append(format_html_code_block(query, 'sql'))
            html.append(f'<h2>Plan JSON</h2>')
            html.append(format_html_code_block(plan, 'json'))
            html.append(f'<h2>Validation result</h2>')
            html.append(html_gen_c)
            html.append('<script>hljs.highlightAll();</script>')
            html.append(html_gen_d)
            html = '\n'.join(html)

            # Write file.
            html_fname = os.path.join(output_path, f'{name}.html')
            with open(html_fname, 'w') as f:
                f.write(html)

        except Exception as e:
            print(f'{type(e).__name__} for {name}: {e}')

    if len(sys.argv) <= 1:

        # Run isthmus and the validator for all queries.
        for query_fname in sorted(os.listdir(query_path)):
            name = query_fname.split('.')[0]
            with open(os.path.join(query_path, query_fname), 'r') as f:
                query = f.read()
            run_isthmus(name, query)

    else:

        # Interpret command line as a query.
        run_isthmus('cmdline', ' '.join(sys.argv[1:]))