"herddb.utils.IllegalDataAccessException: table does not define column" on NOT IN statement

[dmercuriali]

"herddb.utils.IllegalDataAccessException: table does not define column" on NOT IN statement

[nicoloboschi]

I wrote a simple reproducer here master...nicoloboschi:fix-not-in-query

The issue is that JoinOp#execute creates a 'left data scanner' which contains a column name (DataScanner#getFieldNames) that is not a real table column but a temporary column used for projection computation

During the projection the records are added to a temporary disk map (if we are not in 'local' or 'diskless' mode) creating a MaterializedRecordSet (see

herddb/herddb-core/src/main/java/herddb/model/planner/JoinOp.java

Line 126 in e434d7a

MaterializedRecordSet recordSet = tableSpaceManager.getDbmanager().getRecordSetFactory()

)

In order to serialize the record, RecordSerializer#accessRawDataFromValue is called with the temporary field name as property and it throws error as the property is not a real table column
By-passing that check, my test reproducer pass and the scan result is correct

@eolivelli @diegosalvi AFAIK MaterializedRecordSet do not represent records set owned by a table, so I think that check is useless. Any thoughts?

[eolivelli]

DataAccessorForFullRecord is meant to access a "Record" without any memory copy.
If you see a DataAccessorForFullRecord for something that is not a record then we have a worse problem.

but here we have a RuntimeProjectedDataAccessor, that is a projection, that maps a Record to a different schema, it usually selects columns (by index) and changes the order.

it looks to me that we have a problem somewhere because the mapping should be perfect.

you can run your test and enable "INFO" level for the CalcitePlanner and see the actual access plan, you will see the Projection and what's going on