viant/bgc

Datastore Connectivity for BigQuery in go

bgc's Issues

Fields being inadvertently removed when inserting on Go 1.14

Hello!

We have an app using your library and we insert a bunch of data through PersistSingle and PersistAll.
However the same code that ran just fine on go 1.12 when I upgraded to go 1.14 started throwing errors on inserts related to missing fields.

The first one was a boolean flag which ran like this:

record := &migration{Version: 1, Dirty: false}
_, _, err := conn.PersistSingle(record, migrationsTable, nil)

Resulted in errors like these:

  "status": {
    "errorResult": {
      "message": "Error while reading data, error message: JSON table encountered too many errors, giving up. Rows: 1; errors: 1. Please look into the errors[] collection for more details.",
      "reason": "invalid"
    },
    "errors": [
      {
        "message": "Error while reading data, error message: JSON table encountered too many errors, giving up. Rows: 1; errors: 1. Please look into the errors[] collection for more details.",
        "reason": "invalid"
      },
      {
        "message": "Error while reading data, error message: JSON processing encountered too many errors, giving up. Rows: 1; errors: 1; max bad: 0; error percent: 0",
        "reason": "invalid"
      },
      {
        "message": "Error while reading data, error message: JSON parsing error in row starting at position 0: Missing required field: dirty.",
        "reason": "invalid"
      }
    ],
    "state": "DONE"
  },

I am guessing something changed on golang that made normalizeValue remove fields when it should not.

Versions:

bgc 0.7.0
dsc 0.10.2

I should note that I tried upgrading both dsc and bgc to the latest versions, and the errors persisted.