Thread 'main' panicked at 'assertion failed: rec_number > 0'

[pacman82]

Thread 'main' panicked at 'assertion failed: rec_number > 0'

[pacman82]

Hello @Htorne , thanks for filing this Bug report.

I do not know about the runneradmin path, but you can look at the referenced file here: https://github.com/pacman82/odbc-api/blob/d6d83449d2164d7e8a47dcb437071584782da663/odbc-api/src/handles/diagnostics.rs#L55

This Bug must be fixed upstream in odbc-api, but it is also a crate I do maintain.

I have a hypothesis on what is happening:

The rec_number variable is an i16 used to index a diagnostic record. My guess is it overflows, because an API call causes more than 32767 warnings. I further assume what that many warnings are likely generated by something which causes one warning by row. Which also fits neatly with you describing the tool working fine with the smaller tenant.

The fix I have in mind is a one liner, but I must think about how I do best reproduce it.

In the meanwhile you could help me, by verifying my assumptions. Please let the command run with the -vvvv flag to get more verbose output. Do you see a lot of messages?

Also you likely have a job at hand, which you want to get done: Reducing the --batch-size to something smaller than 32767 might solve your problem, provided only one warning per row is generated. Don't worry, it will still process the entire 60GB+ data set, just in smaller junks. If this works it would further support the diagnosis. Please let me know.

[pacman82]

PS. if it turns out to be an issue with too much warnings, I would also like to know what the warning is about. Maybe there is another issue hiding underneath.

Thanks!

[pacman82]

Released a version 0.4.2 which includes the upstream fix. Please tell me if this solves your issue.

[Htorne]

Thanks pacman82 it worked!

[pacman82]

Mind telling me about the warnings you got?

[Htorne]

2020-12-03T13:05:09+00:00 - WARN - State: 01000, Native error: 5701, Message: [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Changed database context to 'master'.
2020-12-03T13:05:09+00:00 - WARN - State: 01000, Native error: 5703, Message: [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Changed language setting to us_english.
2020-12-03T13:05:09+00:00 - INFO - Batch size set to 100000
2020-12-03T13:05:09+00:00 - DEBUG - ODBC column description for column 1: ColumnDescription { name: [73, 100], data_type: Numeric { precision: 15, scale: 1 }, nullable: Nullable }
2020-12-03T13:05:09+00:00 - DEBUG - ODBC buffer description for column 1: BufferDescription { nullable: true, kind: Text { max_str_len: 17 } }
2020-12-03T13:05:09+00:00 - DEBUG - ODBC column description for column 2: ColumnDescription { name: [70, 105, 114, 115, 116, 78, 97, 109, 101], data_type: Other { data_type: SqlDataType(-9), column_size: 40, decimal_digits: 0 }, 
nullable: NoNulls }
2020-12-03T13:05:09+00:00 - DEBUG - ODBC buffer description for column 2: BufferDescription { nullable: true, kind: Text { max_str_len: 40 } }
2020-12-03T13:05:09+00:00 - DEBUG - ODBC column description for column 3: ColumnDescription { name: [76, 97, 115, 116, 78, 97, 109, 101], data_type: Other { data_type: SqlDataType(-9), column_size: 40, decimal_digits: 0 }, 
nullable: NoNulls }
2020-12-03T13:05:09+00:00 - DEBUG - ODBC buffer description for column 3: BufferDescription { nullable: true, kind: Text { max_str_len: 40 } }
2020-12-03T13:05:09+00:00 - DEBUG - ODBC column description for column 4: ColumnDescription { name: [67, 105, 116, 121], data_type: Other { data_type: SqlDataType(-9), column_size: 40, decimal_digits: 0 }, nullable: Nullable }
2020-12-03T13:05:09+00:00 - DEBUG - ODBC buffer description for column 4: BufferDescription { nullable: true, kind: Text { max_str_len: 40 } }
2020-12-03T13:05:09+00:00 - DEBUG - ODBC column description for column 5: ColumnDescription { name: [67, 111, 117, 110, 116, 114, 121], data_type: Other { data_type: SqlDataType(-9), column_size: 40, decimal_digits: 0 }, 
nullable: Nullable }
2020-12-03T13:05:09+00:00 - DEBUG - ODBC buffer description for column 5: BufferDescription { nullable: true, kind: Text { max_str_len: 40 } }
2020-12-03T13:05:09+00:00 - DEBUG - ODBC column description for column 6: ColumnDescription { name: [80, 104, 111, 110, 101], data_type: Other { data_type: SqlDataType(-9), column_size: 20, decimal_digits: 0 }, nullable: 
Nullable }
2020-12-03T13:05:09+00:00 - DEBUG - ODBC buffer description for column 6: BufferDescription { nullable: true, kind: Text { max_str_len: 20 } }
2020-12-03T13:05:09+00:00 - DEBUG - ODBC column description for column 7: ColumnDescription { name: [71, 101, 111, 109, 101, 116, 114, 121], data_type: Other { data_type: SqlDataType(-151), column_size: 0, decimal_digits: 0 }, 
nullable: Nullable }
2020-12-03T13:05:09+00:00 - DEBUG - ODBC buffer description for column 7: BufferDescription { nullable: true, kind: Text { max_str_len: 0 } }
2020-12-03T13:05:45+00:00 - WARN - State: 01004, Native error: 0, Message: [Microsoft][ODBC Driver 17 for SQL Server]String data, right truncation
2020-12-03T13:05:45+00:00 - WARN - State: 01004, Native error: 0, Message: [Microsoft][ODBC Driver 17 for SQL Server]String data, right truncation

Then I get around 300.000 errors like this

2020-12-03T13:05:45+00:00 - WARN - State: 01004, Native error: 0, Message: [Microsoft][ODBC Driver 17 for SQL Server]String data, right truncation
2020-12-03T13:05:45+00:00 - WARN - State: 01004, Native error: 0, Message: [Microsoft][ODBC Driver 17 for SQL Server]String data, right truncation
2020-12-03T13:05:45+00:00 - WARN - State: 01004, Native error: 0, Message: [Microsoft][ODBC Driver 17 for SQL Server]String data, right truncation
2020-12-03T13:05:45+00:00 - WARN - State: 01004, Native error: 0, Message: [Microsoft][ODBC Driver 17 for SQL Server]String data, right truncation
2020-12-03T13:05:45+00:00 - WARN - State: 01004, Native error: 0, Message: [Microsoft][ODBC Driver 17 for SQL Server]String data, right truncation
2020-12-03T13:05:45+00:00 - WARN - State: 01004, Native error: 0, Message: [Microsoft][ODBC Driver 17 for SQL Server]String data, right truncation
2020-12-03T13:05:45+00:00 - WARN - State: 01004, Native error: 0, Message: [Microsoft][ODBC Driver 17 for SQL Server]String data, right truncation
2020-12-03T13:05:45+00:00 - WARN - State: 01004, Native error: 0, Message: [Microsoft][ODBC Driver 17 for SQL Server]String data, right truncation
2020-12-03T13:05:45+00:00 - WARN - State: 01004, Native error: 0, Message: [Microsoft][ODBC Driver 17 for SQL Server]String data, right truncation

My guess is that it has to do with conversion of the MSSQL data type Geography to Utf8 Byte Array?