Creating an enum column with collation utf8_czech_ci causes panic

[mibk]

Creating an enum column with collation utf8_czech_ci causes panic

[timsehn]

It shouldn't panic but `utf8_czech_ci` is not supported and may not be a valid MySQL collation? https://docs.dolthub.com/sql-reference/sql-support/miscellaneous#collations-and-character-sets

…

On Mon, May 29, 2023 at 4:39 AM Michal Bohuslávek ***@***.***> wrote: I'm using ***@***.*** To reproduce, use the in-memory example server, and just: CREATE TABLE `foo` ( `bar` enum('Y','N') COLLATE utf8_czech_ci NOT NULL ) ENGINE=InnoDB; using the mysql client connected to the server. It appears getRuneWeight is *nil* in sql/collations.go:WriteWeightString: getRuneWeight := collationArray[c].Sorter i := 0 buf := *weightBuffers.Get().(*[]byte) if cap(buf) < len(str)*4 { buf = make([]byte, len(str)*4) } for len(str) > 0 { // All strings (should) have been decoded at this point, so we can rely on Go's internal string encoding runeFromString, strRead := utf8.DecodeRuneInString(str) if strRead == 0 || strRead == utf8.RuneError { return ErrCollationMalformedString.New("hashing") } runeWeight := getRuneWeight(runeFromString) buf[i*4] = byte(runeWeight) buf[i*4+1] = byte(runeWeight >> 8) buf[i*4+2] = byte(runeWeight >> 16) buf[i*4+3] = byte(runeWeight >> 24) _, err := hash.Write(buf[i*4 : i*4+4]) if err != nil { return err } str = str[strRead:] i++ } weightBuffers.Put(&buf) See the full stack trace here time="2023-05-29T13:31:57+02:00" level=error msg="mysql_server caught panic: runtime error: invalid memory address or nil pointer dereference /Users/u/.local/opt/go-v1.20.4/src/runtime/panic.go:260 (0x104acbc) \tpanicmem: panic(memoryError) /Users/u/.local/opt/go-v1.20.4/src/runtime/signal_unix.go:837 (0x104ac8c) \tsigpanic: panicmem() ***@***.***/sql/collations.go:884 (0x14a2605) \tcom/dolthub/go-mysql-server/sql.CollationID.WriteWeightString: runeWeight := getRuneWeight(runeFromString) ***@***.***/sql/collations.go:906 (0x14a28d8) \tcom/dolthub/go-mysql-server/sql.CollationID.HashToUint: err := c.WriteWeightString(hash, str) ***@***.***/sql/types/enum.go:79 (0x157a7f4) \tcom/dolthub/go-mysql-server/sql/types.CreateEnumType: hashedVal, err := collation.HashToUint(value) ***@***.***/sql/types/conversion.go:345 (0x1573936) \tcom/dolthub/go-mysql-server/sql/types.ColumnTypeToType: return CreateEnumType(ct.EnumValues, collation) ***@***.***/sql/parse/parse.go:2898 (0x1a6b053) \tcom/dolthub/go-mysql-server/sql/parse.columnDefinitionToColumn: internalTyp, err := types.ColumnTypeToType(&cd.Type) ***@***.***/sql/parse/parse.go:2881 (0x1a6acee) \tcom/dolthub/go-mysql-server/sql/parse.TableSpecToSchema: column, err := columnDefinitionToColumn(ctx, cd, tableSpec.Indexes) ***@***.***/sql/parse/parse.go:2447 (0x1a67153) \tcom/dolthub/go-mysql-server/sql/parse.convertCreateTable: schema, collation, err := TableSpecToSchema(ctx, c.TableSpec, false) ***@***.***/sql/parse/parse.go:1270 (0x1a5d128) \tcom/dolthub/go-mysql-server/sql/parse.convertDDL: return convertCreateTable(ctx, c) ***@***.***/sql/parse/parse.go:172 (0x1a52f16) \tcom/dolthub/go-mysql-server/sql/parse.convert: return convertDDL(ctx, query, n, false) ***@***.***/sql/parse/parse.go:118 (0x1a52b8e) \tcom/dolthub/go-mysql-server/sql/parse.parse: node, err := convert(ctx, stmt, s) ***@***.***/sql/parse/parse.go:77 (0x1be7004) \tcom/dolthub/go-mysql-server/sql/parse.ParseOne: return parse(ctx, query, true) ***@***.***/server/handler.go:310 (0x1be6fe9) \tcom/dolthub/go-mysql-server/server.(*Handler).doQuery: parsed, prequery, remainder, _ = parse.ParseOne(ctx, query) ***@***.***/server/handler.go:577 (0x1be98a7) \tcom/dolthub/go-mysql-server/server.(*Handler).errorWrappedDoQuery: remainder, err := h.doQuery(c, query, mode, bindings, callback) ***@***.***/server/handler.go:164 (0x1be55c8) \tcom/dolthub/go-mysql-server/server.(*Handler).ComMultiQuery: return h.errorWrappedDoQuery(c, query, MultiStmtModeOn, nil, callback) ***@***.***/go/mysql/conn.go:1447 (0x145fbf1) \tcom/dolthub/vitess/go/mysql.(*Conn).execQuery: remainder, err = handler.ComMultiQuery(c, query, resultsCB) ***@***.***/go/mysql/conn.go:972 (0x145b404) \tcom/dolthub/vitess/go/mysql.(*Conn).handleNextCommand: for query, err = c.execQuery(query, handler, multiStatements); err == nil && query != \"\"; { ***@***.***/go/mysql/server.go:484 (0x14764e7) \tcom/dolthub/vitess/go/mysql.(*Listener).handle: err := c.handleNextCommand(l.handler) /Users/u/.local/opt/go-v1.20.4/src/runtime/asm_amd64.s:1598 (0x1066b00) \tgoexit: BYTE\t$0x90\t// NOP " — Reply to this email directly, view it on GitHub <#1801>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABJAR3FM4OE7FUPFWB52NB3XISDHLANCNFSM6AAAAAAYSWBGXU> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[mibk]

OK, so the real issue is that it's just not supported yet. I believe it is a valid collation but is just an alias for utf8mb3_czech_ci, which seems to be indicated here in sql/collations.go:

	Collation_utf8_turkish_ci          = Collation_utf8mb3_turkish_ci
	Collation_utf8_czech_ci            = Collation_utf8mb3_czech_ci
	Collation_utf8_danish_ci           = Collation_utf8mb3_danish_ci

[timsehn]

@Hydrocharged will fix the panic and get the collation supported today. I'm not sure if he considered collation aliases, hence the panic.

[Hydrocharged]

Collation aliases are fully supported. The panic came from an oversight with enum and set, as those use the collation while creating the type, which occurs before we check if the collation has been implemented. I added a few more collation checks, so now we'll return the proper error (which just says to let us know if you need us to implement the collation).

The linked PR includes a fix for the panic, and it adds the missing collation as well.