Table IO Managers should capture column schemas with appropriate metadata tag

[slopp]

Table IO Managers should capture column schemas with appropriate metadata tag

[sryza]

Something for us to think about here is that there can be a difference between:

• The column types for the in-memory dataframe object
• The column types in the database.

E.g., in memory, a pandas string column will often have type "obj", while in the database it might have type "varchar".

To be consistent with the rest of the product, I think we'd want the main "Columns" section for the asset to show the column types as they appear in the database.

[jamiedemaria]

E.g., in memory, a pandas string column will often have type "obj", while in the database it might have type "varchar".

good point.

the main "Columns" section for the asset

this section being the one populated by dagster/column_schema correct'? if so we should just be able to add that metadata in. i think it might require adding a DB query to get the schema of the table though, depending on how each of the different DB's connector objects works and what they return on a successful create/update. i dont see issues with adding that though

[sryza]

this section being the one populated by dagster/column_schema correct'?

exactly