Is the Connection object thread-safe?

[snowflakedb]

Is the Connection object thread-safe?

[howryu]

Yes. Connection is thread-safe and it can be shared by multiple statements.

[gregakespret]

Awesome! Can we rely on this in the future as well (given that this is not documented anywhere)?

[howryu]

Yes. You can rely on it. If it does not work, it will be treated as a bug.

I will add documenting this to my todo list.

[gregakespret]

Perfect, thanks!

[vaibhavsingh007]

Hi, so I just ran into an issue where k records from across n Spark dataframes are written as k+y where y is non-deterministic, using concurrent write using same SF jdbc connection (drivers: spark-snowflake_2.11-2.5.2-spark_2.4.jar, snowflake-jdbc-3.9.1.jar).

write config:

SNOWFLAKE_SOURCE_NAME = "net.snowflake.spark.snowflake"
    
    # Set options below
    sfOptions = {
        "sfURL" : "****",
        "sfAccount" : "****", # Also needed create stage privilege.
        "sfUser" : username,
        "sfPassword" : password,
        "sfDatabase" : database,
        "sfSchema" : schema,
        "sfWarehouse" : warehouse,
        "sfRole" : role
    }
    
    sparkDF.write \
    .format(SNOWFLAKE_SOURCE_NAME) \
    .options(**sfOptions) \
    .option("dbtable", f"MY_WORKSPACE.{table}") \
    .mode('append') \
    .save()

How can I synchronize this to achieve data integrity?

[vaibhavsingh007]

@howryu any thoughts?