Saving parquet to Google Cloud Storage with `df.write_parquet()`

[nagomiso]

Saving parquet to Google Cloud Storage with `df.write_parquet()`

[ritchie46]

I agree that this is a great feature. We should add this natively on the rust side.

[Mmoncadaisla]

I'm not quite experienced with Polars and understand that this is already known and you'd like a more straightforward interaction (apologies for the noise if so), just in case it's any useful to anyone, it is possible to directly write parquet files to GCS (as well as to other storage providers)

import polars as pl
import gcsfs

# df = pl.read_parquet('file_path')

# Assuming `df` is your Polars DataFrame, and that GOOGLE_APPLICATION_CREDENTIALS env variable si correctly set
fs = gcsfs.GCSFileSystem()

# Define your GCS bucket and file path
destination = "gs://bucket/folder/file.parquet"

# Write the DataFrame to a Parquet file directly in GCS
with fs.open(destination, mode='wb') as f:
    df.write_parquet(f)