jrcribb/wrappers

Postgres Foreign Data Wrapper development framework in Rust.

Wrappers

Wrappers is a development framework for Postgres Foreign Data Wrappers (FDW), written in Rust. Its goal is to make Postgres FDW development easier while keeping Rust language's modern capabilities, such as high performance, strong types, and safety.

Wrappers is also a collection of FDWs built by Supabase. We currently support the following FDWs, with more under development:

FDW	Description	Read	Modify
HelloWorld	A demo FDW to show how to develop a basic FDW.
BigQuery	A FDW for Google BigQuery	✅	✅
Clickhouse	A FDW for ClickHouse	✅	✅
Stripe	A FDW for Stripe API	✅	✅
Firebase	A FDW for Google Firebase	✅	❌
Airtable	A FDW for Airtable API	✅	❌
S3	A FDW for AWS S3	✅	❌
Logflare	A FDW for Logflare	✅	❌
Auth0	A FDW for Auth0	✅	❌
SQL Server	A FDW for Microsoft SQL Server	✅	❌
Redis	A FDW for Redis	✅	❌

Features

• Minimum interface and easy to implement.
• Support for rich data types.
• Support both sync and async backends, such as RDBMS, RESTful APIs, flat files and etc.
• Built on top of pgrx, providing higher level interfaces, without hiding lower-level C APIs.
• WHERE, ORDER BY, LIMIT pushdown are supported.

Documentation

• Usage Docs
• Developer Docs (docs.rs)

Installation

Wrappers is a pgrx extension, you can follow the pgrx installation steps to install Wrappers.

Basically, run below command to install FDW after pgrx is installed. For example,

cargo pgrx install --pg-config [path_to_pg_config] --features stripe_fdw

Developing a FDW

To develop a FDW using Wrappers, you only need to implement the ForeignDataWrapper trait.

pub trait ForeignDataWrapper {
    // create a FDW instance
    fn new(...) -> Self;

    // functions for data scan, e.g. select
    fn begin_scan(...);
    fn iter_scan(...) -> Option<Row>;
    fn end_scan(...);

    // functions for data modify, e.g. insert, update and delete
    fn begin_modify(...);
    fn insert(...);
    fn update(...);
    fn delete(...);
    fn end_modify(...);

    // other optional functions
    ...
}

In a minimum FDW, which supports data scan only, new(), begin_scan(), iter_scan() and end_scan() are required, all the other functions are optional.

To know more about FDW development, please visit the Wrappers documentation.

Basic usage

These steps outline how to use the a demo FDW HelloWorldFdw, which only outputs a single line of fake data:

1. Clone this repo

git clone https://github.com/supabase/wrappers.git

2. Run it using pgrx with feature:

cd wrappers/wrappers
cargo pgrx run pg15 --features helloworld_fdw

3. Create the extension, foreign data wrapper and related objects:

-- create extension
create extension wrappers;

-- create foreign data wrapper and enable 'HelloWorldFdw'
create foreign data wrapper helloworld_wrapper
  handler hello_world_fdw_handler
  validator hello_world_fdw_validator;

-- create server and specify custom options
create server my_helloworld_server
  foreign data wrapper helloworld_wrapper
  options (
    foo 'bar'
  );

-- create an example foreign table
create foreign table hello (
  id bigint,
  col text
)
  server my_helloworld_server
  options (
    foo 'bar'
  );

4. Run a query to check if it is working:

wrappers=# select * from hello;
 id |    col
----+-------------
  0 | Hello world
(1 row)

Running tests

In order to run tests in wrappers:

docker-compose -f .ci/docker-compose.yaml up -d
cargo pgrx test --features all_fdws,pg15

Limitations

• Windows is not supported, that limitation inherits from pgrx.
• Currently only supports PostgreSQL v14, v15 and v16.
• Generated column is not supported.

Contribution

All contributions, feature requests, bug report or ideas are welcomed.

License

Apache License Version 2.0