Giter Site home page Giter Site logo

refinery's Introduction

refinery Logo

Powerful SQL migration toolkit for Rust.

Crates.io docs.rs MIT licensed Build Status


Refinery strives to make running migrations for different databases as easy as possible. It works by running your migrations on a provided database connection, either by embedding them on your Rust code, or via the refinery_cli.

Currently postgres, tokio-postgres , mysql, mysql_async, rusqlite and tiberius are supported. If you are using a driver that is not yet supported, namely SQLx you can run migrations providing a Config instead of the connection type, as Config impl's Migrate. You will still need to provide the postgres/mysql/rusqlite/tiberius driver as a feature for Runner::run and tokio-postgres/mysql_async for Runner::run_async. refinery works best with Barrel but you can also have your migrations in .sql files or use any other Rust crate for schema generation.

Usage

  • Add refinery to your Cargo.toml dependencies with the selected driver as feature eg: refinery = { version = "0.8", features = ["rusqlite"]}
  • Migrations can be defined in .sql files or Rust modules that must have a function called migration that returns a String.
  • Migrations can be strictly versioned by prefixing the file with V or not strictly versioned by prefixing the file with U.
  • Migrations, both .sql files and Rust modules must be named in the format [U|V]{1}__{2}.sql or [U|V]{1}__{2}.rs, where {1} represents the migration version and {2} the name.
  • Migrations can be run either by embedding them in your Rust code with embed_migrations macro, or via refinery_cli.

Example: Library

use rusqlite::Connection;

mod embedded {
    use refinery::embed_migrations;
    embed_migrations!("./tests/sql_migrations");
}

fn main() {
    let mut conn = Connection::open_in_memory().unwrap();
    embedded::migrations::runner().run(&mut conn).unwrap();
}

For more library examples, refer to the examples.

Example: CLI

NOTE:

  • Contiguous (adjacent) migration version numbers are restricted to u32 (unsigned, 32-bit integers).
  • Non-contiguous (not adjacent) migration version numbers are restricted to u32 (unsigned, 32-bit integers).
export DATABASE_URL="postgres://postgres:secret@localhost:5432/your-db"
pushd migrations
    # Runs ./src/V1__*.rs or ./src/V1__*.sql 
    refinery migrate -e DATABASE_URL -p ./src -t 1
popd

Example: Deadpool

let mut conn = pool.get().await?;
let client = conn.deref_mut().deref_mut();
let report = embedded::migrations::runner().run_async(client).await?;

Non-contiguous VS Contiguous migrations

Depending on how your project/team has been structured will define whether you want to use contiguous (adjacent) migrations V{1}__{2}.[sql|rs] or non-contiguous (not adjacent) migrations U{1}__{2}.[sql|rs]. If migration sequential numbering reflects the order they were developed and, they are deployed in the order they are numbered, you won't run into any problems using contiguous migrations. This is because you can be sure the next migration being run is always going to have a version number greater than the previous.

With non-contiguous migrations there is more flexibility in the order that the migrations can be created and deployed. If developer 1 creates a PR with a migration today U11__update_cars_table.sql, but it is reviewed for a week. Meanwhile, developer 2 creates a PR with migration U12__create_model_tags.sql that is much simpler and gets merged and deployed immediately. This would stop developer 1's migration from ever running if you were using contiguous migrations because the next migration would need to be > 12.

Implementation details

refinery works by creating a table that keeps all the applied migrations' versions and their metadata. When you run the migrations Runner, refinery compares the applied migrations with the ones to be applied, checking for divergent and missing and executing unapplied migrations.
By default, refinery runs each migration in a single transaction. Alternatively, you can also configure refinery to wrap the entire execution of all migrations in a single transaction by setting set_grouped to true.

Rollback

refinery's design was based on flyway and so, it shares its earlier philosophy on undo/rollback migrations. Flyway has since changed it's opinion but refinery hasn't. To undo/rollback a migration, you have to generate a new one and write specifically what you want to undo.

MSRV

refinery aims to support stable Rust, the previous Rust version, and nightly.

Async

Starting with version 0.2 refinery supports tokio-postgres, mysql_async and Tiberius For Rusqlite, the best way to run migrations in an async context is to run them inside tokio's spawn_blocking for example.

Contributing

🎈 Thanks for your help to improve the project! No contribution is too small and all contributions are valued, feel free to open Issues and submit Pull Requests.

License

This project is licensed under the MIT license.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in refinery by you, shall be licensed as MIT, without any additional terms or conditions.

refinery's People

Contributors

akhilles avatar aloucks avatar glueball avatar jxs avatar krakaw avatar lavrd avatar letscodehu avatar lorentzlasson avatar lucperkins avatar mcobzarenco avatar miro-andrin avatar mpiorowski avatar omid avatar otak avatar rockstar avatar samgj18 avatar serprex avatar spacekookie avatar stusmall avatar superstator avatar taqtiqa-mark avatar tmcw avatar tobiasdebruijn avatar tobymurray avatar tranzystorekk avatar ursmetz avatar vn971 avatar walfie avatar wseaton avatar xmac94x avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

refinery's Issues

Build broken with rusqlite 0.24

When depending on rusqlite 0.24 and refinery 0.3, theres a version conflict of the underlying libsqlite3-sys crate version. The dependency of rusqlite = {version = ">= 0.23, < 0.25", optional = true} in refinery-core does not seem to fix this.

Cargo.toml

# ...
[dependencies]
rusqlite = { version = "^0.24.1", features = ["uuid", "chrono", "serde_json", "load_extension"] }
refinery = { version = "^0.3.0", features = ["rusqlite"] }
# ...

Error

error: failed to select a version for `libsqlite3-sys`.
    ... required by package `rusqlite v0.23.0`
    ... which is depended on by `refinery-core v0.3.1`
    ... which is depended on by `refinery v0.3.0`
    ... which is depended on by mylib v0.2.2 (/home/rene/git/mylib)`
versions that meet the requirements `^0.18.0` are: 0.18.0

the package `libsqlite3-sys` links to the native library `sqlite3`, but it conflicts with a previous package which links to `sqlite3` as well:
package `libsqlite3-sys v0.20.1`
    ... which is depended on by `mylib v0.2.2 (/home/rene/git/mylib)`

failed to select a version for `libsqlite3-sys` which could resolve this conflict

Feature request: r2d2_sqlite connection pooling support

Does refinery have an option to support r2d2_sqlite for connection pooling?

The specific error I'm getting is as follows:

]error[E0277]: the trait bound `&mut r2d2::PooledConnection<r2d2_sqlite::SqliteConnectionManager>: refinery_migrations::traits::sync::Query<std::vec::Vec<refinery_migrations::AppliedMigration>>` is not satisfied
  --> wopplebloxd/src/db/migrations.rs:23:44
   |
23 |         embedded::migrations::runner().run(&mut connection);
   |                                            ^^^^^^^^^^^^^^^ the trait `refinery_migrations::traits::sync::Query<std::vec::Vec<refinery_migrations::AppliedMigration>>` is not implemented for `&mut r2d2::PooledConnection<r2d2_sqlite::SqliteConnectionManager>`
   |
   = note: required because of the requirements on the impl of `refinery::Migrate` for `&mut r2d2::PooledConnection<r2d2_sqlite::SqliteConnectionManager>`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0277`.
error: could not compile `wopplebloxd`.

To learn more, run the command again with --verbose.

I'm having to use connection pooling here because I'm setting up a web server with actix, and I'm storing my db connection in my global state object, which is access by multiple threads I assume that actix is spawning.

"ERROR: no database specified" when trying to create migrations table

I am getting

Error: `error asserting migrations table`, `db error: ERROR: no database specified`

when refinery tries to create the migrations table. What database should the migrations table be created in?

This is on linux, against CockroachDB (using postgres driver) -- I suspect the latter is more pedantic than postgres about specifying a database (but it's just a theory).

Building under stable-x86_64-pc-windows-gnu fails

Hi,

Currently refinery seems to fail to build under stable-x86_64-pc-windows-gnu and nightly-x86_64-pc-windows-gnu.

stable-x86_64-pc-windows-msvc on the other hand builds refinery fine.

Error message:

   Compiling refinery v0.2.1
error: could not compile `refinery`.

Caused by:
  process didn't exit successfully: `rustc 
--crate-name refinery 
--edition=2018 'C:\Users\almetica\.cargo\registry\src\github.com-1ecc6299db9ec823\refinery-0.2.1\src\lib.rs' 
--error-format=json 
--json=diagnostic-rendered-ansi,artifacts 
--crate-type lib 
--emit=dep-info,metadata,link -C debuginfo=2 
--cfg 'feature="default"' 
--cfg 'feature="postgres"' -C metadata=b4b0923c9635f1a1 -C extra-filename=-b4b0923c9635f1a1 
--out-dir 'C:\Users\almetica\Documents\Development\almetica\almetica\target\debug\deps' -L 'dependency=C:\Users\almetica\Documents\Development\almetica\almetica\target\debug\deps' 
--extern 'barrel=C:\Users\almetica\Documents\Development\almetica\almetica\target\debug\deps\libbarrel-c14e7acdbe063085.rmeta' 
--extern 'refinery_macros=C:\Users\almetica\Documents\Development\almetica\almetica\target\debug\deps\refinery_macros-fdbe15d768c175f7.dll' 
--extern 'refinery_migrations=C:\Users\almetica\Documents\Development\almetica\almetica\target\debug\deps\librefinery_migrations-ff4312ab516cca44.rmeta' 
--cap-lints allow -C target-cpu=native -L 'native=C:\Users\almetica\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-x86_64-pc-windows-gnu-0.4.0\lib'` 
(exit code: 0xc0000005, STATUS_ACCESS_VIOLATION)
warning: build failed, waiting for other jobs to finish...
error: build failed

Maybe rust-lang/rust#33434 is related?

Edit: --release works fine. So only debug build is affected. I will try to pinpoint the exact cause further.

rusqlite version bump?

Currently refinery depends on rusqlite 0.18, while the latest version is 0.21. Because rusqlite links to a native library, it is not possible to have multiple versions of it in the same build. Consider upgrading to the latest version of rusqlite.

Lack of ability to change logic of detection of earlier applied migrations

My project used to have self-written migration manager, now I'm trying to figure out if I can explain to refinery how my migration version control is organized. In other words, before refinery retrieves previously applied migration versions, I need to check if database uses old fashion version control and if it does, retrieve applied versions in old-fashion way and make refinery think that those migrations have been applied as if refinery had applied them by itself.

Diesel_migrations exposes trait MigrationConnection so I can simply wrap it and make it know if some migrations have been applied earlier. In contrast, refinery has Query trait (which is subset of MigrationConnection) but hides it.

I think idea of having MigrationConnection is good enough to reuse.

How to use Migrate, MigrateGroup traits for sqlite?

I'm getting an error like this:

error[E0277]: the trait bound `rusqlite::Connection: refinery::MigrateGrouped<'_>` is not satisfied
  --> src/sqldata.rs:61:24
   |
61 |   migrations::runner().run(&mut conn).map_err(|e| e.into())
   |                        ^^^ the trait `refinery::MigrateGrouped<'_>` is not implemented for `rusqlite::Connection`

error[E0277]: the trait bound `rusqlite::Connection: refinery::Migrate` is not satisfied
  --> src/sqldata.rs:61:24
   |
61 |   migrations::runner().run(&mut conn).map_err(|e| e.into())
   |                        ^^^ the trait `refinery::Migrate` is not implemented for `rusqlite::Connection`

What should I do to get these traits into scope for rusqlite::Connection?

In my Cargo.toml I added these (I'm also adding barrel)

barrel = { version = "0.6.2", features = ["sqlite3"] }
refinery = { version = "0.1.10", feature = ["rusqlite"] }

And in the file where I'm getting the error, I added some new 'use' stmts:

use barrel::backend::Sqlite;
use barrel::{types, Migration};
use migrations;
use refinery::embed_migrations;
use refinery::{Migrate, MigrateGrouped};
use rusqlite::{params, Connection};

And the actual migration code looks like this:

mod embedded {
  use refinery::embed_migrations;
  embed_migrations!("src/migrations");
}

pub fn dbinit(dbfile: &Path) -> Result<(), Box<dyn Error>> {
  let conn = Connection::open(dbfile)?;

  migrations::runner().run(&mut conn).map_err(|e| e.into())
}

I have a PR open on this project with all my changes:

bburdette/pdfq#4

Return details about applied migrations

Currently, performing migrations does not return any information. It may be useful to return information about the migrations that were successfully applied during a run.

My use case here is simply that I'd like to see what, if anything, was changed as a result of running a migration. At present I manually query the refinery_schena_history for and after applying migrations in order to find out what changed.

The refinery_schema_history version colum is too small for time based versions.

Hi,

It's pretty common to use time based version numbers for migration files (e.g. migrations of Active Record or Diesel). The current implementation of refinery is only using the column type INT4 for the version column, which is too small to hold the current time based on a YEAR-MONTH-DAY-HOUR-MINUTE basis.

Is there any special reason not to use INT8 for the version column? The rust code seems to use a i32, so a switch to i64 should be possible.

(I'm using a postgres database).

Provide example of how to use SQLx with SQLite

Hey there,
the README currently states that

If you are using a driver that is not yet supported, namely SQLx you can run migrations providing a Config instead of the connection type, as Config impl's Migrate. You will still need to provide the postgres/mysql/rusqlite driver as a feature for Runner::run and tokio-postgres/mysql_async for Runner::run_async.

Having never used this crate i can't quite make sense of that passage. I see that you could provide a Config to the Runner::run method. But i'm not sure how to initialize it, considering i want to use SQLx with SQLite. And how can i pass the the rusqlite driver along with the Config?

An example, or some pointers, of how to use refinery alongside SQLx and SQLite would be greatly appreciated.

no method named `run` found for struct `refinery::Runner` in the current scope

Hello,

I'm a Rust beginner, and I've tried to set this crate up to do migrations on my database. Unfortunately, I'm getting the following error:

error[E0599]: no method named `run` found for struct `refinery::Runner` in the current scope
  --> wopplebloxd/src/db/migrations.rs:22:40
   |
22 |         embedded::migrations::runner().run(&mut connection);
   |                                        ^^^ method not found in `refinery::Runner`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0599`.

For context, here's my migrations.rs file:

use super::definitions::{ Connection };
// use rusqlite::Connection;

mod embedded {
    use refinery::embed_migrations;
    
    embed_migrations!("wopplebloxd/migrations");
}

pub struct SqliteMigrator;

impl Default for SqliteMigrator {
    fn default() -> SqliteMigrator {
        SqliteMigrator {
            
        }
    }
}

impl SqliteMigrator {
    pub fn migrate(&self, connection : &mut Connection) {
        embedded::migrations::runner().run(&mut connection);
    }
}

....which references definitions.rs:

pub type Pool = r2d2::Pool<r2d2_sqlite::SqliteConnectionManager>;
pub type Connection = r2d2::PooledConnection<r2d2_sqlite::SqliteConnectionManager>;

I'm using r2d2 & r2d2-sqlite, as I'm building a web app with actix. If I don't do that, then I get errors because the plain-old rusqlite::Connection can't be shared across multiple threads (which are spawned automatically by actix).

I also have a migrations folder that contains a single .sql file called V1__initial.sql.

.sql migrations only allow a single statement

Nice project, I tried to use it with one of my projects but stumbled across a limitation in the refinery_cli. It only allows a single statement in any .sql file. In my case I want to create multiple tables but it fails with the error

Error: error applying migration V2__demo, database error: ERROR: cannot insert multiple commands into a prepared statement

.sql file:

create table roles (
    role varchar(32) PRIMARY Key,
);

create table powers (
    role_id SERIAL PRIMARY KEY,
);

support for mongo?

Hi, sorry if this already answered here somewhere, but I am curious if support for mongodb and other NoSql databases is on the roadmap or if I should be looking at another tool for doing mongo migrations for my rust micro-services.

Thanks!

Feature Request (or Question?): Support deadpool / tokio postgres

I would appreciate some help with my poor understanding of the type system...

let mut client = db_pool.get().await?.deref_mut().deref_mut();
    let report = embedded::migrations::runner()
        .run_async(&mut client)
        .await?;

using deadpool with tokio-postgres and I get the following compile error (that I do not understand, really :)

 the trait `refinery::AsyncMigrate` is not implemented for `&mut tokio_postgres::client::Client`

merge refinery_migrations on refinery crate?

afair, when we started developing refinery, refinery_cli used functions from refinery that were not meant to be public, since refinery_cli also has to be an independent crate , we moved them to refinery_migrations.
We have since pub'ed migrate_from_config, so that reason doesn't seem to be legit anymore.
from a quick glance at refinery_cli it seems the only function that is still used without being doc'ed and oficially pub'ed is find_migration_filenames, maybe we can also make it pub and document it to be able to be used with migrate_from_config, and then merge refinery_migrations into refinery?

No migrations to apply

Hello, I seem to have similar problem as #115.

➤ exa migrations/
V1__initial.sql  V2__serverid.sql

➤ env RUST_LOG=info cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.13s
     Running `target/debug/neomason-discord-bot`
[2020-07-11T18:21:28Z INFO  refinery_core::traits] schema history table is empty, going to apply all migrations
[2020-07-11T18:21:28Z INFO  refinery_core::traits::sync] applying migration: V1__initial
...

Despite having two migrations to run, refinery only run one. I tried looking at RUST_LOG=debug output, but it didn't provide any additional output.
I also assume I'm doing something wrong, but my file seems to be named correctly.

Do you have any suggestions?

Compiler crashes on windows 10

Hello,

Adding refinery 0.3.0 to Cargo.toml on windows 10 crashes rustc 1.47.0 (18bf6b4f0 2020-10-07).
I don't know if this is known and do not know how to report this kind of issue to rust-lang.

Edit: Problem still happens with 0.4.0

Error compiling refinery-migrations

I get the following compiler error while trying to compile a "hello world" project using mysql_async:

Compiling refinery-migrations v0.2.0
error: expected an item keyword
   --> C:\Users\####\.cargo\registry\src\github.com-1ecc6299db9ec823\refinery-migrations-0.2.0\src\config.rs:259:21
    |
259 |                     let url = build_db_url("mysql", &config);
    |                     ^^^

error: aborting due to previous error

error: could not compile `refinery-migrations`.

To learn more, run the command again with --verbose.

Please, find attached a minimal example that shows the error. Doing a cargo build (with up to date stable rust) produces the given error.

test.zip

Migration files are no longer detected when using the master branch

While investigating an int conversion issue when using CockroachDB, I noticed that migrations didn't work at all on the master branch:

INFO  refinery_migrations::traits::r#async > no migrations to apply

I'll see this even after dropping all the tables, including the schema migrations table.

This seems to be a regression from the version on crates.io. I didn't see a branch for the older version but was able to dig out the release commit for 0.2. I applied the int conversion patch to confirm that the old version works as expected (and it does).

https://github.com/aloucks/refinery/tree/version-i64

Build error trait Query<AppliedMigration> is not implemented for `postgres::client::Client`

I'm just starting with Rust and hope this is not some dumb user error, but I'm confident I did the same as code in docs and tests.
I couldn't check if tests work though because I could not make them run locally as well (See also #71).

cargo run --bin migrations
   Compiling barrel v0.6.6-alpha.0 (https://github.com/belak/barrel#504e8b7d)
   Compiling database_core v0.1.0 (/home/user/source/bwu_app/database_core)
error[E0277]: the trait bound `postgres::client::Client: refinery_core::traits::sync::Query<std::vec::Vec<refinery_core::runner::AppliedMigration>>` is not satisfied
  --> database_core/src/mig.rs:16:30
   |
16 |     migrations::runner().run(&mut client).unwrap();
   |                              ^^^^^^^^^^^ the trait `refinery_core::traits::sync::Query<std::vec::Vec<refinery_core::runner::AppliedMigration>>` is not implemented for `postgres::client::Client`
   |
   = note: required because of the requirements on the impl of `refinery_core::traits::sync::Migrate` for `postgres::client::Client`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0277`.
error: could not compile `database_core`.

To learn more, run the command again with --verbose.
active toolchain
----------------

nightly-x86_64-unknown-linux-gnu (default)
rustc 1.43.0-nightly (dd6718796 2020-03-16)
[dependencies]
barrel = { git="https://github.com/belak/barrel", features = ["pg"]} # contains a PR that fixes a build error
postgres = "0.17.2"
refinery = { git = "https://github.com/rust-db/refinery" } 

Run migration in seleceted schema in postgresql

How can I run migrations in a selected schema postgresql? Always it runs the migrations in the public schema and I don't see how can I choose it in the documentation.

Update:
Even if I choose schema with barrel, doesn't work

roadmap following 0.1

for a 0.2 I think then we could:

  • starting support for async drivers
  • cli option to run mod migrations? if so do you have ideas for this?

Config from a string?

I'm going to preface this with I'm a rust newb. I am trying to build a server app using actix/sqlx and I'm using a custom config struct and all I want to do is pass a postgres url to configure refinery - but there doesn't seem to be a way to do that.

Unable to run migrations on database with `?sslmode=require` using `Config`

Because refinery config does not support database params, my current code looks like this:

    let url = Url::parse(&db_url).context("invalid DATABASE_URL")?;
    let mut db = Config::new(ConfigDbType::Postgres);
    if !url.username().is_empty() {
        db = db.set_db_user(url.username());
    }
    if let Some(password) = url.password() {
        db = db.set_db_user(password);
    }
    if let Some(host) = url.host_str() {
        db = db.set_db_host(host);
    }
    if let Some(port) = url.port() {
        db = db.set_db_port(&port.to_string());
    }
    let mut db_name = url.path().trim_start_matches('/').to_string();
    if let Some(query) = url.query() {
        db_name.push('?');
        db_name.push_str(query); 
        // ^^^^^  this causes the database url to be correct,
        //        but is results in failure to connect over TLS
    }
    if !db_name.is_empty() {
        db = db.set_db_name(&db_name);
    }

Cargo.toml

[dependencies]
refinery = { version = "0.3.0", features = ["postgres"] }

Error

Error: `could not connect to database`, `error performing TLS handshake: no TLS implementation configured`

Caused by:
    0: error performing TLS handshake: no TLS implementation configured
    1: no TLS implementation configured

Clarify license

The README.md implies that the project is licensed only under MIT:

License

This project is licensed under the MIT license.

But the Cargo.toml manifests suggest it's dual licensed under MIT or Apache-2.0:

[package]
name = "refinery"
version = "0.2.0"
authors = ["Katharina Fey <[email protected]>", "João Oliveira <[email protected]>"]
license = "MIT OR Apache-2.0"

I think this should be clarified. If dual license was the intent, I'd follow the recommendations at https://rust-lang.github.io/api-guidelines/necessities.html#crate-and-its-dependencies-have-a-permissive-license-c-permissive

Ability to migrate to a specific version?

Hello, and thanks for this crate!

I was wondering whether it's on the roadmap to support running migrations up to a specific version, rather than all the way. If not, I'd be curious to know what your stance on the matter is (particularly because I am fairly inexperienced at the DB layer, so any insight is very welcome)!

Warning about malformed migration names?

Hey! So, today I stumbled about possible improvement to how migration loading is handled. Currently, when using the sql-files based approach, refinery does not report SQL files in the migrations directory that do not conform to the naming convention. It wont say a word about having something like "V1_name" instead of "V1__proper_name" (bad version prefix). Maybe this is a possible improvement to make?:)

Feature Request: Renaming migrations table

I'd like for it to be possible to change the table name (and possibly the schema name) of the migrations table.

This could potentially be accomplished by passing a config argument to the runner api; or alternatively by allowing optional arguments in the "include" macros.

refinery does not share its perspective on undo with Flyway

From the readme/Rollback section:

refinery's design is based on flyway and so, shares its perspective on undo/rollback migrations. To undo/rollback a migration,
you have to generate a new one and write specifically what you want to undo.

However according to this, undo is actually supported by Flyway.

Personally, I don't see how it is possible to switch between different branches of my project and work with the same database if refinery does not allow you to rollback changes.

My proposal is it to:

  • add migrations prefixed with "U" as in Flyway
  • add "refinery migrate undo" command

Expose features to select db drivers

I was planning to try refinery_cli for some postgres work, but noticed the crate depends on libsqlite3 whether or not I want to use sqlite.

I didn't have it installed on my system at the time, so the build failed. I thought I'd mention it here as a feature request. Seems like others may have specific dbs in mind and will also want to pick and choose.

Thanks!

refinery-migrations 0.2.3 is a breaking change

refinery 0.2.1 depends on refinery-migrations ^0.2.2. refinery-migrations 0.2.2 depends on rusqlite ^0.21, which depends on libsqlite3-sys ^0.17. refinery-migrations 0.2.3 upgraded from rusqlite ^0.21 to rusqlite ^0.23, which also upgraded libsqlite-3sys to ^0.18.

The refinery-migrations update breaks builds for people who use other crates which currently depend on a different version of libsqlite3-sys, like current sqlx-core using ^0.17. The rusqlite upgrade should've been done in a new major version (0.3) of refinery-migrations and refinery.

No migrations to apply

Hi,

I assume I'm doing something wrong, but Refinery doesn't seem to want to apply my migrations.

jkaye@jkaye-linux:~/git/wanderlust$ cat refinery.toml 
[main]
db_type = "Postgres"
db_host = "localhost"
db_port = "5432"
db_user = "postgres"
db_pass = "..."
db_name = "postgres"

jkaye@jkaye-linux:~/git/wanderlust$ ls migrations/
1_create_users.sql

jkaye@jkaye-linux:~/git/wanderlust$ cat migrations/1_create_users.sql 

create table external_auth_source
(
        external_auth_source_id serial primary key,
        source_name varchar(255) not null,
        constraint ux_external_auth_source_name unique(source_name)
);

create table traveler
(
        traveler_id serial primary key,
        external_auth_source_id int not null foreign key references external_auth(external_auth_source_id),
        external_auth_id varchar(255) not null,
        constraint ux_traveler_external_auth unique(external_auth_source_id, external_auth_id)
);

jkaye@jkaye-linux:~/git/wanderlust$ refinery migrate files
schema history table is empty, going to apply all migrations
no migrations to apply

Any ideas?

Config object does not accept url as db_path

Hello,

First, thanks for the library.
I've tried to use this method with a postgresql database to start migration:

let mut db_config = Config::new(ConfigDbType::Postgres).set_db_path(env::var("postgres_pool").unwrap().as_str());

My env variable is with an url format. I've also tried key value format but i could not connect to my database. Here is the error message:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { kind: Connection("could not connect to database", Error { kind: Config, cause: Some("host missing") }), report: None }',

Am I using set_db_path method correctly? If not, is there a way to pass url or key value postgres path to refinery Config?

Thanks !

Feature Request: Access to DB connection inside migration

I've been searching around for some way to manage my database migrations in Rust. I'm considering using Diesel, but Diesel only supports defining migrations in SQL. I anticipate wanting to make migrations which would be difficult or impractical to represent in SQL alone - modifying blobs, making complex format changes, etc. - so that is a non-starter for me.

Refinery's README says that Refinery's design is based on Flyway and it supports defining migrations in Rust code instead of SQL, which looked promising. Flyway's document for Java-based migrations says explicitly that this kind of use case is what they're targeted at:

Java-based migrations are a great fit for all changes that can not easily be expressed using SQL.

These would typically be things like

  • BLOB & CLOB changes
  • Advanced bulk data changes (Recalculations, advanced format changes, …)

However, it doesn't seem Refinery's "migrations in a Rust module" support gets me what I need. For example, see Flyway's getting started tutorial for Java-based migrations: https://flywaydb.org/getstarted/java. Implementing this tutorial in the same way does not seem to be possible in Refinery, as the migration would need access to the database connection. (The specific modification in the example is trivial to implement in SQL, but imagine calculating the updated name was an operation that was impractical in SQL.)

Are there any plans to support more complex, manually coded migrations in Refinery?

Automatic Migration Generation

Django and other more mature ORMs such as TypeORM can automatically generate migrations based on the data structure. This increases productivity and frees users from having to keep DB layout and code data structures in sync.

In Enterprise settings, there are tooling that take the declarative database management concepts even further for more robust database management: https://docs.microsoft.com/en-us/sql/relational-databases/data-tier-applications/data-tier-applications?view=sql-server-2017

However for simple use cases the Django style automigrations will suffice.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.