golang-migrate/migrate

Database migrations. CLI and Golang library.

migrate

Database migrations written in Go. Use as CLI or import as library.

• Migrate reads migrations from sources and applies them in correct order to a database.
• Drivers are "dumb", migrate glues everything together and makes sure the logic is bulletproof. (Keeps the drivers lightweight, too.)
• Database drivers don't assume things or try to correct user input. When in doubt, fail.

Forked from mattes/migrate

Databases

Database drivers run migrations. Add a new database?

• PostgreSQL
• PGX v4
• PGX v5
• Redshift
• Ql
• Cassandra / ScyllaDB
• SQLite
• SQLite3 (todo #165)
• SQLCipher
• MySQL / MariaDB
• Neo4j
• MongoDB
• CrateDB (todo #170)
• Shell (todo #171)
• Google Cloud Spanner
• CockroachDB
• YugabyteDB
• ClickHouse
• Firebird
• MS SQL Server
• rqlite

Database URLs

Database connection strings are specified via URLs. The URL format is driver dependent but generally has the form: dbdriver://username:password@host:port/dbname?param1=true&param2=false

Any reserved URL characters need to be escaped. Note, the % character also needs to be escaped

Explicitly, the following characters need to be escaped: !, #, $, %, &, ', (, ), *, +, ,, /, :, ;, =, ?, @, [, ]

It's easiest to always run the URL parts of your DB connection URL (e.g. username, password, etc) through an URL encoder. See the example Python snippets below:

$ python3 -c 'import urllib.parse; print(urllib.parse.quote(input("String to encode: "), ""))'
String to encode: FAKEpassword!#$%&'()*+,/:;=?@[]
FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D
$ python2 -c 'import urllib; print urllib.quote(raw_input("String to encode: "), "")'
String to encode: FAKEpassword!#$%&'()*+,/:;=?@[]
FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D
$

Migration Sources

Source drivers read migrations from local or remote sources. Add a new source?

• Filesystem - read from filesystem
• io/fs - read from a Go io/fs
• Go-Bindata - read from embedded binary data (jteeuwen/go-bindata)
• pkger - read from embedded binary data (markbates/pkger)
• GitHub - read from remote GitHub repositories
• GitHub Enterprise - read from remote GitHub Enterprise repositories
• Bitbucket - read from remote Bitbucket repositories
• Gitlab - read from remote Gitlab repositories
• AWS S3 - read from Amazon Web Services S3
• Google Cloud Storage - read from Google Cloud Platform Storage

CLI usage

• Simple wrapper around this library.
• Handles ctrl+c (SIGINT) gracefully.
• No config search paths, no config files, no magic ENV var injections.

CLI Documentation

Basic usage

$ migrate -source file://path/to/migrations -database postgres://localhost:5432/database up 2

Docker usage

$ docker run -v {{ migration dir }}:/migrations --network host migrate/migrate
    -path=/migrations/ -database postgres://localhost:5432/database up 2

Use in your Go project

• API is stable and frozen for this release (v3 & v4).
• Uses Go modules to manage dependencies.
• To help prevent database corruptions, it supports graceful stops via GracefulStop chan bool.
• Bring your own logger.
• Uses io.Reader streams internally for low memory overhead.
• Thread-safe and no goroutine leaks.

Go Documentation

import (
    "github.com/golang-migrate/migrate/v4"
    _ "github.com/golang-migrate/migrate/v4/database/postgres"
    _ "github.com/golang-migrate/migrate/v4/source/github"
)

func main() {
    m, err := migrate.New(
        "github://mattes:personal-access-token@mattes/migrate_test",
        "postgres://localhost:5432/database?sslmode=enable")
    m.Steps(2)
}

Want to use an existing database client?

import (
    "database/sql"
    _ "github.com/lib/pq"
    "github.com/golang-migrate/migrate/v4"
    "github.com/golang-migrate/migrate/v4/database/postgres"
    _ "github.com/golang-migrate/migrate/v4/source/file"
)

func main() {
    db, err := sql.Open("postgres", "postgres://localhost:5432/database?sslmode=enable")
    driver, err := postgres.WithInstance(db, &postgres.Config{})
    m, err := migrate.NewWithDatabaseInstance(
        "file:///migrations",
        "postgres", driver)
    m.Up() // or m.Step(2) if you want to explicitly set the number of migrations to run
}

Getting started

Go to getting started

Tutorials

• CockroachDB
• PostgreSQL

(more tutorials to come)

Migration files

Each migration has an up and down migration. Why?

1481574547_create_users_table.up.sql
1481574547_create_users_table.down.sql

Best practices: How to write migrations.

Coming from another db migration tool?

Check out migradaptor. Note: migradaptor is not affiliated or supported by this project

Versions

Version	Supported?	Import	Notes
master	✅	import "github.com/golang-migrate/migrate/v4"	New features and bug fixes arrive here first
v4	✅	import "github.com/golang-migrate/migrate/v4"	Used for stable releases
v3	❌	import "github.com/golang-migrate/migrate" (with package manager) or import "gopkg.in/golang-migrate/migrate.v3" (not recommended)	DO NOT USE - No longer supported

Development and Contributing

Yes, please! Makefile is your friend, read the development guide.

Also have a look at the FAQ.

---

Looking for alternatives? https://awesome-go.com/#database.

migrate's Issues

error: pq: column "dirty" does not exist in line 0

Hi,

I googled this error, the original repo has something on it, but it's not clear how to fix it.
I am using the binary from this repo:

go get -u -d github.com/golang-migrate/migrate/cli github.com/lib/pq
go build -tags 'postgres' -o /usr/local/bin/migrate github.com/golang-migrate/migrate/cli

With only this migration, on an empty postgres database:

BEGIN;
CREATE TABLE user (
  id uuid not null,
  sub varchar(255) not null,
  secret varchar(255),
  given_name varchar(255),
  family_name varchar(255),
  email varchar(255) not null,
  inserted_at timestamp without time zone not null default now(),
  updated_at timestamp without time zone not null default now()
);
create unique index user_id_index on user (id);
create unique index person_sub_index on person (sub);
COMMIT;

and it bombes out with the above error.

Syntax error if multiple queries in a sql file

I've these queries in my migration file.
CREATE TABLE users ( id BIGINT NOT NULL PRIMARY KEY AUTO_INCREMENT, name varchar(48), email varchar(24), created_at timestamp NOT NULL DEFAULT NOW(), updated_at timestamp NOT NULL DEFAULT NOW(), created_by varchar(64) NOT NULL, updated_by varchar(64) NOT NULL ); CREATE INDEX user_email_indx ON users(email);
When I run Up() it complains that there is an error near the beginning of CREATE INDEX.
Am I missing something?

New Driver Interface

Goals:

1. Configurable timeouts
   • Granularity of configuration? e.g. different timeouts for different operations
2. No possibility of dangling locks. See: #13 (comment)

Possible Goals:

1. Composability
   • Allow drivers to be composed of another driver(s)
     • maybe expose a methods for getting:
       • the db connection (*sql.Conn or *sql.DB or some interface) - for overwriting migration management queries
       • the config - for shared config
2. Safety
   • Struct fields should not be exported unnecessarily. e.g many DB driver Config structs have all of their fields exported. This is dangerous since the DB driver has a pointer to the Config, whose exported field values may be changed inadvertently. Another option would be to keep a copy of the Configstruct in the DB driver, but if any of the Config struct fields are pointers, we have the same problem...
     • Add NewConfig() method for creating configs with unexported struct fields.
     • Unexported Config struct fields may make sharing config between composed/dependent drivers harder

Ideas:

1. Require context.Context as the first param for Driver receiver functions
   1. Which receiver functions should require context?
   2. How does a context timeout/cancel affect the DB state? Is it consistent across all DBs?
   3. Which DBs support context.Context?
   4. Will break DB drivers not explicitly maintained by this repo
2. Add retry with timeout helper for DBs that don't support acquiring a lock within a timeout
3. Make the Open method on the Driver interface take a url.URL struct instead of a string
   • See: #69
   • Prevents double parsing of the url string
4. Run each migration in a transaction managed by migrate. See: #196
   • Safely support RDBMSs that don't support transactional DDL
5. Add EnsureVersionTable() method
   • See: #226 (comment)
6. Add a Metadata() method
   • See: #319 (comment)
7. Remove Drop() method
   • See: #193 (comment)
8. Create DB
   • See: #347
   • Would require migrating migration storage to it's own DB in a DBMS and tracking the migration status for a DB. e.g. the schema migration table would need to track the latest migration, the db, and if it's dirty
     • To be backwards compatible, we'd need to support migrations for the schema migration table
9. Use migrate to manage version and metadata table schemas
10. Return concrete types in DB driver implementation with compile time type checks
    • See: #583 (comment)

Trouble installing cli for postgres

I am trying to try out the migrate CLI, and am running into a problem with the installation steps in the README. I run these commands:

$ go get -u -d github.com/golang-migrate/migrate/cli github.com/lib/pq
$ go build -tags 'postgres' -o /usr/local/bin/migrate github.com/golang-migrate/migrate/cli
go build github.com/golang-migrate/migrate/database/postgres: no buildable Go source files in {GOPATH}/src/github.com/golang-migrate/migrate/database/postgres

If I run the go build with a tag other than postgres it works.

Incorrect help text?

migrate/cli/main.go

Line 117 in faf4307

formatPtr := createFlagSet.String("format", "", "Specify the format of the version using a Go time format string. For yyyymmddhhmmss use format of 20060102150405. If not specified the version will be the unix timestamp.")

Specifically the sub-string: For yyyymmddhhmmss use format of 20060102150405.
Seems to me this should instead read as: For 20060102150405 use format of yyyymmddhhmmss.

Getting error - "no buildable Go source files" in golang-migrate/migrate/database/postgres

I have included "github.com/golang-migrate/migrate" in Gopkg.toml but strangely getting this error on build.

Add `new` command for creating revisions

Migrate does not provide new command for creating revisions right now, so developers have to create revisions manually, which is tedious and error prone.

I think that creating a new revision should be as simple as:
migrate new -title "create-user-table" -source file://.migrations/versions -database postgres://localhost:5432.

The command above creates two file in ./migrations/versions named $TIMESTAMP_$TITLE.up.sql and $TIMESTAMP_$TITLE.down.sql, then users can write migration scripts in those files.

This could simplify process for creating revisions and avoid nasty issues created by manually creating revision files.

DROP command denied

Hey,

I'm wondering why the tool requires DROP privileges? As I am getting the following error when using migrate -database database_string -path migrations/ up:

error: Error 1142: DROP command denied to user 'db_user'@'host' for table 'schema_migrations' in line 0: TRUNCATE `schema_migrations`

pq: unterminated dollar-quoted string at or near

Getting this error - pq: unterminated dollar-quoted string at or near

create or replace function update_tsvector() returns trigger
as $update_tsvector$
begin
  UPDATE article SET tsv = setweight(to_tsvector(name), 'A');
  RETURN NULL;
end;
$update_tsvector$ LANGUAGE plpgsql;


CREATE TRIGGER update_tsvector
AFTER INSERT OR update of name ON article
FOR EACH ROW EXECUTE PROCEDURE update_tsvector();

Please help to resolve this.

Retrieving migration files residing in a non-master git branch

I am trying to run migrate on a github source:
migrate -source "github://friahi65:key@myrepo/ddi.core.configcache/db/migrations#toolkit" -database $DATABASE_URL up

where after # I am specifying the branch, however, it seems like it does not get parsed as I get the following error:
GET https://api.github.com/repos/myrepo/ddi.core.configcache/contents/db/migrations: 404 Not Found []
with no mention of branch. Could someone please let me know how I can get the migration files residing in a specific branch?
PS: the curl to github api works like this:
curl --user "friahi65:key" "https://api.github.com/repos/myrepo/ddi.core.configcache/contents/db/migrations?ref=toolkit"

Thanks,
Fatemeh

Rebuild release v3.1.0

v3.1.0 reports its version as 3.0.1:

root@localhost:~$ migrate -version
3.0.1

Add support for vgo (go.mod file)

Will make it easier for other projects to use this with vgo.

It should be simple as there is already a Gopkg.lock file: vgo install ./...

Suuport for Firebird SQL

Any plans on supporting Firebird SQL?

https://github.com/nakagami/firebirdsql
works well with
https://golang.org/pkg/database/sql/

CockroachDB driver test sporadically hangs

Support sqlite in released CLI

Currently, SQLite is not built into the release CLI.

It means that SQLite is not officially supported by migrate since it's not in our test suite. This is due to issues with cross-compiling and CGO. mattn/go-sqlite3, the most popular sqlite driver and only one that is included in the compatibility test, requires cgo. In fact, it looks like all golang sqlite drivers require cgo at this time.

References:

• 15611d4
• mattes/migrate#165
• mattes/migrate#244
• mattes/migrate#238

So if you want to use sqlite with the migrate CLI, you'll need to build it yourself or use the docker image.

Statically linked binaries in release packages

Hi there,

could you please recompile latest binaries with statically linking. Current version cannot use in alpine OS.

FROM alpine:latest

ENV MIGRATE_VERSION=3.2.0
RUN set -xe && \
	curl -Ls https://github.com/golang-migrate/migrate/releases/download/v${MIGRATE_VERSION}/migrate.linux-amd64.tar.gz | \
	tar -xz -C /usr/local/bin && \
	mv /usr/local/bin/migrate.linux-amd64 /usr/local/bin/migrate

cli fails with "source driver: unknown driver gcs (forgotten import?)"

When I run the 3.2.0 cli version of gomigrate, I get:

$ ./gomigrate -source gcs://some-bucket/some-prefix -database postgres://localhost:5432/some_db up
error: source driver: unknown driver gcs (forgotten import?)

Provide some better authentication method

I was trying to make work this tool with my Postgres DB, but I found that I need to specify explicitly in the -database URL parameter my user and password, like this:

migrate -database postgres://user:pass@localhost:5432/my-db -path src/github.com/paveltrufi/my-project/database/migrations/ up

So I was wondering if you would be able to add some parameters to the CLI in order to avoid writing my credentials in the DB URL. It could be even better if by default you would pick the setting from and .env file or something like this.

BTW it seems like the CLI tool enforces you to use SSL on the connection (I had to configure a self-signed certificate for my dockerized DB which was kinda difficult but I eventually managed to do it). Is there any way to make a migration without the SSL enabled?

Thanks in advance.

feat(mysql): try creating database if not exist

refer to: mattes/migrate#279

I created an issue for this because I'm interested in having this feature. Thoughts?

Sign up for packagecloud.io

Installing via apt-get does not currently work as golang-migrate doesn’t currently have an account on packagecloud.io.

Please move cli directory to "cmd/migrate"

Reason: so "go install" will create binary named "migrate" on $GOPATH/bin not "cli".

Thank you.

Fix broken builds

We're having the following issues with the builds:

• Detecting the MySQL 8 docker image as being up and ready to run when it's ready
  • Could be an issue with binding to 2 sockets or an issue w/ our docker networking config
• Race condition in Cassandra
  • Probably a result of the Cassandra docker image not being ready or gocql driver not being thread-safe

This issue blocks future releases as the releases are dependent on successful builds

Dockerize builds/CI

Goals:

1. Use same binary for all release channels e.g. github release page, packagecloud, and docker
2. Shipped binaries built with exact same dependencies as tests

Approach:

• Use staged docker builds

"Fix" db driver locks

The DB locks are suppose to be managed by the DB to prevent multiple migrations from being run on the same DB at the same time.

Drivers that fake this behavior:

• Cassandra
• Ql
• SQLite
• Stub

We should either:

1. Don't pretend that the DB is providing the lock and remove the fake DB lock implementation
2. Implement locks in the DB like the CockroachDB driver does
   • Note: Because application level locks aren't supported by the DB, lock table creation may face similar issues depending on the DB

Report coverage on more files

The coverage profile filename specified to each test run (via test-with-flags) is re-used. e.g. The filename is not regenerated for each go test call

This may no longer be an issue in Go 1.10

duplicate key value violates unique constraint "pg_type_typname_nsp_index"

We hit an error in our tests similar to #235

pq: duplicate key value violates unique constraint "pg_type_typname_nsp_index" in line 0: CREATE TABLE "schema_migrations" (version bigint not null primary key, dirty boolean not null)

What bothers me is that this is not ErrLocked. So it seems like the lock does not work as expected.
Or do I get it wrong?
We use Postgres and we call it like this

func updateSchema(db *sqlx.DB) error {
	driver, err := migratepg.WithInstance(db.DB, &migratepg.Config{})
	if err != nil {
		return err
	}
	m, err := migrate.NewWithDatabaseInstance("file://storage/postgres/migrations", "postgres", driver)
	if err != nil {
		return err
	}
	err = m.Up()
	if err == migrate.ErrNoChange {
		logrus.Debug("Database schema already up to date")
		err = nil
	}
	return err
}

We use migrate ver 3.2.0
We use pq driver

Question.

Hi, First off thanks for a nice tool.
I have created three func in go that can migrate up, down or tell me version. But I can't really seem go get my head around how to actually start, like init the whole thing. Right now I have a running instance of cockroach and I have created init_up and init_down for all schemas, data etc.
Is there a guide or document that describe the initial steps before first migration?
I hope that you get my question, else please tell me to elaborate. Thanks.

Support templating in migration files

It would be better for us to not have to jump through hoops to get content of migration files templated so we can do things like per-tenant schemas etc... Go has a perfectly usable string templating library available OOTB and it would better for our workflow if the content of migrations could be string templates that use variables defined in the source url so we could host migration files in a GitHub repo and have external variables be replaced in the content before sending it to the driver. Currently files have to be string-replaced using some external tool which limits how we can use the migrate utility.

I'm not a Go guru but if pointed in the right direction I could probably hack a PR together that implements this generically for all sources.

schema_migrations table is assumed to be in the same schema before running migration file and afterwards

For postgres at least, we create the schema_migrations table in whatever current schema we are ( I think we will always be in public schema ), before running migrations.
After we run the migrations we update the table and assume we are in the same schema because we hope the migration file did not change schema, which can happen. The resulting error is:

error: pq: relation "schema_migrations" does not exist in line 0: TRUNCATE "schema_migrations"

No WithInstance function in the Cassandra driver

Just had a need for it since the Open method does not support multiple hosts, and found out the driver doesn't ship one.

Migrate in docker

Hey, what do you think about automate a docker image in this repository ?

`migrate` CLI reports `error: no scheme` despite having supplied connection string.

Hi all,

I wasn't sure whether this should go into https://github.com/mattes/migrate or here, but my best guess after seeing mattes/migrate#311 this issue was to put it here.

I am experiencing an unexpected error from the migrate CLI while following the documentation.

I built and installed the migrate CLI according to the instructions https://github.com/golang-migrate/migrate/tree/master/cli#with-go-toolchain here. Specifically, I ran the following. (I changed the install directory to ~/bin, but I don't expect that to make a difference.)

go get -u -d github.com/mattes/migrate/cli github.com/lib/pq
go build -tags 'postgres' -o ~/bin/migrate github.com/mattes/migrate/cli

I can successfully run migrate -version. Its output is reproduced below.

$ migrate -version
dev

When I try to run the migrate CLI as below. (Certain values have been redacted for privacy, but the general structure has not changed.)

migrate -database "postgresql://redacted:[email protected]:5432/redacted" version

I get the following error message.

error: no scheme

Just to make sure that I didn't make a syntax error somewhere, I ran an example from the doc verbatim, as below. I got the same error message.

migrate -database postgres://localhost:5432/database up 2

Digging briefly into the source code, it appears from

migrate/util.go

Line 55 in 22f2495

if len(u.Scheme) == 0 {

here that error: no scheme is errScheme, which is returned when the parsed URL does not contain a proper scheme. This is unexpected to me. I would expect the scheme to be parsed as postgres.

Does anyone here have any ideas as to why this is happening? I am willing to help debug further, if anyone has recommendations as to what to try. I hope someone can provide some insight here.

Thank you all in advance.

Feature Request: virtual filesystem source

I am interested in using this package for migrations. Currently I am embedding my migrations in my binary using a virtual filesystem that implements the interface defined in the golang.org/x/tools/godoc/vfs package.

It would be nice to have a source similar to the file or go-bindata source for loading packages from a vfs.FileSystem. This would also allow compatibility with a number of tools similar to go-bindata or statik that use the golang.org/x/tools/godoc/vfs/zipfs or golang.org/x/tools/godoc/vfs/mapfs implementations to embed files in the binary.

Support for golang migrations

We've been using migrate for a while and we recently ran into an issue where we needed some fairly complex logic that was only possible running custom migrations in golang. Has there been any consideration for supporting golang migrations? It's a strange case because it doesn't really fit the database model. In theory a migration written in golang could be run against any db backend.

It would be similar to the shell migrations that have been suggested but not implemented. It seems like @JensRantil has put a lot of thought into this idea. Is this something thats still on the radar?

Replace the original repo in go-awesome

As the original repo has been abandoned (mattes/migrate#311), this fork should replace the original repo in go-awesome.

When there are no migrations applied yet, version command returns: "error: no migration"

After for example, I rollback all the way, running the version command returns an error with exit code 1:
error: no migration
Why is this an error ? I wanted to assert the version from inside a bash script and this is confusing.

Also, it seems that many commands ( all ? ) write their result to stderr not stdout.

Running migration scripts without a double confirmation is extremely dangerous

Down migration scripts usually contains operations like drop table, drop index, etc. Which are extremely dangerous. Please consider to add a double confirmation before apply any queries on the database. Thank you.

CLI `down` runs all down migrations by default, making it unnecessarily dangerous

I'm creating this issue to bring up an issue already raised on mattes/migrate: mattes/migrate#289. I think it's appropriate to re-create the issue because that issue received no response and that repo is deprecated.

Copying from there:

The current behavior of down is as follows:

migrate down 1 # run one down migration
migrate down # run *all* down migrations

Since the down migrations typically drop tables or columns, they are destructive. Running all down migrations is often equivalent of dropped the entire DB.

This make it very easy to run a large destructive operation. In my use case, running all down migrations is rarely what I intend.

I would work on a PR to make down require an argument. Does this align with the project goals? Has this been discussed before?

Support for including missed migrations

Hi!

I have forked from this repo to enhance the migrate up command to apply missing migration files which exist on the file system (or migrations location) but don't appear in the schema_migrations table. The need for this has come from having to bump migrations in our codebase which is a pain in the backside for others who may have already ran previous now non existent migrations etc

In order for this to be achieved, it has to store all migration versions. In any call to Version() it would return the last migration applied. So far I have implemented this for mysql and all others drivers currently have noop functions.

Would this be something that could make it back into main repo please?

Thanks

file does not exist error

I'm trying to run the migrations with migrate -path $filepath -database postgres://postgres:password@db:5432/dev?sslmode=disable up where $filepath is a path to my sql files. The contents are 1526653772_init_down.sql and 1526653772_init_up.sql. But it's saying "file does not exist". It's not a file, I know, it's a directory where my sql files are stored, as noted in #9. Is this not the proper argument anymore? if so, what is? what does the path argument describe?

Go build fails in docker with error "no buildable Go source files"

At the point in dockerfile where the command is

RUN go get -v -t  .
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o bin/app

The build fails with error

go build github.com/golang-migrate/migrate/database/postgres: no buildable Go source files in /go/src/github.com/golang-migrate/migrate/database/postgres
The command '/bin/sh -c CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o bin/app' returned a non-zero code: 1

In main.go, I'm using this , like

import (
         "github.com/golang-migrate/migrate"
	 "github.com/golang-migrate/migrate/database/postgres"
)

func main() {
         // conn is a local variable, initalized in `init()`
	driver, err := postgres.WithInstance(conn.DB(), &postgres.Config{})
	m, err := migrate.NewWithDatabaseInstance(
		"file://migrations",
		"postgres",
		driver,
	)

        if err = m.Up(); err != nil && err.Error() != "no change" {
		m.Down()
		panic(err)
	}
}

OrientDB support

Hello! It'd be great to be able to use this tool with OrientDB https://orientdb.com

cli fails with "database driver: unknown driver mysql (forgotten import?)"

Note, we are using the old version, which stays to post issues here.

Built it like this:

$ go get -u -d github.com/mattes/migrate/cli github.com/go-sql-driver/mysql
$ go install -tags 'mysql'  github.com/mattes/migrate/cli

when I run it like this:

$ cli -database "mysql://xxx:yyy@/zzz" -source file://go/src/github.com/xxx/xxx.org/db/schema up

I get:

error: database driver: unknown driver mysql (forgotten import?)

Any ideas?

sqlite3: wrong driver in test

https://github.com/golang-migrate/migrate/blob/master/database/sqlite3/sqlite3_test.go#L52

The sqlite3 test is loading the ql, I believe that should be sqlite3`?

I'm also getting the error on sqlite3 (using the library API).

2018/06/13 20:23:22 source driver: unknown driver file (forgotten import?)

Importing the sqlite3 driver didn't help either.

There is no access to docker image on docker hub

running

docker run -it --rm golang-migrate/migrate --help

returns error

docker: Error response from daemon: pull access denied for golang-migrate/migrate, repository does not exist or may require 'docker login'.

Cassandra: Cannot achieve consistency level ALL

The following error is happening when trying to perform a deployment.
Cannot achieve consistency level ALL in line 0: TRUNCATE "schema_migrations

The cause is that one of the nodes in our cluster is currently down, but truncate requires a consistency level of ALL.

Maybe there is a way to DELETE the record instead of TRUNCATE the table. Since each microservice handles the schema upgrade at startup, if any one of the N nodes is not available it could actually block the deployment.

Thoughts on switching to a delete pattern from the truncate?

Support go migrations

SQL migrations is a good thing, but likely useless in many cases. Migrate must support go migrations, i.e a function that accepts an instance of the database or a transaction object (in case of SQL datastores).
Something along the lines of this to up the version:

func up(tx *sql.Tx) error {
         _, err := tx.Exec("ALTER TABLE calls ADD stats text;")
	return err
}

down also looks the same:

func down(tx *sql.Tx) error {
	_, err := tx.Exec("ALTER TABLE calls DROP COLUMN stats;")
	return err
}

original issue: mattes/migrate#344

Add an example for beginners

It would be a great help if you could include an example so that beginners can better understand the concept of migrations and how to implement stuff using this tool.

Use dep

Hold off on using dep until it supports build tags. We don't want to pull in unnecessary dependencies. e.g. if you're only using migrate for postgres, there's no reason why you should have a dependency on cassandra driver

golang/dep#291

Remove build constraints

Since we're using dep for dependency management, we no longer need build constraints as all dependencies are pulled in.

The larger binary sizes are worth the reduced confusion and ease of use

#59 may be a blocker