flexxnn / sequelize-auto-migrations Goto Github PK
View Code? Open in Web Editor NEWMigration generator && runner for sequelize
License: MIT License
Migration generator && runner for sequelize
License: MIT License
Change 'process.env.PWD' to 'process.env.PWD || process.cwd()' on runmigration.js and makemigration.js to fix problem
Hi guys,
I've been using the script to further automate work with sequelize.
Specifically:
All you need to do is change the model in the code.
You can check the gist here - https://gist.github.com/alexanderjsx/41dd74f0671f6bb7e349b1494a93de5c
Hope this will help some one.
Hello there.
Thanks for the great project first of all. Generation of migrations is really missed feature in sequalize.
Just wondering was you thinking about storing state of migration in database directly like in Django/south/alembic and so on? This will allow applying migrations automatically, and each machine will not require to store file.
Most migration systems do it in this way, so just interesting was you thinking about some profits when decided to store state in filesystem or it was just to save time.
I am a manager of a small development company and we are using this package a lot. I am going to implement storing in db in some feature. What would you recommend, to make a fork or create pull-request?
I have an actor model based around the MySQL sakila demo database. There are no CreatedAt or UpdatedAt fields in either the table or model. Yet when I create an initial migration this tool automatically adds them which causes the server to throw an "Unhandled rejection SequelizeDatabaseError: Unknown column 'createdAt' in 'field list'" error when sync'd.
I know how to get around this error by disabling timestamps in my server configuration, but the behavior is quite annoying. The migration shouldn't be adding columns to models that don't exist either in the model definition nor the original database.
Custom paths defined in .sequelrc
aren't supported since the lib uses hardcoded paths.
Each migration should ideally run in a transaction, to avoid a situation of a migration being partially applied.
We use ARRAY
types and are testing out using sequelize-auto-migrations
. ARRAY
types don't generate correctly.
Minimal repro below:
'use strict';
const Sequelize = require('sequelize');
module.exports = function (sequelize) {
const Test = sequelize.define('test', {
array: {
type: Sequelize.ARRAY(Sequelize.TEXT),
allowNull: false,
defaultValue: []
}
}, {
});
return Test;
};
Expected: Syntactically valid migration generated
'use strict';
var Sequelize = require('sequelize');
/**
* Actions summary:
*
* createTable "tests", deps: []
*
**/
var info = {
"revision": 2,
"name": "testSchema",
"created": "2017-08-02T16:42:10.042Z",
"comment": ""
};
var migrationCommands = [{
fn: "createTable",
params: [
"tests",
{
"id": {
"type": Sequelize.INTEGER,
"autoIncrement": true,
"primaryKey": true,
"allowNull": false
},
"array": {
"type": Sequelize.ARRAY(Sequelize.TEXT), // This line is wrong in the generated migration
"defaultValue": [],
"allowNull": false
},
"createdAt": {
"type": Sequelize.DATE,
"allowNull": false
},
"updatedAt": {
"type": Sequelize.DATE,
"allowNull": false
}
},
{}
]
}
];
module.exports = {
pos: 0,
up: function(queryInterface, Sequelize)
{
var index = this.pos;
return new Promise(function(resolve, reject) {
function next() {
if (index < migrationCommands.length)
{
let command = migrationCommands[index];
console.log("[#"+index+"] execute: " + command.fn);
index++;
queryInterface[command.fn].apply(queryInterface, command.params).then(next, reject);
}
else
resolve();
}
next();
});
},
info: info
};
Actual:
'use strict';
var Sequelize = require('sequelize');
/**
* Actions summary:
*
* createTable "tests", deps: []
*
**/
var info = {
"revision": 2,
"name": "testSchema",
"created": "2017-08-02T16:42:10.042Z",
"comment": ""
};
var migrationCommands = [{
fn: "createTable",
params: [
"tests",
{
"id": {
"type": Sequelize.INTEGER,
"autoIncrement": true,
"primaryKey": true,
"allowNull": false
},
"array": {
"type": TEXT[], // This syntax is incorrect
"defaultValue": [],
"allowNull": false
},
"createdAt": {
"type": Sequelize.DATE,
"allowNull": false
},
"updatedAt": {
"type": Sequelize.DATE,
"allowNull": false
}
},
{}
]
}
];
module.exports = {
pos: 0,
up: function(queryInterface, Sequelize)
{
var index = this.pos;
return new Promise(function(resolve, reject) {
function next() {
if (index < migrationCommands.length)
{
let command = migrationCommands[index];
console.log("[#"+index+"] execute: " + command.fn);
index++;
queryInterface[command.fn].apply(queryInterface, command.params).then(next, reject);
}
else
resolve();
}
next();
});
},
info: info
};
Thank you so much for this package. It's been a big help so far.
It imposes uniqueness on the foreign keys of the relation table, which is unreasonable. It should at most be a joint uniqueness.
I have a model that blows up a diff migration because of a model with 2 indexes that haven't changed. It gets picked up as a diff but in the parsedifferences routine the rhs value is null so line 367 blows up. It seems I can only solve this by adding this check.
let index = _.clone(df.rhs);
// sometimes a diff is picked up with no RHS?
if (index)
{
index.actionType = 'addIndex';
index.tableName = tableName;
index.depends = [ tableName ];
actions.push(index);
}
This change makes the diff migration carry on and it seems to work fine. I've found that if I actually make a difference it also "works", but only one time. For example, I thought I fixed this by adding names to all my indexes.
My sequelize model has these indexes
indexes: [
{
name: 'account_uuid_idx',
unique: true,
fields: ['uuid']
},
{
name: 'account_username_btree_idx',
method: 'BTREE',
fields: ['username']
}
]
}```
Linux, node v10.15.3 , npm 6.8.0, "sequelize-auto-migrations": "^1.0.3"
Thanks for your eyeballs!
If I add explicitly the primary index I get an error of wrong name.
Example:
{ fn: "addIndex", params: [ "admins", [{ "name": "id" }], { "indexName": "PRIMARY", "name": "PRIMARY", "indicesType": "UNIQUE", "type": "UNIQUE" } ] },
Error:
Executing (default): ALTER TABLE
adminsADD UNIQUE INDEX
PRIMARY (
id) DatabaseError [SequelizeDatabaseError]: Incorrect index name 'PRIMARY' at Query.formatError (/node_modules/sequelize/lib/dialects/mysql/query.js:239:16) at Query.run (/node_modules/sequelize/lib/dialects/mysql/query.js:54:18) at processTicksAndRejections (internal/process/task_queues.js:97:5) { parent: Error: Incorrect index name 'PRIMARY' at Packet.asError (/node_modules/mysql2/lib/packets/packet.js:712:17) at Query.execute (/node_modules/mysql2/lib/commands/command.js:28:26) at Connection.handlePacket (/node_modules/mysql2/lib/connection.js:425:32) at PacketParser.onPacket (/node_modules/mysql2/lib/connection.js:75:12) at PacketParser.executeStart (/node_modules/mysql2/lib/packet_parser.js:75:16) at Socket.<anonymous> (/node_modules/mysql2/lib/connection.js:82:25) at Socket.emit (events.js:314:20) at addChunk (_stream_readable.js:298:12) at readableAddChunk (_stream_readable.js:273:9) at Socket.Readable.push (_stream_readable.js:214:10) at TCP.onStreamRead (internal/stream_base_commons.js:188:23) { code: 'ER_WRONG_NAME_FOR_INDEX', errno: 1280, sqlState: '42000', sqlMessage: "Incorrect index name 'PRIMARY'", sql: 'ALTER TABLE
adminsADD UNIQUE INDEX
PRIMARY (
id)', parameters: undefined }, original: Error: Incorrect index name 'PRIMARY' at Packet.asError (/node_modules/mysql2/lib/packets/packet.js:712:17) at Query.execute (/node_modules/mysql2/lib/commands/command.js:28:26) at Connection.handlePacket (/node_modules/mysql2/lib/connection.js:425:32) at PacketParser.onPacket (/node_modules/mysql2/lib/connection.js:75:12) at PacketParser.executeStart (/node_modules/mysql2/lib/packet_parser.js:75:16) at Socket.<anonymous> (/node_modules/mysql2/lib/connection.js:82:25) at Socket.emit (events.js:314:20) at addChunk (_stream_readable.js:298:12) at readableAddChunk (_stream_readable.js:273:9) at Socket.Readable.push (_stream_readable.js:214:10) at TCP.onStreamRead (internal/stream_base_commons.js:188:23) { code: 'ER_WRONG_NAME_FOR_INDEX', errno: 1280, sqlState: '42000', sqlMessage: "Incorrect index name 'PRIMARY'", sql: 'ALTER TABLE
adminsADD UNIQUE INDEX
PRIMARY (
id)', parameters: undefined }, sql: 'ALTER TABLE
adminsADD UNIQUE INDEX
PRIMARY (
id)', parameters: undefined }
When I clean all 'PRIMARY' records on migration, I don't get that error.
And I think it should be repaired because I'm using this tool with sequelize-auto to generate the models, and it generates this index explicitly. So both wouldn't be compatible.
I'm using this version: [email protected] (git+https://github.com/flexxnn/sequelize-auto-migrations.git#0c3b385fffa474b2db825a2b48ce0873a8b9286b)
I prefer to use https://github.com/RobinBuschmann/sequelize-typescript for defining my models. Do your module work with it?
I have a model generated by Sequelize with the following information:
module.exports = (sequelize, DataTypes) => {
const user = sequelize.define('test', {
firstName: DataTypes.STRING,
lastName: DataTypes.STRING,
email: DataTypes.STRING,
testField: DataTypes.STRING,
anotherTest: DataTypes.STRING
}, {});
user.associate = function(models) {
// associations can be defined here
};
return user;
};
Running makemigration
generates the following:
'use strict';
var Sequelize = require('sequelize');
/**
* Actions summary:
*
* createTable "users", deps: []
*
**/
var info = {
"revision": 1,
"name": "user",
"created": "2019-10-20T04:47:04.496Z",
"comment": ""
};
var migrationCommands = [{
fn: "createTable",
params: [
"users",
{
},
{}
]
}];
module.exports = {
pos: 0,
up: function(queryInterface, Sequelize)
{
var index = this.pos;
return new Promise(function(resolve, reject) {
function next() {
if (index < migrationCommands.length)
{
let command = migrationCommands[index];
console.log("[#"+index+"] execute: " + command.fn);
index++;
queryInterface[command.fn].apply(queryInterface, command.params).then(next, reject);
}
else
resolve();
}
next();
});
},
info: info
};
Doesn't look like there is any code related to the fields. Running runmigration
confirms this since it only creates the table with no fields in it.
Is it something I'm just doing wrong?
DECIMAL params of model's field are ignored when making a migration.
Model:
module.exports = (sequelize, DataTypes) => {
const purchaseProducts = sequelize.define('purchaseProducts', {
id: {
type: DataTypes.INTEGER.UNSIGNED,
allowNull: false,
autoIncrement: true,
primaryKey: true,
},
price: {
type: DataTypes.DECIMAL(6, 2),
allowNull: false,
},
});
return purchaseProducts;
};
Generated migration:
...
"purchaseProducts",
{
"id": {
"type": Sequelize.INTEGER.UNSIGNED,
"primaryKey": true,
"autoIncrement": true,
"allowNull": false
},
"price": {
"type": Sequelize.DECIMAL,
"allowNull": false
}
},
{}
...
At the moment I am writing the (6, 2)
params to the migration by hand after generating.
The package in NPM registry needs to be updated to reflect the current state in Github.
I have downloaded from NPM version 1.0.2 but the makemigration.js & runmigration.js are missing the Option parameters:
{ name: 'migrations-path', type: String, description: 'The path to the migrations folder' },
{ name: 'models-path', type: String, description: 'The path to the models folder' },
Appreciate your help.
Thanks.
I read a tutorial on how to use, installing:
npm install --save github:scimonster/sequelize-auto-migrations#a063aa6535a3f580623581bf866cef2d609531ba
Edit package.json
"scripts": {
...
"db:makemigrations": "./node_modules/sequelize-auto-migrations/bin/makemigration.js",
}
After:
npm run db:makemigrations
But when running, nothing happens, just open the file 'makemigration.js'.
I have a .sequelizerc file:
const { resolve } = require('path');
module.exports = {
config: resolve(__dirname, 'src', 'config', 'database.js'),
'models-path': resolve(__dirname, 'src', 'app', 'models'),
'migrations-path': resolve(__dirname, 'src', 'database', 'migrations'),
'seeders-path': resolve(__dirname, 'src', 'database', 'seeds'),
};
Could that be it? if anyone can help me, i will be very grateful.
With Sequelize 5.x installed the makemigration tool only seems to pick up on the model names but none of the property information.
Trying to create the first migration
node_modules/.bin/makemigration --name init
resulting
No changes found
although there are 3 new models in the models models-path
There are PRs open from 2017....
Is there an actively maintained fork?
Current behavior: generates a JSONTYPE
field which causes Unhandled rejection TypeError: Cannot read property 'toString' of undefined
PR inbound.
Hello.
If model of previous state have no indexes and i will add index in this model then i will still get message "No changes found".
I found bug. Could you fix this code section:
if (models[model].options.indexes.length > 0)
{
let idx_out = {};
for (let _i in models[model].options.indexes)
{
let index = parseIndex(models[model].options.indexes[_i]);
idx_out[index.hash+''] = index;
delete index.hash;
// make it immutable
Object.freeze(index);
}
models[model].options.indexes = idx_out;
} else {
models[model].options.indexes = {}; // Important change, because models[model].options.indexes is [] but it must be {}
}
tables[models[model].tableName].indexes = models[model].options.indexes;
Defining a type of 'citext' (used for Postgresql and SQLite), generates 'undefined'.
I have a many-many relationship table that uses the foreign keys as a composite primary key. Whenever I run makemigration, a changeColumn command is generated. It's not breaking the migrations, but it does result in unnecessary commands being executed.
My model is under a different schema than 'public' but that metadata isn't being saved in the migration so running the migration fails.
Executing (default): ALTER TABLE "public"."table" DROP COLUMN "column"
"type": Sequelize.TEXT(long) is generated. Should be: "type": Sequelize.TEXT('long')
Release new version. A lot of merges are not part of the current version of NPM. viz, 1.0.3
Last Commit: 6cd3c1f
Hello all (to anyone still actively using sequelize-auto-migrations
). To all who have contributed PRs over the years - THANK YOU! Your PR is not in vain. Hence, we've created a fork that we're actively maintaining.
It's available here:
https://github.com/matmar10/sequelize-lazy-migrations
The new npm name is sequelize-lazy-migrations
and you can install it via that name:
npm install --save-dev sequelize-lazy-migrations
My company is still actively utilizing and relying on this library. Requests to add contributors have gone unanswered. Hence this was the best course of action.
Why doesn't this create regular migrations?
It seems like it creates it's own migration format and requires it's own special command to execute. Why not just create migrations using the sequelize migration format?
I am using sequalize and I'm trying to run auto migrations with the following library: https://github.com/flexxnn/sequelize-auto-migrations with NODE_ENV=development npx sequelize-auto-migrations --name initial
but i get the following error:
Class constructor Authentication cannot be invoked without 'new'
const Sequelize = require('sequelize');
const sequelize = require('../../config/database');
const { serializeModel } = require('../utils/serializeRelationships');
const { Op } = require('sequelize');
const hooks = {};
const tableName = 'authentication';
const defaultScope = {};
class Authentication extends Sequelize.Model {}
Authentication.init({
id: {
type: Sequelize.UUID,
primaryKey: true,
},
dType: {
type: Sequelize.STRING,
field: 'dtype',
},
serviceId: {
type: Sequelize.STRING,
field: 'service_id',
},
session: {
type: Sequelize.STRING,
},
createdAt: {
type: Sequelize.DATE,
field: 'created_at',
},
updatedAt: {
type: Sequelize.DATE,
field: 'updated_at',
},
}, {
sequelize, defaultScope, scopes: {}, hooks, tableName, timestamps: true,
});
Authentication.prototype.toJSON = function toJSON() {
return serializeModel(this);
};
module.exports = Authentication;
node: v12.16.3
Im not using babel or anything else.
I tried to add a new column which is a BIGINT. The generated migration file didnt include the "Sequelize."BIGINT in the type, only "BIGINT".
Adding "Sequelize." fixed it, so it probably is a easy-to-solve issue.
tl;dr: Covert BIGINT to Sequelize.BIGINT in migration file.
I'm using Postgresql
Keep the nice work :) Thanks
I'm using POINT columns, and while this package apparently isn't very tested on those, it's worked great for me so far. However I encountered a problem while trying to create SPATIAL indexes, since the SPATIAL type was being ignored, and they were being created as normal indexes.
A quick patch fixed it: NiciusB@6c26e8d
But since the solution is hacky, and there's probably a cleaner way to handle indicesType, I won't create a pull request. I'll be using my github repo instead of this npm package until a solution is shipped, so I'd be great if this was eventually fixed.
Thanks!
Hello there,
I don't see any parameters can choose which environment to migrate, and in example only development present in config file.
Does anyone know how to migrate to production?
thanks
I got
var migrationCommands = [{
fn: "createTable",
params: [
"account",
{
},
{}
]
},
with account models
/* eslint-disable key-spacing, no-multi-spaces */
module.exports = function (DB, { INTEGER, BIGINT, DATE, STRING, ENUM, BOOLEAN, DATEONLY, NOW }) {
const Model = DB.define('account',
{
id: { type: INTEGER, allowNull: false, primaryKey: true, autoIncrement: true },
test_param: { type: BIGINT, allowNull: false, defaultValue: 1000 },
first_name: { type: STRING, allowNull: true, defaultValue: 'abc', field: 'first-name' },
last_name: { type: STRING, allowNull: false, defaultValue: '' },
nickname: { type: STRING, allowNull: false, defaultValue: '' },
gender: { type: ENUM, allowNull: false, values: ['male', 'female', 'unknown'], defaultValue: 'unknown' },
birth_date: { type: DATEONLY, allowNull: true },
last_login_dt: { type: DATE, allowNull: true },
created_at: { type: DATE, allowNull: true, defaultValue: NOW },
email: { type: STRING, allowNull: false, unique: true },
password: { type: STRING, allowNull: false },
is_deleted: { type: BOOLEAN, allowNull: false, defaultValue: false },
is_blocked: { type: BOOLEAN, allowNull: false, defaultValue: false },
// city_id -> city.id
},
{
timestamps: false,
underscored: true,
tableName: 'account'
}
)
Model.associate = (models) => {
Model.belongsTo(models.city)
// Model.hasMany(models.team, { foreignKey: { allowNull: false } })
}
return Model
}
occurs with "sequelize": "5.16.0" version
✘ ~/projects/rgs-decoupled buffer ● node ./node_modules/sequelize-auto-migrations/bin/makemigration --name create_db
module.js:491
throw err;
^
Error: Cannot find module '/home/bjorn/projects/rgs-decoupled/models'
at Function.Module._resolveFilename (module.js:489:15)
at Function.Module._load (module.js:439:25)
at Module.require (module.js:517:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/home/bjorn/projects/rgs-decoupled/node_modules/sequelize-auto-migrations/bin/makemigration.js:53:17)
at Module._compile (module.js:573:30)
at Object.Module._extensions..js (module.js:584:10)
at Module.load (module.js:507:32)
at tryModuleLoad (module.js:470:12)
at Function.Module._load (module.js:462:3)
at Function.Module.runMain (module.js:609:10)
at startup (bootstrap_node.js:158:16)
at bootstrap_node.js:598:3
{
code: 'ER_PARSE_ERROR',
errno: 1064,
sqlState: '42000',
sqlMessage: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '"SequelizeMeta" (name varchar UNIQUE)' at line 1
,
sql: 'CREATE TABLE IF NOT EXISTS "SequelizeMeta" (name varchar UNIQUE)',
parameters: undefined
},
sql: 'CREATE TABLE IF NOT EXISTS "SequelizeMeta" (name varchar UNIQUE)',
parameters: {}
}
I have issues where makemigrations
generated files without columns but I want to test if it occurs on example as well, how do I test it?
The following might help people trying to get through migrations, I hope it is documented in README.
$ npm ci
$ npm add -D sequelize-cli sequelize-auto-migrations
$ mkdir migrations
$ node_modules/.bin/makemigration --name init --preview
I have a ts project, which is structured like this:
root/ts
root/js
root/node_modules
I ran sequelize init inside ts because I want to have my models in there. How can I now tell auto-migrations to look in ts for the models folder?
For example, for two double fields, I got:
"contribution": {
"type": Sequelize.DOUBLE PRECISION
},
"reputation": {
"type": Sequelize.DOUBLE PRECISION
},
instead of:
"contribution": {
"type": Sequelize.DOUBLE
},
"reputation": {
"type": Sequelize.DOUBLE
},
I get this error running node ./node_modules/sequelize-auto-migrations/bin/makemigration --name 'init'
command. It asks for some path. It's not motioned in the README, and not in .../makemigration --help
. What is it? How do I create initial migration?
When you run sequelize db:migrate, it would reject with
File: _current.json does not match pattern: /\.js$/
File: meta does not match pattern: /\.js$/
Placing the file in another directory, even a subdirectory of the migrations
folder would solve this problem.
Hy there,
I use sequelize in an existing project which forces me to put everything that is not configuration in a src folder.
- package.json
+ config
-- config_sequelize.json
-- sequelize_init.json
+ src
++ models
++ migrations
++ models
++ routes
++ seeders
...
(+ for folders, - for files)
For sequelize I define this in the package.json
"sequelize": "sequelize --options-path=config/sequelize_init.js",
sequelize_init.json:
const path = require('path');
module.exports = {
config: path.join(__dirname, '/config_sequelize.json'),
'migrations-path': path.join(__dirname, '../src/migrations'),
'seeders-path': path.join(__dirname, '../src/seeders'),
'models-path': path.join(__dirname, '../src/models')
};
Is this possible with sequelize-auto-migrations?
Passing the sequelize_init.js failed.
Would be great as a feature :)
Greetings from Munich
Sequelize natively supports undoing migrations.
Is this something in progress or is there a plan to add this soonish?
When there is a validate object defined, even though nothing has changed for any column, any table. A change will be detected.
Put in a similar PR to fix this bug in this repo - manuelvillar/sequelize-auto-migrations-ng#18
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.