edwardanthony / nestjs-seeder Goto Github PK
View Code? Open in Web Editor NEWAn extension library for NestJS to perform seeding.
License: MIT License
An extension library for NestJS to perform seeding.
License: MIT License
`import { Prop, Schema, SchemaFactory } from '@nestjs/mongoose';
import { HydratedDocument } from 'mongoose';
import { Factory } from 'nestjs-seeder';
export type DriverDocument = HydratedDocument;
@Schema()
export class Driver {
@factory((faker) => faker.number.int())
@prop()
userId: string;
@prop()
@factory((faker) => faker.number.int())
serviceId: string;
@prop()
@factory((faker) => faker.datatype.boolean())
status: boolean;
@factory((faker) => ({
type: 'Point',
coordinates: [faker.location.longitude, faker.location.latitude],
}))
@prop({
type: {
type: String,
default: 'Point',
},
coordinates: [Number],
})
currentLocation: {
type: string;
coordinates: number[];
};
}
export const DriverSchema = SchemaFactory.createForClass(Driver);`
Here currenLocation is not populated while seeding, other feilds are created as expected. What am i missing here?
Hi,
in some cases would be useful to share some values down the chain when generating seed data. An example could be:
@Seedable(faker => { gender: faker.random.number(1) })
export class UserEntity {
@Factory((_, ctx) => ctx.gender)
public gender: number;
@Factory((faker, ctx) => faker.name.firstName(ctx.gender))
public firstName: string;
@Factory((faker, ctx) => faker.name.lastName(ctx.gender))
public lastName: string;
}
The same concept could be applied when using the factory in the seeder directly.
async seed(): Promise<any> {
// Generate 10 female users.
const users = DataFactory.createForClass(User).generate(10, { gender: 1 });
// [...]
}
I am using typeorm(mysql configs) and my user.entity.ts looks like this:
@Entity('users')
export class User {
@Field(type => ID)
@PrimaryGeneratedColumn()
readonly id: number;
@Field()
@Column()
@Factory(faker => faker.name.findName())
firstName: string;
@Field()
@Column()
@Factory(faker => faker.name.findName())
lastName: string;
@Field()
@Index({ unique: true })
@Column()
@Factory(faker => faker.internet.email())
email: string;
@Exclude({ toPlainOnly: true })
@Column()
@Factory(faker => faker.internet.password())
password: string;
but @factory() doesn't seem to work because I get this error when i run yarn seed
UnhandledPromiseRejectionWarning: QueryFailedError: ER_NO_DEFAULT_FOR_FIELD: Field 'first_name' doesn't have a default value
maybe the solution is to export the factory schema like you specified in your example here
export const userSchema = SchemaFactory.createForClass(User);
and that's for mongodb only and I can't find anything like it in typeorm
Great project!
but i got
Type 'Record<string, FactoryValue>' is missing the following properties from type 'User': gender, name, age, role, and 50 more.ts(2769)
in sample project.
users.seeder.ts line 18.
any ideas how to solve this?
Problem:
const data = DataFactory.createForClass(Message).generate(1, {
body: 'Test this overwrites'
});
// Output { "body": "Exercitationem rerum ab quae libero nobis debitis deleniti iusto aut." }
// Expected { "body": "Test this overwrites" }
// message.schema.ts
import { Prop, Schema, SchemaFactory } from '@nestjs/mongoose';
import { Date, Document, Types } from 'mongoose';
import { Factory } from 'nestjs-seeder';
import { Chat } from 'src/chats/chat.schema';
import { User } from 'src/users/user.schema';
@Schema()
export class MessageHistory {
@Prop({ default: '' })
message: string;
@Prop({ type: Date, default: Date.now })
timestamp: Date;
}
// Generate a Mongoose Schema before use as Subdocument
const MessageHistorySchema = SchemaFactory.createForClass(MessageHistory);
@Schema({
timestamps: true,
toJSON: {
virtuals: true
}
})
export class Message {
@Prop({ type: String, ref: 'User' })
user: User;
@Prop({ type: Types.ObjectId, ref: 'Chat', index: true })
chat: Chat;
@Factory((faker) => faker.lorem.sentence())
@Prop({ default: '' })
body: string;
@Prop({ type: Date, default: undefined })
deletedAt: Date;
@Prop({ type: [MessageHistorySchema] })
editHistory: MessageHistory[];
}
export type MessageDocument = Message & Document;
export const MessageSchema = SchemaFactory.createForClass(Message);
MessageSchema.virtual('edited').get(function (this: MessageDocument) {
return this.editHistory.length > 0;
});
it is not totally clear to me how to use this with sequelize.
also: i have a request-scoped database connection, how could i adapt this to the seeder ?
and also: is it possible to check that the seeder only runs on dev environment ?
when executing the seeder with the --refresh
flag, my drop function is async as it needs to use the await
keyword to delete items in the DB, however by the look of it the seed method doesn't wait for the drop method to finish its execution as i see errors from mongoDB complaining about duplicate keys MongoError: E11000 duplicate key error collection
.
Basically the old record was not dropped before the new write. and i can confirm that the record is properly deleted from the database which means that the drop method is being executed properly in the event loop just not being waited for by the seed method.
this also brings me back issue #18 that i raised some time ago and that is to have a command to only execute the drop function.
As we can see in the package.json
, the faker dependency is not updated to latest (5.1.0
at time of writing).
https://github.com/edwardanthony/nestjs-seeder/blob/master/package.json#L31
Can we bump this to 5.1.0
?
I have a specific use case where I need static ids, but random data. Is there a way to run DataFactory.generate, but to pass the current iteration number to the context?
For example
// school.seeder.ts
public async seed() {
const schools = DataFactory.createForClass(School).generate(10)
}
// school.entity.ts
@Factory((_, ctx) => ctx.idx) // GET CURRENT ITERATION FOR THIS CONTEXT
@PrimaryGeneratedColumn()
schoolId: number
I guess I could just replace them before saving them, but I was hoping there was a way of injecting this data into the current iteration context.
Or is there maybe another way I'm not contemplating? Would this be helpful as a PR?
Let me know what you think!
As faker dependency was removed a few months ago due to malicious code incident:https://www.infoworld.com/article/3647750/faker-npm-package-back-on-track-after-malicious-coding-incident.html, this package is now presenting some issues:
In factory.decorator.d.ts:2:23
Cannot find type definition file for 'faker
2 /// <reference types="faker" />
Also factory.decorator.d.ts:5:54
Cannot find namespace 'Faker'
5 export type FactoryValueGenerator = (faker?: Faker.FakerStatic, ctx?: Record<string, any>) => FactoryValue;
I managed to partially resolve it by editing this file and just removing FactoryValueGenerator
, however i don't think this is the best approach
drop and seed is running parallelly. Some times drop is executed later
seed logs below
[Nest] 71391 - 11/12/2021, 10:15:36 AM VERBOSE [DatabaseLogger] QUERY: INSERT INTO "user_entity"("id", "username", "email", "realName", "password", "createdAt", "updatedAt", "deletedAt") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, DEFAULT), (DEFAULT, $7, $8, $9, $10, $11, $12, DEFAULT), (DEFAULT, $13, $14, $15, $16, $17, $18, DEFAULT), (DEFAULT, $19, $20, $21, $22, $23, $24, DEFAULT), (DEFAULT, $25, $26, $27, $28, $29, $30, DEFAULT), (DEFAULT, $31, $32, $33, $34, $35, $36, DEFAULT), (DEFAULT, $37, $38, $39, $40, $41, $42, DEFAULT), (DEFAULT, $43, $44, $45, $46, $47, $48, DEFAULT), (DEFAULT, $49, $50, $51, $52, $53, $54, DEFAULT), (DEFAULT, $55, $56, $57, $58, $59, $60, DEFAULT) RETURNING "id", "createdAt", "updatedAt", "deletedAt" - PARAMETERS: ["Zachery33","[email protected]","Layla O'Connell","ZpoQ4b3kQtmp2hb","2019-09-20T09:29:08.866Z","2021-08-21T15:53:42.908Z","Ara89","[email protected]","Amira Yost","eXR6jDQyUJFtStZ","2021-01-28T12:01:35.545Z","2021-10-07T20:18:28.332Z","Laney_Morar20","[email protected]","Elisabeth McClure","jt8QyO9wlmvwZfd","2020-03-14T11:22:18.946Z","2021-01-03T06:58:04.940Z","Domenico_Nienow","[email protected]","Abelardo Renner","kE4LFej0bVqEwXx","2019-03-11T03:11:45.895Z","2020-08-14T01:27:47.289Z","Samara_Prohaska","[email protected]","Cristal Sanford","nFPXRmVE8lHBEEK","2021-02-12T07:25:06.748Z","2020-08-17T10:10:54.228Z","Lori41","[email protected]","Eddie Klein","AptXp33EAPCelf5","2019-06-22T14:44:49.606Z","2020-06-05T05:27:15.399Z","Breanna51","[email protected]","Hettie Crist","cuVqgbavuW5grDI","2019-05-10T22:14:47.607Z","2020-10-20T20:06:17.335Z","Erica59","[email protected]","Winona Bailey","aCOxRFe1KhfwKJx","2020-12-09T07:44:21.090Z","2020-10-27T06:17:15.609Z","Keith_Haley","[email protected]","Madilyn Upton","BlOQWWqsI4Ss9T3","2021-02-03T00:43:54.026Z","2020-12-30T10:59:20.107Z","Tevin_Nikolaus","[email protected]","Alanna Stanton","EyorM4Cza06clFv","2019-10-30T18:23:47.188Z","2020-10-02T19:22:52.241Z"]
UserSeeder completed
[Nest] 71391 - 11/12/2021, 10:15:36 AM VERBOSE [DatabaseLogger] QUERY: DELETE FROM "user_entity" - PARAMETERS: []
We should use
async run(): Promise<any> {
const promises = this.shouldRefresh()
? [this.drop().then(() => this.seed())]
: [this.seed()];
return Promise.all(promises);
}
Hi , i got some trouble,how can i deal with them
node_modules/nestjs-seeder/dist/decorators/factory.decorator.d.ts:2:23 - error TS2688: Cannot find type definition file for 'faker'.
2 ///
~~~~~
node_modules/nestjs-seeder/dist/decorators/factory.decorator.d.ts:5:54 - error TS2503: Cannot find namespace 'Faker'.
5 export declare type FactoryValueGenerator = (faker?: Faker.FakerStatic, ctx?: Record<string, any>) => FactoryValue;
~~~~~
There are no instructions in README informing that the peer dependency @faker-js/faker should be installed on its own.
Hello😃
it might not belong here (sorry if it does not😌 ), i posted on stackoverflow as well, But i thought someone here would be mor likely to respond to my issue.
I didn't see anything on the current documentation or in the web on how to seed id's from associative entities. Is that possible to do so ?
Here is the duplicate of my issue with additional explanations.
https://stackoverflow.com/questions/66946423/seed-associative-entities-using-nestjs-typeorm-nestjs-seeder
Could this package be updated to allow @faker-js/faker version ^8.0.0?
Webpack's bundle block using, can't use ts-node or typescript to build.
$ node src/seeder.js
internal/modules/cjs/loader.js:969
throw err;
^
Error: Cannot find module './seeds/account.seeder'
Require stack:
- /Volumes/dev/organic/src/seeder.js
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:966:15)
at Function.Module._load (internal/modules/cjs/loader.js:842:27)
at Module.require (internal/modules/cjs/loader.js:1026:19)
at require (internal/modules/cjs/helpers.js:72:18)
at Object.<anonymous> (/Volumes/dev/organic/src/seeder.js:4:24)
at Module._compile (internal/modules/cjs/loader.js:1138:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1158:10)
at Module.load (internal/modules/cjs/loader.js:986:32)
at Function.Module._load (internal/modules/cjs/loader.js:879:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12) {
code: 'MODULE_NOT_FOUND',
requireStack: [ '/Volumes/dev/organic/src/seeder.js' ]
}
i want to use seeding with typeOrm but stay db error how can i solve this
I implemented the example from the README.md and get this error:
node_modules/nestjs-seeder/dist/decorators/factory.decorator.d.ts:2:23 - error TS2688: Cannot find type definition file for 'faker'.
2 /// <reference types="faker" />
node_modules/nestjs-seeder/dist/decorators/factory.decorator.d.ts:5:54 - error TS2503: Cannot find namespace 'Faker'.
5 export declare type FactoryValueGenerator = (faker?: Faker.FakerStatic, ctx?: Record<string, any>) => FactoryValue;
Hi,
Wanted to say this a great package! You did excellent work here!
The "faker": "^4.1.0"
should be in the peer dependencies because you need to let the consumer determine the version of Faker.
Want me to create a PR for it?
First of all thank you for this very helpful seeder that integrates very well with nestjs
Wanted to request a feature for being able to drop seeded data basically calling the drop method without any seeding.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.