keithwhor/nodal

API Services Made Easy With Node.js

nodal's Issues

Add support for seeding a database

I am opening this ticket as a place for discussion around possible ways to implement command line seed support (ala rails db:seed) This need is coming up in a few tickets #58 #48.

So to start, in my situation I was porting an older small app of mine that was backed by mongo. I have a command line node tool to pump in seed data when I stood up new instances. During my port to nodal, which I was doing as a public example of a nodal app, I really wanted a single db cli command (see #50) to prepare, migrate and seed the sample data so that someone could fork the repo and use the built in deploy to heroku button and have a running app with real data

So my first thought is do we have a large json doc that can be used to seed models (like schema.json) or do we actually generate a real module that can be required and run to allow for model validations, model relations etc (ala rails)

so for a quick and dirty comparison of the thoughts (excuse errors, thinking as a I go here)

a seed.json (which has to have proper ordering)

{
  "user": {
    "id": 1,
    "username": "mark",
    "email": "[email protected]"
  },
  "tweet": {
    "id": 1,
    "body": "Hey There",
    "user_id": 1
  }
}

or in js (remember, just a rough idea)

  'use strict';

  const Nodal = require('nodal');
  const User = Nodal.require('app/models/user.js');
  const Tweet = Nodal.require('app/models/user.js');

  let markUser;
  User.create({ username: "mark", email: "[email protected]"}, (err, model) => {
    markUser = model;
  });

  Tweet.create({ body: "Hey There", user_id: markUser.id}, (err, model) => {

  });

or even

  'use strict';

  const Nodal = require('nodal');
  const User = Nodal.require('app/models/user.js');
  const Tweet = Nodal.require('app/models/user.js');

  User.create({ username: "mark", email: "[email protected]"}, (err, model) => {

      Tweet.create({ body: "Hey There", user_id: model.id}, (err, model) => {
      });

  });

Okay seed (no pun intended) laid, thoughts?

Middleware is only 'exec' when calling render()

I was planning on adding Strong Param support and couldn't decide between implementing it in the router or controller superclass so I thought I could just make a middleware out of it.
The benefit is that it could be disabled to keep backward compatibility for older apps.

However from the looks of it, the middleware are only executed when render() is called.

My take is, if it's moved to router delegation, one could implement more of nodal features as middleware (that can be disabled) instead of hardcoding them into the system. Honeypot, ip-redirect, etc.

Are controllers really views like in django, but not real controllers like in mvc?

I saw that each controller have several methods that are realted to name of the http method, but not actions.
So... To clarify, are they really controllers or much like views from django (python)? Because if they are like views, maybe there is possibility to rename them?

Adding Rails like Strong Parameter support

I could finish an implementation of Strong Parameter support as a middleware, but not sure if that would be accepted?

Using a Resource "DSL" for creating routes.

Would a Rails like syntax for creating routes fit in with the design of the framework?
I'm using array splats to take more than one argument. Would have to check the order.
Still learning ES6, so please let me know if I'm missing something here.

const UserController = Nodal.require('app/controllers/user_controller.js');
const PhotoController = Nodal.require('app/controllers/photo_controller.js');

//All verbs
router.resources(UserController) // /user/<verbs>
router.resources(UserController, "/avatars") // /avatars/<verbs>

//Routes with just one data-set
//Trying to avoid passing magic boolean arguments, so using a different method name
router.singularResource(UserController) // /user/<verbs sans index no id>
router.singularResource(UserController, "/avatars") // /avatars/<verbs sans index, no id>

//Single routes
router.get(UserController)  // /users/:id - calls get()/show()
router.get(UserController, "/avatars")  // /avatars/:id - calls get()/show()
router.get(UserController, "/api/v1/avatars")  // /api/v1/avatars/:id - calls get()/show()

//Scoped routes
router.resources(UserController, {scope: "/api/v1"}); // /user/<verbs>
router.get(UserController, "avatars", {scope: "/api/v1"});  // /api/v1/avatars/:id - calls get()/show()

router.scope("/admin", scopedRoute => 
   scopedRoute.resources(UserController); // /admin/user/<verbs>
   scopedRoute.get(PhotoController, "/picture");  // /admin/pictures/:id - calls get()/show()
);

//Nested Routes
router.resources(UserController, nestedRoute => 
   nestedRoute.resources(PhotoController);
); // /user/<verbs>/photos/<verbs>

Btw, I genuinely love the source code. This is the cleanest javascript/MVC-framework source code I've ever seen. I've seen plenty. Seriously, well done!

PostgreSQL types - can not alter and set to BIGSERIAL

Will need write a workaround using sequences

Nodal on Windows: skips author on new

Windows specific issue. Author is skipped when calling nodal new. Perhaps we should use a cross-platform stdin module.

Should be worked on at #64

Is there a module to entry data?

I am setting up a API server with Nodal. Do you have a module or so to entry data that uses the API generated in Nodal?

Nodal on Windows: unresponsive on initialize

Nodal new command doesn't seem to complete on Powershell (Windows 10) and instead gets stuck at Copying Nodal directory structure and files... although the work around is to use Ctrl+c after a few seconds to unpause Nodal but this sometimes skips Installing packages in this directory... altogether.

Loosely related to #22 #53.
Should be reviewed at #64

Does it work with MySQL?

Socket.io integration

On model generation externalInterface is not populated properly

Fix on next RC

Logo

I just wanted to make sure you're aware - Nordic Semiconductor uses the same logo.

Relationships: Finalize definitions + integrate with ORM

Probably want Rails-style support: has_one, belongs_to, has_many. Need final syntax for it. Will have to refactor modules dependent on current structure.

has_one and belongs_to are supported by ORM currently (just in Model.prototype.relationships), need to force .join() to receive properties.

Should this be the standard behavior? (Always force explicit load of relationships.) Thinking yes. For has_many can just do an id IN () query and assign everything after. (+1 query for every has_many relationship).

Development environment setup

It seems like the current recommendation for setting up a development environment is to manually install and configure a Postgres server. I'm going to suggest using [Docker Compose(https://docs.docker.com/compose/) as an alternative. I tried this myself with Nodal and it makes things a breeze. Basically the getting started would be reduced to docker-compose up. (Assuming Docker and Docker Compose are installed.)

Note that this doesn't necessarily preclude users from continuing to use their own setup, but given how popular Docker currently is, I suspect a lot of people would prefer the Docker route. Aside from documentation, only three changes are needed:

Add a Dockerfile to build a container for the Nodal app

FROM node
EXPOSE 3000

RUN npm install -g --silent nodal

WORKDIR /usr/src/app
CMD ["npm", "start"]

Add docker-compose.yml to define the service configuration

web:
  build: .
  ports:
   - "3000:3000"
  volumes:
   - .:/usr/src/app
  links:
   - db
  environment:
   - DATABASE_HOST=db
   - DATABASE_PORT=5432
   - DATABASE_USER=nodal
   - DATABASE_PASSWORD=nodal
   - DATABASE_DB=nodal_development
db:
  image: postgres
  environment:
   - POSTGRES_USER=nodal
   - POSTGRES_PASSWORD=nodal
   - POSTGRES_DB=nodal_development

Modify the development database configuration to use environment variables

This is the only slightly tricky bit. For the above to work, the production and development database configuration would have to be the same since the above relies on environment variables. However, I think this is actually an added bonus. If users don't want to use Docker, then using something like dotenv to read a set of environment variables configuring a local server would be an easy alternative.

I'm happy to put together a PR if this sounds like something you're interested in.

Generate README.md and index controller from template on nodal new

I am making this an issue vs just submitting a pr to see how folks feel about the idea.

When you run nodal new its prompts you for a name that is used in package.json. However the default index_controller has a harcoded My Nodal Application in it. Including #43 in this though, it would be nice to have the name used at the time of nodal new in both the index controller and the readme.

I would like to propose we template-ize index_controller and a readme and use them to generate the necessary files

cc: #43 #36

Proposal: find_by for models

Creating an issue for discussion, mentioned in the chat before but I think it has scrolled off.

This is a proposal for a find_by method to be added to the model.js to simplify fetching a model by a non id key. I am not trying to propose a proliferation of find_by_X at all, just a simplification.

As a example, using part 3 of the video the AccessToken#login has the following

User.query()
    .where({username: params.body.data.username})
    .end(....)

I am proposing a shorthand

User.find_by({username: params.body.data.username}, (err, models) => {
});

And yeah upfront it adds no benefit other than a simplified api (and does not return a composer)..

Rough single key only test impl https://gist.github.com/intabulas/0f1a529ecb255d7a4952

Creating multiple servers

How do I go about create multiple servers for say app and api?

ORM: Standardize how to output external interfaces for models

• .external() queries external interface, is this needed, or should it be explicit every time (remove method, use .interface instead)?
• potential: move external interfacing over to .render method for controller?
  • throw error if trying to interface with a field that was not grabbed
  • keep ComposerResult definitive aggregate field for models (can contain partial data)

Add a initial README.md to the generated Nodal application

It would be worthwhile adding a skeleton README.md to the output of nodal new that includes some basic information about bootstrapping the db, starting the server and any other helpful information. This is will be required if #36 gets done to enable the Deploy on Heroku button

error following video "role does not exist"

running nodal db:create on a new machine I got ERROR: role "postgres" does not exist

It may be too late to rerecord the video. But the solution is to createuser postgres -s to create that role.

solution may be to display example usage of createuser in the error message.

Error when running 'nodal s' using node 4.0.0

/nodal_test/app/init.js:25
app.initialize(function() {
^

TypeError: app.initialize is not a function

Allow for optional callback in cli db commands vs straight process.exit

While working on #43 / #36 the postdeploy in the heroku app.json allows for a single script. Depending on the state of the app when the web flow is initiated, there might also be migrations existing (say for someone deploying a sample app).

In this case having a super command, lets call it db:bootstrap, that would call both db:prepare and db:migrate inside maybe a async.series would be very helpful. Right now db commands have process.exit(0) in them.

This issue proposes we add an optional callback param to the db functions that would if excited would be called instead of process.exit(0) so that the next call in the series would fire. This also means we will need to arg and flags to all methods for consistency.

Needs more thought, but its the shell of the idea

Graphql support?

How would that fit into the nodal ecosystem? I was going to try it out and prototype it, any thoughts/concerns @keithwhor?

Rewrite README.md to include Daemon (0.3.1)

Add deploy to heroku button

https://blog.heroku.com/archives/2014/8/7/heroku-button y/n ?

trouble installing

Following the instructions in the readme, but getting this when I try to install nodal:

module.js:328
throw err;
^

Error: Cannot find module 'minimatch'
at Function.Module._resolveFilename (module.js:326:15)
at Function.Module._load (module.js:277:25)
at Module.require (module.js:354:17)
at require (internal/module.js:12:17)
at Object. (/usr/local/lib/node_modules/npm/node_modules/rimraf/node_modules/glob/glob.js:44:17)
at Module._compile (module.js:398:26)
at Object.Module._extensions..js (module.js:405:10)
at Module.load (module.js:344:32)
at Function.Module._load (module.js:301:12)
at Module.require (module.js:354:17)
at require (internal/module.js:12:17)

Syntax error when calling 'nodal new'

installed a fresh nodal build, ran into this error when creating a new project

/usr/local/lib/node_modules/nodal/cli/cli.js:7
  const colors = require('colors/safe');
  ^^^^^
SyntaxError: Use of const in strict mode.
    at exports.runInThisContext (vm.js:73:16)
    at Module._compile (module.js:443:25)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Function.Module.runMain (module.js:501:10)
    at startup (node.js:129:16)
    at node.js:814:3

0.5.0 Release Guidelines

Things needed before release:

• ✅ Full method documentation
• ❔ Docs generator from documented methods (put on website)
• ❔ Step-by-step Instructional README, offload API docs to website
• ✅ Allow for hyphens in project names (don't just replace with underscore)
• ✅ Composer: Relationships through other Models (many:many)
• ✅ Composer: Where by joined fields
• ✅ Composer: Multiple joined children (children, pets in tests)
• ✅ Composer: groupBy, aliased columns, transformations = experimental (Removed)
• ✅ Composer: Ensure subsequent filters do not remove joined columns.
• ✅ Composer: Subsequent wheres by joined fields should work
• ✅ Model: Change Model.joins(Parent) to Model.joinsTo(Parent).
• ✅ Model: Change Model#relationship to Model#include

Websocket for API services.

Hi @keithwhor, thanks for this amazing framework, I'm really happy to finally have a Django-like ORM to work with in Node JS.

I just want to ask you, What are your thoughts about Websocket support?, It would be great if API server works with Websocket, updating subscribers when a given model have CUD operations.

Make "everything" into a Middleware for pipeline based code

So you can extract ParamParser, Cookies, Session, Loggers into their own modules and let people insert the middleware they want as replacement or above

Suggestions for middleware

• Logging
• Cookies
• Response
• Request
• Routing
• Gzip

Won't scheduled tasks block your incoming API requests?

I'm trying to understand the design patterns here, why would you run scheduled tasks on your API server?

cli help

nodal cli doesn't seem to have a help guide.
Please add.

nodal s error on windows 7

Hi, i followed the getting started guide and when i try execute nodal s command i get this exception:

events.js:141
      throw er; // Unhandled 'error' event
      ^

Error: spawn npm ENOENT
    at exports._errnoException (util.js:856:11)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:178:32)
    at onErrorNT (internal/child_process.js:344:16)
    at doNTCallback2 (node.js:452:9)
    at process._tickCallback (node.js:366:17)
    at Function.Module.runMain (module.js:459:11)
    at startup (node.js:138:18)
    at node.js:974:3

Pls tell me what i doing wrong.

Gitter room for Nodal

Hi Keith,

Could you please a Gitter room for Nodal?

Replace native dependency `bcrypt` with `bcryptjs`

Native dependency bcrypt is a native (C++) module that requires node-gyp and python to build and could be replaced with a pure JS version. I am a fan of native modules myself but I don't believe using them in projects like this is appropriate especially when an excellent alternative such as bcyptjs exists. Although bcryptjs does not perform as fast as C++ version (2.7x slower which isn't too bad for a JS implementation) It would be ideal to use bcryptjs as a dependency to make the installation process easier and more friendly to NodeJS newcomers at a cost which is reasonable. I've been using bcryptjs in production and have only noticed small delays in logins during load times which is nothing truly significant. Also bcryptjs is a drop-in replacement.

Note that the native version of the addon can still be used if we depend on the pure JS version. A user can install bcrypt globally and we can check if it is possible to require it (although I personally believe this is not an ideal method yet not an anti-pattern).

If you are open to this I can create a PR.

Missings docs

@keithwhor good job, I like the simplicity of Nodal, so keep the focus on it. There is no doc on the nodaljs.com and I think this should be fixed very quickly.

Generate db.json when generating app to use app name for database name

Right now db.json has a hardcoded nodal_development database name. Would be nice to generate the config when running nodal new so that the db name could include the app name vs the default

Optionally prompt during nodal new for the db name to allow it to be configured a bit (can prompt have dynamic defaults?

Only issue I see is running doT over a file that uses doT itself.

Meaningful errors

Currently Nodal can throw meaningless errors such as ECONNRESET or at best ERROR: password authentication failed for user "postgres". These errors should be replaced with more meaningful errors such as

• Failed to connect to the database. Make sure you have installed and configured PostgreDB
• Failed to authenticate for user "postgres". Make sure you have configured your database settings at "./config/db.json".

Too many DB connections with new Daemon (0.3.1)

Not closing DB connections properly on app cleanup when Daemon relaunches

Roadmap and project development insights

I just ran into nodal via echojs, saw the intro video and looked around the source code. I'm actually really excited about the project. So far I like what I've seen but I still have to take it for a run.
Here's the thing, I'm trying to decide wether I should spend any time on it or not, or when I should spend that time- now or after X feature is implemented.

Is there a roadmap you could share or that I have not seen yet? Is there an official blog where I could follow progress/updates/reasoning? I've seen some of the github issues but I don't think github is the appropriate medium for this.

For instance, I'm curious as to why you decided to implement- I think- the ORM from scratch. Personally is one of the key factors for me to consider using a framework or not.

thanks

Rethinking Task Support

This issue is being opened based on a conversation between @keithwhor @schahriar and myself in the nodal gitter chat room.

After test implementations for dependencies in cli commands we all felt at this time is better to to use tasks as a way to execute complex commands as well as supporting an eco-system of third party tasks. This would also limit the churn on the cli to implement everyones favorite commands (ex: rails'esk db:reset)

This ticket proposes the following

• Update the cli task generator to generate straight tasks and optionally scheduled tasks. These tasks are generated into the application, Currenty nodal g:task generates a Nodal.SchedulerTask which still can be run one off via nodal task <taskname>
• Update the cli task command to accept optional options (argv) to pass into the executed task. ie: nodal task do_some_cool_stuff --silent
• Update the scheduler to use the same interface (but will empty args) so the same task can be a one off or scheduled task
• Include an example task/tasks in a nodal new that are useful. Maybe a command for a postgres vacuum?
• Consider packaging these as a separate package (eg: nodal-example-tasks), include it in the default app package.json and include some (tdb) initialization after npm i to copy them in the app (vs default them in src/)

note: desire would be unless deemed really useful, example/default tasks on nodal new should not require us to add new third party packages to a nodal app

Invalid select generation after 83222e68

I noticed while working on #50 that db:migrate was broken in master.. Roots to the changes you made in 83222e6 adding .update() to composer

it generates the query for migrations as

SELECT ("schema_migrations"."id") AS "id" FROM (null) AS "schema_migrations" since subQuery is null in the migrate command.

I will work on a fix since its blocking #50

Why IIFES around everything?

Why are all of the modules in the examples wrapped in an IIFE?

Promises

Given they're in the language standard now, it would probably be the better idea to use them or at least support them - especially since people want to use async/await or generator coroutines in new code.

CLI Refactoring

I am opening this ticket as a place for discussion around refactoring the CLI portions of nodal based in part on @keithwhor comment in #50

While there are issue for smaller fixes now (like #50) this is to share and discuss direction and decisions on how a CLI refresh she be acomplished.

cc: @schahriar

.destroy Cascade, .destroyAll() for ModelArray

Need these features.

Also need to make sure custom .save() and .destroy() methods are triggered from the ModelArray methods. Will have to think on it.

Replace the prompt module with inquirer

ref: https://github.com/SBoudrias/Inquirer.js

This issue tracks a low impact/risk task to replace the existing use of the flatiron prompt module with inquirer. Prompt is only used (currently) in nodal new to prompt users. One of its drawback is lack of access to previous answers in following questions, so doing thing like defaulting a database name to the entered app name is not achievable. Its also has not been updated in over a year and given nodejitsu's acquisition by godaddy who knows if the flatiron stuff will be maintained.

Inquirer allows us to define prompt defaults as functions an thus get at previous anwers. It only contains a collection of nice console inputs (check boxes, select list, etc) that in the future would provide useful. An example of that would be picking the db adapter (when we get more adapters) to use

cc: @schahriar

Negative interface matching?

So instead of defining [id, username, created_at] as interface wouldn't it be better to do allow for the opposite and only define the things to hide?

I propose thse following syntax: ['!password'] will hide the password and use all default fields. This should also work in cascade relationships like [{user: ['!password']}].

Now I don't need to worry to keep updating the interfaces everywhere when I created or remove a field from my model.

Name for routes

Define routes with their name once and we can use them globally, such as: in views or client script.