Middleware Mounting

[strongloop]

Middleware Mounting

[bajtos]

@ritch actually I have already 14 story points on my plate for sprint 55, I don't think I'll be able to finish this by the end of this sprint. I can keep it in the sprint backlog as a stretch goal.

[bajtos]

As a developer, I want to customise certain parts of middleware configuration depending on the environment (development, production).

loopback-boot should implement config overloading via middleware.production.json and friends.

[bajtos]

I am wondering if middleware phases should be handled at the boot time as a mechanism to control the order in which loopback-boot executes app.use() for different middleware, or whether it should be used by loopback runtime to actually perform the request handling.

[bajtos]

Constraints

To support per-environment customization of middleware config, we need to implement merging of data from environment-specific files. To make the merge algorithm robust but also consistent with the way how other config files are merged, we should use middleware id as the key.

The configuration file should allow the developer to define custom phases.

Proposed solution

This is based on the initial proposal outlined in loopback-example-component

The middleware id is a require-able string:

• a path relative to app root dir, e.g. ./middleware/custom
• an absolute path, e.g. /usr/var/foo/bar/middleware.js
• a module exporting a middleware constructor, e.g. compression
• a module-relative path, e.g. loopback-component-passport/middleware

The config value is either:

• an array of arguments to pass to middleware constructor (see "morgan.config" in the example below)
• or the value to be passes to the middleware constructor as the single argument (see "compresion.config" in the example below)

The list of phases specified in the JSON file is merged with the list of built-in phases provided by loopback.

boot.compileToBrowserify should throw an error if the client configuration includes middleware.

Example

server/middleware.json

{
  "phases": [
    "initial",
    "session",
    "auth",
    "parse",
    "log"
  ],
  "middleware": {
    "compression": {
      "phase": "init",
      "enabled": true,
      "config": {
      }
    },
    "morgan": {
      "phase": "log",
      "enabled": true,
      "config": [
        "dev",
        {
          "immediate": true
        }
      ]
    }
  }
}

server/middleware.production.json

{
  "middleware": {
    "compression": {
      "config": {
        "threshold": 1024
      }
    },
    "morgan": {
      "enabled": false
    }
  }
}

/to @ritch @raymondfeng Please review and let me know if there is anything awry. I'll start the implementation tomorrow morning my time.

[bajtos]

An alternative: move the content of middleware to the top level, rename phases to _phases or even _meta.phases.

[ritch]

I am not sure I fully comprehend the merging requirements / per-environment customization so I'll defer to your judgement there. As far as the config structure (argument handling, config object, etc) that all looks fine to me.

[ritch]

An alternative: move the content of middleware to the top level, rename phases to _phases or even _meta.phases.

-1

[raymondfeng]

1. I have started to work on the ability to register utility functions so that they can be referenced in JSON files by name. See strongloop/loopback@f278ac8. I recently found out that hapi has the similar concept: http://hapijs.com/tutorials/server-methods. For the middleware id, we should support this flavor in addition to module paths.
2. I wonder if we should separate phase and middleware configurations into different files. In the big picture of policy framework, we want to unite middleware and hooks. The phase configuration will define phases and dependencies/ordering.
3. I also wonder if we should separate middleware definition from configuration as follows:

• middleware/logger.json

  {
  "name": "logger",
  "module": "morgan",
  "options": {
   "immediate": "boolean"
  } 
  }
  

• server/middleware-config.json

{
  "logger": {
    "immediate": true,
   "phase": "initial"
  }
}

[bajtos]

I have started to work on the ability to register utility functions so that they can be referenced in JSON files by name

Could you please provide an example how (where) do you want to use the utility function in the JSON config?

I wonder if we should separate phase and middleware configurations into different files. In the big picture of policy framework, we want to unite middleware and hooks. The phase configuration will define phases and dependencies/ordering.

That's an interesting point. What is the timeframe, when are we going to implement the policy framework?

I believe we can support both middleware.json > phases and policy phases defined in e.g. phases.json, as long as loopback-boot is able to merge and/or map the middleware phases to policy phases.

If my assumption is correct, then we don't have to worry about policy phases now.

[bajtos]

I also wonder if we should separate middleware definition from configuration as follows:

This is for the next version. The initial implementation does include middleware definition, the middleware is a file exporting a constructor/factory function.

I have started to work on the ability to register utility functions so that they can be referenced in JSON files by name. See strongloop/loopback@f278ac8. I recently found out that hapi has the similar concept: http://hapijs.com/tutorials/server-methods. For the middleware id, we should support this flavor in addition to module paths.

I am ok with running an utility function to provide configuration values. Since the utility function is a concept implemented in loopback, processing of utility function calls referenced by middleware config should be implemented in loopback too. I'll rework app.middleware API to support that use case too.

However, I won't implement processing of utility function calls referenced by middleware config, as that's not strictly necessary for middleware phases and I want to focus on getting the first complete version out of the door as soon as possible.

As for the middleware id, I really don't understand what problem is solved by utility functions.

[raymondfeng]

@bajtos I'm not proposing to use the utility function to provide configuration values. I was suggesting that the utility function name should also be checked when a middleware id is resolved to a function. Here is an example:

"middleware": {
    "compression": { // Is this the middleware id?
      ...
    },
    "./middleware/app-custom-middleware": {
      ...
    },
    "my-component/component-custom-middleware": {
      ...
    },
   "$middleware-registered-as-a-utility": {
      ...
    }

Let's defer this for now.

[bajtos]

Let's defer this for now.

👍

[bajtos]

Ad documentation: strongloop/loopback#863 and https://gist.github.com/bajtos/e7eaba736ff096916b71

I am closing this issue as done. Note that the feature was not yet released to npmjs.org, we are waiting for #70.