8eecf0d2/netlify-local

Local Netlify service emulation

netlify-local's Issues

Correctly implement Redirect feature

This is a pretty insane feature considering what it's called, what it's actually capable of, and how it's implemented...

Notice: It's confusing and important to understand if the "redirect rule" in question is a true 3xx style redirect or a more fancy rewrite.

The main features to implement:

• Traditional Redirect
• Traditional Rewrite (to filesystem / build.publish)
• Proxy Rewrite
• Placeholder / Pattern matching

Redirect Features

Basic Redirects / Rewrites, Http Status Codes

Netlify will return status code 301 for what it calls basic redirects.
Netlify will perform a rewrite if a status code is set and not equal to 301, 302, 303.

• Support "basic redirect"
• Support "basic rewrite"

Custom 404

By default Netlify will serve a 404.html file if it's provided and no redirect rules or static files match the path.

• Support for custom 404.html

Trailing Slashes

I'm not even gonna bother with this yet...

Placeholders

Basically param swapping between from and to - should be easy enough to implement with express and a url pattern library.

from = "/:partA/:partB"
to = "/:partB/:partA"
# "/foo/bar" → "/bar/foo"

• Support placeholder / pattern replacement

Splats 🤦‍♀️

Similar to placeholders except the * and :splat param are bound.

• Support special splat placeholder

Query Params

More or less states that query params can be used as placeholders, the syntax is weird but toml is luckily more readable that Netlify's _redirect file formatting.

• Support query placeholder

Hostnames & Protocols, GeoIP & Language

These feature are getting way beyond my personal use cases - plus I don't see them being entirely useful for local development, it'd be great to support them but I'm not holding my breath.

Correctly implement Netlify Static File Server

I'm not entirely sure if there are any special quirks when it comes to Netlify's static file serving but I'm guessing there are - would be good to list them and start implementing if within scope.

CLI Defaults don't work

For whatever reason cli defaults don't work.

The following commands should all return the same result.

netlify-local

netlify-local -n netlify.toml

netlify-local -w webpack.config.js

netlify-local -n netlify.toml -w webpack.config.js

Repo installs fail

It can be quite helpful to do repo installs

$ npm -g i '8eecf0d2/netlify-local'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/netlify-local/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

npm ERR! path /usr/local/lib/node_modules/netlify-local/.dist/cli.js
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall chmod
npm ERR! enoent ENOENT: no such file or directory, chmod '/usr/local/lib/node_modules/netlify-local/.dist/cli.js'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2018-12-12T13_20_46_337Z-debug.log

Tail of the debug log

10944 info linkStuff [email protected]
10945 silly linkStuff [email protected] has /usr/local/lib/node_modules/netlify-local/node_modules as its parent node_modules
10946 silly linkStuff [email protected] is part of a global install
10947 silly linkStuff [email protected] is installed into a global node_modules
10948 verbose linkBins [ { webpack: './bin/webpack.js' },
10948 verbose linkBins   '/usr/local/lib/node_modules/netlify-local/node_modules/.bin',
10948 verbose linkBins   false ]
10949 silly build [email protected]
10950 info linkStuff [email protected]
10951 silly linkStuff [email protected] has /usr/local/lib/node_modules as its parent node_modules
10952 silly linkStuff [email protected] is part of a global install
10953 silly linkStuff [email protected] is installed into a global node_modules
10954 silly linkStuff [email protected] is installed into the top-level global node_modules
10955 verbose linkBins [ { 'netlify-local': '.dist/cli.js' }, '/usr/local/bin', true ]
10956 timing action:build Completed in 126ms
10957 verbose unlock done using /root/.npm/_locks/staging-3a08f0df5026584d.lock for /usr/local/lib/node_modules/.staging
10958 timing stage:rollbackFailedOptional Completed in 652ms
10959 timing stage:runTopLevelLifecycles Completed in 46801ms
10960 warn optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/netlify-local/node_modules/fsevents):
10961 warn notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
10962 verbose notsup SKIPPING OPTIONAL DEPENDENCY: Valid OS:    darwin
10962 verbose notsup SKIPPING OPTIONAL DEPENDENCY: Valid Arch:  any
10962 verbose notsup SKIPPING OPTIONAL DEPENDENCY: Actual OS:   linux
10962 verbose notsup SKIPPING OPTIONAL DEPENDENCY: Actual Arch: x64
10963 verbose stack Error: ENOENT: no such file or directory, chmod '/usr/local/lib/node_modules/netlify-local/.dist/cli.js'
10964 verbose cwd /root
10965 verbose Linux 4.19.7-1-default
10966 verbose argv "/usr/bin/node10" "/usr/bin/npm-default" "-g" "i" "8eecf0d2/netlify-local"
10967 verbose node v10.13.0
10968 verbose npm  v6.4.1
10969 error path /usr/local/lib/node_modules/netlify-local/.dist/cli.js
10970 error code ENOENT
10971 error errno -2
10972 error syscall chmod
10973 error enoent ENOENT: no such file or directory, chmod '/usr/local/lib/node_modules/netlify-local/.dist/cli.js'
10974 error enoent This is related to npm not being able to find a file.
10975 verbose exit [ -2, true ]

Allow multiple webpack configs

Should handle use case where the webpack config is an array of configs, probably common for small applications where the client and lambda's are in the same repo.

/webpack.config.js

const lambdaConfig = require("./src/lambda/webpack.config.js");
const clientConfig = require("./src/client/webpack.config.js");

module.exports = [
  lambdaConfig,
  clientConfig,
]

Custom Netlify configuration properties

In addition to #11, the Netlify configuration could also be used for other netlify-local options.

[plugins.local.webpack]
  config = "webpack.js"
[plugins.local.server]
  static = true
  lambda = true
  certificates = "private/certs"
  port = 9000
[plugins.local.functions]
  source = "src/lambda/ts"
  files = [
    "foo.js",
    "nested/bar.js"
  ]

This isn't as pretty as I'd like but I'm new to toml so maybe there's a better way to write this

Gracefully handle port in use

[netlify-local] starting http server
events.js:183
      throw er; // Unhandled 'error' event
      ^

Error: listen EADDRINUSE :::8100
    at Server.setupListenHandle [as _listen2] (net.js:1360:14)
    at listenInCluster (net.js:1401:12)
    at Server.listen (net.js:1485:7)

It is fairly common to be running multiple instances of an app framework in development, with ports preselected by a tool such as netlify-local .

ionic service looks for the next available port, and iirc ng serve does the same.

Add command 'build'

netlify.toml has the build command .. it just needs to be run, .. on unix ;-)

For windows, a workaround would be to do the build using docker with the netlify build image.

That would even be useful in some instances for unix users.

Add tests

Since the different Netlify Services aren't completely defined, testing will be based on current assumptions about how things should work. In addition to writing tests, adding more error handlers should be considered (eg, invalid webpack config).

Unit Tests

Config

• parseWebpackConfig()
• parseNetlifyConfig()

Webpack

• Webpack.initialize()
• Webpack.watch()
• Webpack.build()

Server

• Server.initialize()
• Server.listen()
• Server.close()
• Server.routeRedirects()
• Server.routeHeaders()
• Server.routeLambdas()
• Server.lambdaRequest()
• Server.lambdaContext()

Integration (sorta functional)

Redirects

• Standard Redirect
• Standard Rewrite
• Proxy Rewrite
• Placeholder (Path Params)
• Placeholder (Query Params)
• Force

Add Bundle command

Although this is out of the original scope for netlify-local I've been thinking a bit about the best way to implement this..

I think it's critically important to ensure end users can supply their own Webpack configurations and for netlify-local to be very cautious and explicit with any modifications it makes.

An example of the bundle command would require you to specify the Netlify and Webpack configuration files.

netlify-local bundle -n netlify.toml -w webpack.config.js

This would work in very much the same way that the serve command already does, however we need to add some logic to figure out what files are going to be used as Netlify Functions, a neat idea which I'm semi-skeptical about employing is to modify the Netlify configuration with a plugins property 🤭

Once we have a list of functions, we still need to modify the Webpack configuration - using 8eecf0d2/webpack-netlify-lambda-plugin seems like a good solution.

From there the user could define options within their Netlify configuration which will be used by 8eecf0d2/webpack-netlify-lambda-plugin.

[build]
  ...

[plugins]
  [local]
    source = "src/ts"
    functions = [
      "foo.js",
      "nested/bar.js"
    ]

I'm sorta surprised Netlify doesn't require you to specify which files are Functions but considering there's no real options for individual Functions it makes a little bit of sense.

I'd like to see Netlify support options similar to what you get with Serverless configurations (or the other way round and see Serverless support Netlify as a provider!)...

[[functions]]
  name = "foo"
  file = "functions/foo"
  method = "GET"
  ...

Modifying the schema of Netlify configuration scares me a little since they might not have thought about such a use case and even if they have it might not align with their plans for the future.

Correctly implement Netlify Functions

Probably a lot of gotchas with Netlify Functions but implementing what's stated in docs should be simple enough.

Handler

Documentation: https://www.netlify.com/docs/functions#the-handler-method

• Support Callback or Promise handlers.
• Support context arg.
• Correctly handle errors.
• Write correct Typings.

Action required: Greenkeeper could not be activated 🚨

🚨 You need to enable Continuous Integration on all branches of this repository. 🚨

To enable Greenkeeper, you need to make sure that a commit status is reported on all branches. This is required by Greenkeeper because it uses your CI build statuses to figure out when to notify you about breaking changes.

Since we didn’t receive a CI status on the greenkeeper/initial branch, it’s possible that you don’t have CI set up yet. We recommend using Travis CI, but Greenkeeper will work with every other CI service as well.

If you have already set up a CI for this repository, you might need to check how it’s configured. Make sure it is set to run on all new branches. If you don’t want it to run on absolutely every branch, you can whitelist branches starting with greenkeeper/.

Once you have installed and configured CI on this repository correctly, you’ll need to re-trigger Greenkeeper’s initial pull request. To do this, please delete the greenkeeper/initial branch in this repository, and then remove and re-add this repository to the Greenkeeper App’s white list on Github. You'll find this list on your repo or organization’s settings page, under Installed GitHub Apps.

pnpm support

pnpm is a wonderful replacement for npm, but it isnt working with netlify-local.

pnpm -g i netlify-local
Resolving: total 269, reused 205, downloaded 00 0wnloaded 0
Packages: +1
++-
Resolving: total 379, reused 377, downloaded 2, done

/home/jayvdb/vs/node/8.13.0/pnpm-global/1:
+ netlify-local 1.6.0

jayvdb@linux-lwww:~/projects/viperdev/template/ionic> netlify-local serve
module.js:550
    throw err;
    ^

Error: Cannot find module 'serve-static'
    at Function.Module._resolveFilename (module.js:548:15)
    at Function.Module._load (module.js:475:25)
    at Module.require (module.js:597:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/home/jayvdb/vs/node/8.13.0/pnpm-global/1/node_modules/.registry.npmjs.org/netlify-local/1.6.0/node_modules/netlify-local/.dist/server.js:52:19)
    at Module._compile (module.js:653:30)
    at Object.Module._extensions..js (module.js:664:10)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Function.Module._load (module.js:498:3)

After a little patch, and a little fiddle,

> pnpm run test

> [email protected] pretest /home/jayvdb/projects/netlify/netlify-local
> pnpm run build


> [email protected] build /home/jayvdb/projects/netlify/netlify-local
> tsc --project ./tsconfig.json


> [email protected] test /home/jayvdb/projects/netlify/netlify-local
> ts-mocha --project ./test/tsconfig.json test/**/*.ts



  Config
    parseNetlifyConfig
      ✓ should throw when not found
      ✓ should correctly parse build
      ✓ should correctly override build with context
    parseWebpackConfig
      ✓ should throw when not found
      ✓ should correctly import webpack config

  Server
    lifecycle
      ✓ should listen and close
      ✓ should add redirect routes
      ✓ should add redirect header routes
      ✓ should add header routes
      ✓ should add lambda route
      ✓ should correctly format lambda request
      ✓ should correctly format lambda context


  12 passing (202ms)

Expose API

Would be handy to expose an API so test runners can start & stop the server, optionally with the build system.

Usage

import { parseNetlifyConfig, parseWebpackConfig, Server, Webpack } from "netlify-local";

const netlifyConfig = parseNetlifyConfig("netlify.toml");
const port = 3000;
const server = new Server({
  netlifyConfig: netlifyConfig,
  routes: {
    static: true,
    lambda: true,
  },
  certificates: {
    key: <private key as a string>,
    cert: <certificate as a string>,
  },
  port: 9000,
});
await server.listen();
await server.close();

const webpackConfig = parseWebpackConfig("webpack.config.js");
const webpack = new Webpack(webpackConfig);
await webpack.build();
webpack.watch();

Methods

Config

• parseNetlifyConfig()
• parseWebpackConfig()

Server

• constructor
• Server.listen()
• Server.close()

Webpack

• constructor
• Webpack.build()
• Webpack.watch()

Github Actions

Would be cool to integrate Github Actions and drop Travis dependency - have just tried to implement with little success and could not filter by tag or figure out how to only yarn publish based on tag.

Not really interested in testing this so PR appreciated.

Desired Result

Workflow for develop, feature/* and hotfix/* branches

1. yarn install
2. yarn test

Workflow for tagged master branch

1. branch = master, tag = *.*.*
2. yarn install
3. yarn test
4. yarn publish

Correctly parse and implement netlify.toml

Netlify doesn't clearly specify required properties for toml configs and testing what happens within different scenarios is a pain so for the purpose of simplifying things I think it's best to enforce explicit with toml configurations, for example an error will be thrown if the static router is being initialised but there is no build.publish property specified.

Context

• Derive context from current git branch.
• Add a -c --context option to override.
• Inject environment variables if any.
• Support base prop.

Redirects

• Support from, to prop.
• Support status prop.
• Support force prop (could be tricky).
• Support query prop (could be tricky).
• Support conditions prop (could be really tricky).
• Support headers prop.

Headers

• General support for setting headers based on for prop.

Disable functions if build.functions is not set

Currently there is a backtrace if build.functions doesnt exist in the toml file.

That should disable the lambda , allowing basic static site serving without cli extra flags.

Add support for local Https

Would be nice to add support for HTTPS, probably keep it simple and just pass in a directory and expect key.pem, cert.pem to exist.

Example

netlify-local serve --certificates private/certs

An in-range update of @types/node is breaking the build 🚨

The devDependency @types/node was updated from 10.12.8 to 10.12.9.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

@types/node is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details

• ❌ continuous-integration/travis-ci/push: The Travis CI build could not complete due to an error (Details).

---

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.

---

Your Greenkeeper Bot 🌴

express: TypeError: Cannot read property 'length' of undefined

I've not dug into this problem, so I dont know the exact cause.

[netlify-local] TypeError: Cannot read property 'length' of undefined
    at pathtoRegexp (/home/jayvdb/vs/node/8.13.0/lib/node_modules/netlify-local/node_modules/path-to-regexp/index.js:67:49)
    at new Layer (/home/jayvdb/vs/node/8.13.0/lib/node_modules/netlify-local/node_modules/express/lib/router/layer.js:45:17)
    at Function.use (/home/jayvdb/vs/node/8.13.0/lib/node_modules/netlify-local/node_modules/express/lib/router/index.js:464:17)
    at Function.<anonymous> (/home/jayvdb/vs/node/8.13.0/lib/node_modules/netlify-local/node_modules/express/lib/application.js:220:21)
    at Array.forEach (<anonymous>)
    at Function.use (/home/jayvdb/vs/node/8.13.0/lib/node_modules/netlify-local/node_modules/express/lib/application.js:217:7)
    at Server.routeStatic (/home/jayvdb/vs/node/8.13.0/lib/node_modules/netlify-local/.dist/server.js:95:22)
    at Server.initialize (/home/jayvdb/vs/node/8.13.0/lib/node_modules/netlify-local/.dist/server.js:82:14)
    at new Server (/home/jayvdb/vs/node/8.13.0/lib/node_modules/netlify-local/.dist/server.js:66:14)
    at Object.<anonymous> (/home/jayvdb/vs/node/8.13.0/lib/node_modules/netlify-local/.dist/cli.js:80:30)

netlify.toml:

[build]
  command = 'export PATH=./.ci/bin/:"$PATH" && build-netlify'
  publish = 'www'

[[redirects]]
  from = '/api/*'
  to = 'http://localhost:8000/:splat'
  status = 200
  force = true

[[redirects]]
  from = '/*'
  to = '/index.html'
  status = 200

I can work around this by editing /home/jayvdb/vs/node/8.13.0/lib/node_modules/netlify-local/node_modules/path-to-regexp/index.js function pathtoRegexp to add

if (path === undefined) {
    return new RegExp();
  }

The site seems rather ok, with that hack, but ive not exhaustively tested yet