k-foss/ts-esnode

Node.JS Loader hook to transform typescript source and files as node loads them TS-ESNode. TS-Node for ESNext & ESModule Node.JS.

ts-esnode's Issues

Node Module ESModule Support

I've noticed that my dynamic loader doesn't work with ESModule node_modules. This can be fixed by only passing format: 'dynamic' when the default resolve's format isn't 'module'.

Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath is not defined by "exports"

If a TypeScript file imports an NPM package whose package.json contains subpath exports, the loader crashes with Error [ERR_PACKAGE_PATH_NOT_EXPORTED].

Files to Reproduce

package.json

{
  "private": true,
  "type": "module",
  "scripts": {
    "build": "tsc --noEmit",
    "dev": "node --loader @k-foss/ts-esnode --experimental-specifier-resolution=node main.ts"
  },
  "dependencies": {
    "@k-foss/ts-esnode": "^2.0.2",
    "@types/node": "^15.0.2",
    "dotenv": "^9.0.2",
    "typescript": "^4.2.4"
  }
}

tsconfig.json

{
  "compilerOptions": {
    "allowSyntheticDefaultImports": true,
    "module": "ESNext",
    "moduleResolution": "Node",
    "strict": true,
    "target": "ESNext"
  }
}

main.ts

import dotenv from "dotenv";

dotenv.config();

Error Message

sunny@box1:~/code/bug$ npm run dev

> dev
> node --loader @k-foss/ts-esnode --experimental-specifier-resolution=node main.ts

(node:251905) ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:251905) Warning: The dynamicInstantiate loader hook has been removed.
node:internal/process/esm_loader:74
    internalBinding('errors').triggerUncaughtException(
                              ^

Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/main.js' is not defined by "exports" in /home/sunny/code/bug/node_modules/dotenv/package.json
    at new NodeError (node:internal/errors:363:5)
    at throwExportsNotFound (node:internal/modules/esm/resolve:321:9)
    at packageExportsResolve (node:internal/modules/esm/resolve:546:3)
    at resolveExports (node:internal/modules/cjs/loader:478:36)
    at Function.Module._findPath (node:internal/modules/cjs/loader:518:31)
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:927:27)
    at Function.Module._load (node:internal/modules/cjs/loader:774:27)
    at Module.require (node:internal/modules/cjs/loader:1013:19)
    at require (node:internal/modules/cjs/helpers:93:18)
    at getSource (file:///home/sunny/code/bug/node_modules/@k-foss/ts-esnode/out/dist/index.js:131:27) {
  code: 'ERR_PACKAGE_PATH_NOT_EXPORTED'
}

Analysis

node_modules/dotenv/package.json contains:

{
  "main": "lib/main.js",
  "exports": {
    ".": "./lib/main.js",
    "./config": "./config.js",
    "./package.json": "./package.json"
  }
}

It isn't allowable to require("dotenv/lib/main.js"); due to this subpath exports specification.
Instead, the loader should invoke require("dotenv"), i.e. do not resolve export subpath to the actual file.

Environment

• Node 16.1.0
• NPM 7.11.2
• TS-ESNode 2.0.2

Axios: "SyntaxError: Identifier '$default' has already been declared"

The code below fails with error:

file://.../node_modules/axios/index.js:117
let $default;
    ^

SyntaxError: Identifier '$default' has already been declared
    at createDynamicModule (internal/modules/esm/create_dynamic_module.js:37:13)
    at Loader.loaderInstance (internal/modules/esm/loader.js:204:16)
    at async link (internal/modules/esm/module_job.js:37:21)

Commannd:

node --loader @k-foss/ts-esnode --experimental-specifier-resolution=node src/index.ts

// src/index.ts
import axios from "axios";
console.log(axios);

tsconfig:

{
  "compilerOptions": {
    "moduleResolution": "node",
    "lib": [
      "esnext"
    ],
    "target": "esnext",
    "module": "esnext"
  }
}

feat(Tests): Move testing to @K-FOSS/TS-ESTests

I'm working on creating my own jest alternative over at @K-FOSS/TS-ESTests. It is/will be a super lightweight Jest alternative with essentially no dependencies. (Only dep is our super tiny @K-FOSS/TS-Worker module to allow for my other repos to have 100% TypeScript since my worker thread loading code includes 5 lines of JS)

TypeScript Transformers

Somehow provide a way to allow users to specify TypeScript transformers to run during the transpile.

TS-Node Integration

See how much of this work can be porting directly into TS-Node

tsconfig paths are ignored

It seems that paths key of tsconfig.json is being ignored.

Environment: Node 14.0.0 on Ubuntu 18.04

package.json

{
  "private": true,
  "type": "module",
  "scripts": {
    "build": "tsc --noEmit",
    "dev": "node --loader @k-foss/ts-esnode --experimental-specifier-resolution=node main.ts"
  },
  "devDependencies": {
    "@k-foss/ts-esnode": "^1.3.5",
    "typescript": "^3.8.3"
  }
}

tsconfig.json

{
  "compilerOptions": {
    "module": "ESNext",
    "target": "ESNext",
    "moduleResolution": "Node",
    "baseUrl": ".",
    "paths": {
      "hello": ["./demo"]
    }
  }
}

main.ts

import { f } from "hello";
console.log(f());

demo/index.ts

export function f() { return 1; }

npm run build completes successfully, which means that the TypeScript is valid.

npm run dev fails with the following error:

(node:9473) ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
internal/modules/run_main.js:54
    internalBinding('errors').triggerUncaughtException(
                              ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'hello' imported from /mnt/c/Users/sunny/Documents/code/20200424/main.ts
    at packageResolve (internal/modules/esm/resolve.js:615:9)
    at moduleResolve (internal/modules/esm/resolve.js:646:14)
    at defaultResolve (internal/modules/esm/resolve.js:692:13)
    at resolve (file:///mnt/c/Users/sunny/Documents/code/20200424/node_modules/@k-foss/ts-esnode/out/dist/index.js:42:12)
    at Loader.resolve (internal/modules/esm/loader.js:97:40)
    at Loader.getModuleJob (internal/modules/esm/loader.js:243:28)
    at ModuleWrap.<anonymous> (internal/modules/esm/module_job.js:42:40)
    at link (internal/modules/esm/module_job.js:41:36) {
  code: 'ERR_MODULE_NOT_FOUND'
}```

Warning: The dynamicInstantiate loader hook has been removed.

dynamicInstantiate loader hook is gone since Node v14.5.0.
It causes warning:

Warning: The dynamicInstantiate loader hook has been removed.

For some code, including the test suite of this repository, it causes error:

RangeError [ERR_UNKNOWN_MODULE_FORMAT]: Unknown module format: dynamic
    at Loader.getModuleJob (internal/modules/esm/loader.js:236:13)
    at async ModuleWrap.<anonymous> (internal/modules/esm/module_job.js:53:21)
    at async Promise.all (index 3)
    at async link (internal/modules/esm/module_job.js:58:9) {
  code: 'ERR_UNKNOWN_MODULE_FORMAT'
}

nodejs/node#33501 explains why they removed dynamicInstantiate.

Importing a directory with a index.(js,ts,tsx,jsx). (findFiles supporting directories)

Right now you need to import('./DIR/index')` this will make dropping TS-ESNode into existing projects much easier.

Cannot import commonJs file in ESM project

I have a project using ESM, "type": "module", all that jazz. I am using prisma and prisma-appsync which has a code generator that outputs a .js file that I need to bundle in my application.

I cannot figure out how to import a commonjs file in my ESM project. I've tried every form of import and require that I know exist.

I'm able to compile fine with TypeScript (with every combination of allowJS, esModuleInterop, etc), I just can't run my script with node --loader @k-foss/ts-esnode --experimental-specifier-resolution=node bin/appMish.ts

The main error I get is:

ReferenceError: exports is not defined in ES module scope

I also tried wrapping it in a commonjs package and importing that but then I get the "module not found at /foo/bar.js" error and couldn't figure that out either.

The automated release is failing 🚨

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

• Usage documentation
• Frequently Asked Questions
• Support channels

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

---

The release 1.4.0 on branch master cannot be published as it is out of range.

Based on the releases published on other branches, only versions within the range >=1.3.1 <1.3.2 can be published from branch master.

The following commits are responsible for the invalid release:

• feat: Force release (58f3d3b)
• release 1.3.6 (#31) (5cb58c4)

Those commits should be moved to a valid branch with git merge or git cherry-pick and removed from branch master with git revert or git reset.

A valid branch could be next.

See the workflow configuration documentation for more details.

---

Good luck with your project ✨

Your semantic-release bot 📦🚀

TypeScript Type Checking and Reporting

Currently this module does not type-check or report errors. The current ts.transpileModulefunction that is used to transpile a single source file at a time doesn't support inferring TypeScript type errors nor getting in types from external modules.

HMR/Watching

Provide a "dev" mode that will report type errors and also allow for hot swapping of module code upon file chaging.

Error ERR_LOADER_CHAIN_INCOMPLETE: short circuit detection

When trying to launch a script with ts-esnode (both latest and 1.7.0) on Node v16.17.0, I am getting the error
Error [ERR_LOADER_CHAIN_INCOMPLETE]: "@k-foss/ts-esnode 'resolve'" did not call the next hook in its chain and did not explicitly signal a short circuit. If this is intentional, include `shortCircuit: true` in the hook's return
I could find ts-node/issues/1714 (April 2022) for this error, which seems to imply that this error comes from new checks in new Node version.
It would be great if you could look into this and update the package to resolve this.
Thanks

Proper Caching for

Importing from '.' does not work

index.ts

export function log() {
	console.log(1);
}

hello.ts

import { log } from '.';
log();

Run command: node --loader '@k-foss/ts-esnode' --experimental-specifier-resolution=node ./hello.ts

Error: Cannot find module /Users/marcel/ imported from /Users/marcel/hello.ts

The automated release is failing 🚨

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

• Usage documentation
• Frequently Asked Questions
• Support channels

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

---

Invalid npm token.

The npm token configured in the NPM_TOKEN environment variable must be a valid token allowing to publish to the registry https://registry.npmjs.org/.

If you are using Two-Factor Authentication, make configure the auth-only level is supported. semantic-release cannot publish with the default auth-and-writes level.

Please make sure to set the NPM_TOKEN environment variable in your CI with the exact value of the npm token.

---

Good luck with your project ✨

Your semantic-release bot 📦🚀

`TS_PROJECT` Environment Variable Support

TS_PROJECT environment variable to override the find TSConfig function. If the TS_PROJECT variable is set, then use that as forced path to load tsconfig.json

Import of modules not working on windows: Path not formatted correctly

I noticed the function dynamicInstantiate(url) splits the path and creates a require function. The problem on windows is that the URL starts with 'file:///c:/...' your function converts this into '/c:/...'.
It works if the path starts with 'c:/...' (no slash in front).

Could you please fix this? I think the correction should only be done if process.platform === 'win32'.

Github Actions

Setup GitHub actions for automated CD to NPM and GitHub package registry.

Yarn2 support

Hey there, this is a neat project and could hopefully someday do away with needing ts-node for everything.

I want to open this as more of a TO-DO for when I or someone else finds time. Right now I'm using yarn2, which uses a virtual filesystem and doesn't have a node_modules folder. All packages are instead stored in .zip files and extracted when needed.

A few takeaways from my hour trying to get this to work with yarn2.0.0-rc.29 and node 13.9:

• It looks like this package depends on there being a node_modules folder, so that'd need to be dealt with

• Yarn2's virtual filesystem is accessible via a few different routes but the simplest is by starting node as node --require ./.pnp.js from the project root. Delving into exactly what happens when .pnp.js is hooked like this is a big TODO, but I believe it monkeypatches the fs module to automatically unzip packages whenever they're imported / required.

  The catch22 with using yarn2 + this neat package is:

  • When package.json has "type": "module", require is not defined at startup. Since the CLI flag --require ... seems to literally use require(...), this fails with an error.
  • Without --require ./.pnp.js, none of the workspace's packages are available, meaning --loader=@k-foss/ts-esnode results in Cannot find package '@k-foss/ts-esnode ...

So maybe sometime later this month I'll have some bandwidth to tackle this. But compliments again for finding a way to leverage these new node features to potentially make typescript less painful.

Caching

Right now the performance of the module could/can be greatly speed up if a few of the resource intensive function cache their response.

Worker threads

I would like to handle the transpiring/checking of the TypeScript code in Worker Threads I've seen the transpile of large files/projects bring the entire event loop to it's knees and stop serving/responding to web traffic.

TSConfig Paths: error with allowJs

Environment: Node 14.3.0 on Windows 10 version 2004

package.json

{
  "private": true,
  "type": "module",
  "scripts": {
    "build": "tsc --noEmit",
    "dev": "node --loader @k-foss/ts-esnode --experimental-specifier-resolution=node proj/main.ts"
  },
  "devDependencies": {
    "@k-foss/ts-esnode": "^1.5.0",
    "typescript": "^3.9.3"
  }
}

tsconfig.json

{
  "compilerOptions": {
    "allowJs": true,
    "module": "ESNext",
    "target": "ESNext",
    "moduleResolution": "Node",
    "baseUrl": ".",
    "paths": {
      "hello": ["./demo.js"]
    }
  }
}

proj/main.ts

import { f } from "hello";
console.log(f());

demo.js

export function f() { return 1; }

npm run build completes successfully, which means that the TypeScript is valid.

npm run dev fails with the following error:

(node:16144) ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
internal/modules/run_main.js:54
    internalBinding('errors').triggerUncaughtException(
                              ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/C:/Users/sunny/Documents/code/tsesnodebug/proj/demo.js' imported from C:\Users\sunny\Documents\code\tsesnodebug\proj\main.ts
Did you mean to import ../../demo.js?
    at finalizeResolution (internal/modules/esm/resolve.js:269:11)
    at moduleResolve (internal/modules/esm/resolve.js:658:10)
    at defaultResolve (internal/modules/esm/resolve.js:748:11)
    at resolve (file:///C:/Users/sunny/Documents/code/tsesnodebug/node_modules/@k-foss/ts-esnode/out/dist/index.js:56:12) 
    at Loader.resolve (internal/modules/esm/loader.js:97:40)
    at Loader.getModuleJob (internal/modules/esm/loader.js:243:28)
    at ModuleWrap.<anonymous> (internal/modules/esm/module_job.js:42:40)
    at link (internal/modules/esm/module_job.js:41:36) {
  code: 'ERR_MODULE_NOT_FOUND'
}