Giter Site home page Giter Site logo

nktnet1 / import-sync Goto Github PK

View Code? Open in Web Editor NEW
14.0 1.0 2.0 1.31 MB

Synchronously and dynamically import ES6 modules in NodeJS

Home Page: https://www.npmjs.com/package/import-sync

License: MIT License

TypeScript 97.15% Shell 2.85%
cjs commonjs dynamic es6 escmascript2015 esm export import module nodejs

import-sync's Introduction

Import Sync

pipeline   codecov   Maintainability   Snyk Security   GitHub top language

NPM Version   install size   Depfu Dependencies   FOSSA Status   NPM License   GitHub issues

Quality Gate Status   Codacy Badge   DeepSource   codebeat badge   GitHub stars

Downloads Total   Downloads Yearly   Downloads Monthly   Downloads Weekly   Downloads Daily


Synchronously import dynamic ECMAScript Modules similar to CommonJS require

Basic wrapper around esm for compatibility with both ESM and CJS projects in NodeJS

Capable of importing ESM-only libraries such as node-fetch@3 in CJS projects

Try with Replit


1. Installation

npm install import-sync

2. Usage

Try with Replit.

importSync(id, options);
Examples (click to view)

Importing from the same directory

const { someVariable, someFunction } = importSync('./some-module');

Importing .mjs file from a different directory

const { someFunction  } = importSync('../src/someModule.mjs');

Using a different basePath

const { someFunction } = importSync(
  './someModule',
  { basePath: process.cwd() }
);

Using additional esm options as described in esm's documentation

const { someFunction } = importSync(
  './someModule',
  {
    esmOptions: {
      cjs: {
        cache: true
      },
      mode: 'all',
      force: 'true',
    }
  }
);

Importing an ESM-only module

const fetch = importSync('node-fetch'),

2.1. id

Module name or relative path similar to CommonJS require. For example,

  • '../animals/cats.js'
  • './dogs.mjs'
  • './minimal'
    • importSync will look for matching extensions in the order [.js, .mjs, .cjs, .ts]
  • 'node-fetch'
    • importSync can import pure-esm node-fetch (v3) into your cjs project

2.2. options

Option Description Example Default
basePath This will only take effect if the given id starts with ./ or ../.
For example,
  • ./localLib
  • ../src/localLib
and not
  • /home/user/localLib
  • localLib.mjs
  • node-fetch
The ❌ examples above will be interpreted as either absolute paths or library imports.
./myModule
__dirname
esmOptions Options for the esm module as described in esm's documentation.
{
  cjs: true,
  mode: 'auto'
}
undefined

2.3. return

The importSync function returns the exported module content similar to NodeJS require.

If an unknown file path is provided a default Error object is thrown.

3. License

Massachusetts Institute of Technology (MIT)
Copyright (c) 2023 Khiet Tam Nguyen

Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the “Software”),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.

FOSSA Status

4. Limitations

There are currently no known limitations.

5. Caveats

5.1. Idea

import-sync was created to enable the implementation of a global dryrun script that can be run by students undertaking COMP1531 Software Engineering Fundamentals in their major group project. This requires the ability to import external ES Modules from any directory or path for use in both CommonJS and ESM-based projects.

The dryrun serves as a sanity check before the final submission is made, and is located in the centralised COMP1531 course account at the path ~cs1531/bin. Students who are connected to the CSE lab environment (e.g. via VLAB) can run the dryrun script from their major project repository, e.g. at the path ~z5313514/comp1531/project-backend.

5.2. Discovery

Initially, the esm library looked promising. However, when the global dryrun script was executed in a mock student's project directory, the following error occurred:

Error [ERR_REQUIRE_ESM]: require() of ES Module /import/ravel/5/z5313515/project-backend/src/auth.js not supported.
Instead change the require of auth.js in null to a dynamic import() which is available in all CommonJS modules

This is due to the package.json containing "type": "module", as iteration 1 of the student major project uses ESM for the seamless transition to future iterations.

The following approaches were thus attempted, but were unsatisfactory for our purpose:

  1. jewire/rewire/require
    • in iteration 1, the dryrun requires the import of ES6 modules, so jewire (which was used for the dryrun of iteration 0) was no longer satisfying our requirements
    • the same limitations of being CommonJS exclusive applies to rewire and require
  2. import() - ECMAScript dynamic import
    • this was the previous attempt at writing the dryrun
    • However, it relied on asynchronous code. Since COMP1531 is fully synchronous (including the use of sync-request-curl for sending HTTP requests), this became a source of mystery and confusion for students
    • additionally, students had to append the suffix .js to all of their file imports in the project solely to use the dryrun. This resulted in ambiguous error messages and obscure dryrun requirements unrelated to the project
  3. require-esm-in-cjs
    • this library utilises deasync, which when used in NodeJS for Jest tests, could hang indefinitely as seen in Jest's issue #9729
    • since COMP1531 uses Jest as the sole testing framework, deasync was ruled out
  4. Other async-to-sync conversions for dynamic import()
    • synckit: worker_threads, Jest and external imports did not work (unclear reason)
    • sync-rpc: leaves orphan processes when used in Jest as explained in issue #10
    • fibers: obsolete and does not work for node versions later than 16
    • synchronize: documentation link gives 404 and has fiber as a dependency
    • sync/node-sync: uses fiber (note: "redblaze/node-sync" on github, "sync" on npm)

5.3. Result

Upon a more thorough investigation into the initial issue with the esm module, the cause was the introduction of the exception starting from NodeJS version 13, as noted in @fregante's comment:

Further down the thread was a link to the solution by @guybedford

which removes the exception through module extension and serves as a satisfactory workaround. This reduced the codebase of import-sync to simply a wrapper around esm.

Another issue that import-sync (v2) addresses is esm's open issue #904, which yields the error message:

Error [ERR_INVALID_PROTOCOL]: Protocol 'node:' not supported. Expected 'file:'

when importing ESM-only libraries such as node-fetch@3 in a CommonJS module. This is done by overriding the default Module._resolveFilename function to remove the node: prefix, effectively changing any imports of the form (for example):

import http from 'node:http';

to

import http from 'http';

for all imported modules.

For further discussions about this issue, visit:

import-sync's People

Contributors

depfu[bot] avatar mergify[bot] avatar nktnet avatar nktnet1 avatar pimterry avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

import-sync's Issues

Can't import some modules

One example: node-datachannel.

Create a new folder, and run:

$ npm install import-sync node-datachannel
$ node
> importSync = require('import-sync')
> importSync('node-datachannel')
Uncaught Error: 
        Failed to import from:
            node-datachannel.
        Options:
            {}
        Require error message:
            /tmp/tmp.cGqoObetIA/node_modules/import-sync/dist/cjs/import.js:1
Error: Cannot find module 'node-datachannel'
Require stack:
- /tmp/tmp.cGqoObetIA/node_modules/import-sync/dist/cjs/import.js
- /tmp/tmp.cGqoObetIA/node_modules/import-sync/dist/cjs/index.js
- <repl>
    at esmImport (/tmp/tmp.cGqoObetIA/node_modules/import-sync/dist/cjs/import.js:47:16)
    at importSync (/tmp/tmp.cGqoObetIA/node_modules/import-sync/dist/cjs/import.js:72:28)
    at REPL4:1
    at Script.runInThisContext (node:vm:122:12)
    at REPLServer.defaultEval (node:repl:594:29)
    
    at esmImport (/tmp/tmp.cGqoObetIA/node_modules/import-sync/dist/cjs/import.js:50:15)
    at importSync (/tmp/tmp.cGqoObetIA/node_modules/import-sync/dist/cjs/import.js:72:28)

> await import('node-datachannel')
[Module: null prototype] {
  Audio: [Function: Audio],
  DataChannel: [Function: DataChannel],
  ... // I.e. it imports with ESM just fine
}

Happens with Node 20.8.0 and 21.4.0 (latest). Any idea what's going on here? Node-fetch does indeed import as expected, but this module isn't even found for some reason.

process is not defined

Hello,

First of all thanks for taking the time to build/share this package.

I'm running this on an old project that has a foundation built around 'sync requires' and this package is perfect if I can get it to work but I'm getting:

Uncaught ReferenceError: process is not defined
at node_modules/esm/esm.js (esm.js:1:186)
at __require2 (chunk-AUZ3RYOM.js?v=2899e366:18:50)
at import.ts:1:17

The error is trigger upon importing:
import importSync from "import-sync";

Note that I have to install node-modules with --legacy-peer-deps because it's an old neglected mess so not sure if a peer dependency might be breaking this but I don't think so.. I've check that the esm module installed is the latest.

Let me know if you need more details from me, thanks!

Unable to tic in CDK project

I have an AWS CDK project which fails to tsc because of this:

(venv) rodrigocarvajal@rodrigos-MBP BotFactory % tsc
node_modules/import-sync/dist/cjs/types.d.ts:1:22 - error TS6053: File '/Users/rodrigocarvajal/Documents/projects/BotFactory/node_modules/import-sync/src/httptoolkit-esm-types.d.ts' not found.

1 /// <reference path="../../src/httptoolkit-esm-types.d.ts" />
                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

node_modules/import-sync/dist/cjs/types.d.ts:2:17 - error TS7016: Could not find a declaration file for module '@httptoolkit/esm'. '/Users/rodrigocarvajal/Documents/projects/BotFactory/node_modules/@httptoolkit/esm/esm.js' implicitly has an 'any' type.
  Try `npm i --save-dev @types/httptoolkit__esm` if it exists or add a new declaration (.d.ts) file containing `declare module '@httptoolkit/esm';`

2 import esm from '@httptoolkit/esm';
                  ~~~~~~~~~~~~~~~~~~


Found 2 errors in the same file, starting at: node_modules/import-sync/dist/cjs/types.d.ts:1

(venv) rodrigocarvajal@rodrigos-MBP BotFactory % 

My tsconfig.json:

{
    "compilerOptions": {
        "target": "ES2020",
        "module": "commonjs",
        "lib": [
            "es2020",
            "dom"
        ],
        "declaration": true,
        "strict": true,
        "noImplicitAny": true,
        "strictNullChecks": true,
        "noImplicitThis": true,
        "alwaysStrict": true,
        "noUnusedLocals": false,
        "noUnusedParameters": false,
        "noImplicitReturns": true,
        "noFallthroughCasesInSwitch": false,
        "inlineSourceMap": true,
        "inlineSources": true,
        "experimentalDecorators": true,
        "strictPropertyInitialization": false,
        "typeRoots": [
            "./node_modules/@types"
        ]
    },
    "exclude": [
        "node_modules",
        "cdk.out"
    ]
}

My package.json:

{
    "name": "bot-factory",
    "version": "0.1.0",
    "bin": {
        "bot-factory": "bin/bot-factory.js"
    },
    "scripts": {
        "build": "tsc",
        "watch": "tsc -w",
        "test": "cdk synth && npm run build && jest",
        "cdk": "cdk",
        "lint": "eslint --ext .ts lib/ --fix --ignore-pattern \"*.d.ts\" && eslint --ext .ts bin/ --fix --ignore-pattern \"*.d.ts\" && eslint --ext .ts test/ --fix --ignore-pattern \"*.d.ts\""
    },
    "devDependencies": {
        "@types/jest": "^29.5.5",
        "@types/node": "20.7.1",
        "@typescript-eslint/eslint-plugin": "^6.17.0",
        "@typescript-eslint/parser": "^6.17.0",
        "aws-cdk": "^2.132.0",
        "eslint": "^8.56.0",
        "eslint-config-airbnb-base": "^15.0.0",
        "eslint-plugin-import": "^2.29.1",
        "jest": "^29.7.0",
        "ts-jest": "^29.1.1",
        "ts-node": "^10.9.1",
        "typescript": "~5.2.2"
    },
    "dependencies": {
        "@aws-cdk/aws-lambda-python-alpha": "^2.132.0-alpha.0",
        "@httptoolkit/esm": "^3.3.0",
        "aws-cdk-lib": "^2.132.0",
        "constructs": "^10.0.0",
        "source-map-support": "^0.5.21",
        "import-sync": "^2.2.0"
    }
}

In my code I am simply doing:

        const plug = importSync(props.processingStackClassFile);
        const constructorName = Object.keys(plug)[0];

And it works, I can even do a cdk synth and deploy, however tsc fails. Am I doing something wrong here?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.