Giter Site home page Giter Site logo

prebuild-install's Introduction

prebuild

A command line tool for easily making prebuilt binaries for multiple versions of Node.js, Node-API, Electron and NW.js on a specific platform.

$ npm install -g prebuild

npm Node version js-standard-style

Features

  • Builds native modules for any version of Node.js, Node-API, Electron or NW.js, without having to switch between different versions to do so. This works by only downloading the correct headers and telling node-gyp to use those instead of the ones installed on your system.
  • Upload (--upload) prebuilt binaries to GitHub.
  • Support for stripping (--strip) debug information. Strip command defaults to strip but can be overridden by the STRIP environment variable.
  • Install prebuilt modules via prebuild-install.

Building

Building is only required for targets with different ABI versions. To build for all supported ABI versions (example from leveldown):

prebuild --all

Supported ABI versions may change over time without a new prebuild release.

Alternatively, to build for some specific versions you can do:

prebuild -t 0.10.42 -t 0.12.10 -t 4.3.0

To build for Node-API, do:

prebuild -t 3 -r napi

To build against Electron headers, do:

prebuild -t 1.4.10 -r electron

To build against NW.js headers, do:

prebuild -t 0.26.6 -r node-webkit

See allTargets for all available versions.

For more options run prebuild --help. The prebuilds created are compatible with node-pre-gyp

If you'd like to include other files with your prebuilds like additional .node files or other native libraries, you can pass a file-matching regular expression to --include-regex:

prebuild -t 8.0.0 --include-regex "\.(node|a)$"

Note that if you include multiple .node files, you will need to use the prebuild-install's --binary-name parameter to indicate which file should be loaded:

prebuild-install --binary-name main-binary.node

The build file format is selected automatically by node-gyp, however it is possible to specify needed format explicitly with --format parameter. This is particularly useful if unusual flavor is required, which could be specified in 'format-flavor' form (there is no comprehensive list of formats/flavors available so one has to find possible combinations from node-gyp source code). For example, in order to build using Makefiles but assume Android cross-compilation:

prebuild --format make-android

When using the cmake-js backend additional parameters can be passed through.

prebuild --backend cmake-js -- --prefer-clang --CDUV_INCLUDE_DIR=...

Scripts

A prepack script can be specified that is executed once the .node module has been created but before it is compressed and moved. This can be used to perform code signing.

prebuild --prepack 'codesign -v -s MyCompany'

The --preinstall or --prepack parameters can take either a shell command or JS file to be executed.

Uploading

prebuild supports uploading prebuilds to GitHub releases. If the release doesn't exist, it will be created for you. To upload prebuilds simply add the -u <github-token> option:

$ prebuild --all -u <github-token>

If you don't want to use the token on cli you can put it in ~/.prebuildrc:

upload=<github-token>

Note that --upload will only upload the targets that was built and stored in ./prebuilds, so prebuild -u <github-token> -t 4.3.0 will only upload the binary for the 4.3.0 target.

You can use prebuild --upload-all to upload all files from the ./prebuilds folder.

You can use prebuild --upload --tag-prefix <prefix> for specific tag prefixes for the release. The default prefix is v and will result in a tag with an appended version number, for example v1.0.0. For lerna you can use the package name e.g. prebuild --tag-prefix some-package@ and the binaries will be released on the appropriate package's tags, for example [email protected].

You can use prebuild --upload --prerelease to create a prerelease, which will not be shown as the latest release.

Create GitHub Token

A GitHub token is needed for two reasons:

  • Create a GitHub release (leveldown example)
  • Upload the prebuilt binaries to that release

To create a token:

  • Go to this page
  • Click the Generate new token button
  • Give the token a name and click the Generate token button, see below

prebuild-token

The default scopes should be fine.

Node-API Considerations

Declaring Supported Node-API Versions

Native modules that are designed to work with Node-API, which was previously known as N-API, must explicitly declare the Node-API version(s) against which they can build. This is accomplished by including a binary property on the module's package.json file. For example:

"binary": {
  "napi_versions": [2,3]
}

In the absence of a need to compile against a specific Node-API version, the value 3 is a good choice as this is the Node-API version that was supported when Node-API left experimental status.

Modules that are built against a specific Node-API version will continue to operate indefinitely, even as later versions of Node-API are introduced.

Defining the NAPI_VERSION Value

The Node-API header files supplied with Node use the NAPI_VERSION preprocessor value supplied by the user to configure each build to the specific Node-API version for which the native addon is being built. In addition, the module's C/C++ code can use this value to conditionally compile code based on the Node-API version it is being compiled against.

prebuild supports two build backends: node-gyp and cmake-js. The NAPI_VERSION value is configured differently for each backend.

node-gyp

The following code must be included in the binding.gyp file of modules targeting Node-API:

"defines": [
  "NAPI_VERSION=<(napi_build_version)",
]

cmake-js

The following line must be included in the CMakeLists.txt file of modules targeting Node-API:

add_compile_definitions(NAPI_VERSION=${napi_build_version})

prebuild arguments

The --runtime argument must be napi to request Node-API builds. When requesting Node-API builds, the module's package.json file must include a binary property as described above. And the binding.gyp file must include a define for NAPI_VERSION as described above.

One or more --target arguments may be specified to request builds for specific Node-API versions. Node-API versions are positive integer values. Alternatively, --all may be used to request builds for all Node-API versions supported by the module.

In the absence of both --target and --all arguments, prebuild will build the most current version of the module supported by the Node instance performing the build.

Help

$ prebuild -h
prebuild [options]

  --target      -t  version     (version to build or install for)
  --runtime     -r  runtime     (Node runtime [node, napi, electron or node-webkit] to build or install for, default is node)
  --arch        -a  arch        (architecture to build or install for [default: process.arch])
  --all                         (prebuild for all known abi versions)
  --upload      -u  [gh-token]  (upload prebuilds to github)
  --upload-all  -u  [gh-token]  (upload all files from ./prebuilds folder to github)
  --tag-prefix <prefix>         (github tag prefix, default is "v")
  --preinstall  -i  script      (run this script before prebuilding)
  --prepack     -c  script      (run this script before packing, can be used to codesign)
  --path        -p  path        (make a prebuild here)
  --include-regex               (regex to match files that will be distributed [default: '\.node$'])
  --libc                        (use provided libc rather than system default)
  --backend                     (specify build backend, default is 'node-gyp')
  --format                      (specify additional parameters for `node-gyp` backend)
  --strip                       (strip debug information)
  --debug                       (set Debug or Release configuration)
  --verbose                     (log verbosely)
  --version                     (print prebuild version and exit)

JavaScript API

var prebuild = require('prebuild')

.build(opts, version, cb)

Options:

  • .log (optional)
  • .preinstall (optional)
  • .gyp Provide a custom node-gyp instance (optional)
  • .backend Provide a custom node-gyp instance via string. Alternatives are 'node-gyp', 'node-ninja', 'nw-gyp' and 'cmake-js' (optional, defaults to 'node-gyp')
  • .args Additional command line arguments to node-gyp (optional)
  • .debug Pass in --debug on command line to gyp backend (optional)

Example:

prebuild.build({}, version, function (err) {
  // ...
})

Global options:

  • .debug Download or build a debug build (default: false)
  • .arch Processor architecture (default: process.arch)

Develop prebuild

If you want to hack on prebuild you need an environment to play around with. We recommend a setup similar to the following:

  • A fork of prebuild
  • A GitHub token (see above)
  • A native node module
$ git clone [email protected]:<your-nick>/prebuild
$ cd prebuild && npm link && cd ..
$ git clone [email protected]:<your-nick>/some-native-module

Since you did npm link on prebuild it will be installed globally. Now you can go ahead and try things out.

$ cd some-native-module
$ prebuild --all --strip -u <github-token>

This command would:

  • Build some-native-module for all supported targets and store them in ./prebuilds/
  • Strip binaries from debug information
  • Create a release on GitHub, if needed
  • Upload all binaries to that release, if not already uploaded

Before you commit your changes and send us a pull request, do run npm test.

License

MIT

prebuild-install's People

Contributors

akajes avatar avaly avatar baizulin avatar csett86 avatar develar avatar dunkmann00 avatar fishbone1 avatar ian-axiadids avatar iwestlin avatar jchook avatar joaomoreno avatar joonamo avatar jschlight avatar kasalehlia avatar lgeiger avatar lovell avatar mathiask88 avatar mberka avatar nicolai-nordic avatar pfitzseb avatar piranna avatar ralphtheninja avatar saboya avatar sanghyun-park avatar szubster avatar tom-boothman avatar tux-tn avatar victorleach96 avatar vweevers avatar zkochan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

prebuild-install's Issues

--tag-prefix not working

In commit 53f5225 prebuild-install learned a new command line option --tag-prefix, but it seems that using this option does not change the download url.

Proxy support

Isn't it enough to give simple-get an option object with the proxy as the host and the url as path?

Suggested `install` script breaks on Windows

In README.md it is suggested to set the package.json install script to:

...
  "scripts": {
    "install": "prebuild-install || node-gyp rebuild"
  }
...

The || part is however not portable to Windows, so packages that need to be installable on Windows will need an install script written in JavaScript. (Unless I'm missing something obvious, which is likely.)

Restore local prebuild option

The local prebuild feature appears to have been removed in version 5, specifically in this commit: 0593bf5

I'm bundling an NPM package that needs to be installable offline from a TGZ, since some users have firewall restrictions. One of the dependencies I'm bundling is keytar, which uses prebuild-install to install its native code.

With the local prebuild feature, I can easily make my package installable offline by adding the compiled native code for each platform to a "prebuilds" folder in my TGZ. With this feature removed in v5, it's a lot harder to do this as I need to add a preinstall script that copies the native code into the NPM cache, to fake out downloading it.

Since local prebuild seems like a useful feature for making packages installable offline, I'm not sure why it was considered unnecessary and removed. Please consider restoring this feature.

Tests should run on OSX and Windows

Since prebuild-install is meant for installing prebuilts for all platforms, we should really run the tests on all platforms as well and not only on linux.

  • osx
  • windows

Does this work with yarn?

I noticed that standalone mode is determined by checking the _from field in the modules package.json, which is only created by npm clients and not yarn. Is there any better way for determining if a module was installed as a dependency?

Check remote prebuild before using cache

Check if the published prebuild is newer in date than our local cache and invalidate it. This is useful when developing in local, since the local one could be corrupted.

Add log entry when proxy is specified

See

if (proxy) {
var parsedDownloadUrl = url.parse(downloadUrl)
var parsedProxy = url.parse(proxy)
var uriProtocol = (parsedDownloadUrl.protocol === 'https:' ? 'https' : 'http')
var proxyProtocol = (parsedProxy.protocol === 'https:' ? 'Https' : 'Http')
var tunnelFnName = [uriProtocol, proxyProtocol].join('Over')
reqOpts.agent = tunnel[tunnelFnName]({
proxy: {
host: parsedProxy.hostname,
port: +parsedProxy.port,
proxyAuth: parsedProxy.auth
}
})
}

Add options to set location for CA file

When setting download location with --download option, https URLs with self-signed certificates can not be accessed failing with the following error.

 prebuild-install WARN install unable to verify the first certificate

The code seems not to respect NPM configuration for cafile and does not provide its own option too.

Prepare for npm change to _ fields

npm will soon (TBD) no longer store _ fields in package.json files in node_modules.

This is a problem for us:

prebuild-install/bin.js

Lines 53 to 59 in 652f169

} else if (!(typeof pkg._from === 'string')) {
log.info('install', 'installing standalone, skipping download.')
process.exit(1)
} else if (pkg._from.length > 4 && pkg._from.substr(0, 4) === 'git+') {
log.info('install', 'installing from git repository, skipping download.')
process.exit(1)
} else if (opts.compile === true || opts.prebuild === false) {

v5.3.1 breaks compatibility with Node 4.x

Updating major versions for simple-get and tar-fs here breaks compatibility with Node 4.x.

I realize that Node v4 support was dropped in v4.0.0 of this module. However things actually kept working until v5.3.1. This is an issue for e.g. node-snappy which still supports Node 4.x.

Pinning to 5.3.0 fixes the issue for node-snappy, but I think it would be nice to continue supporting Node 4.x while it's not too much of an inconvenience, or introduce a new major version to indicate that breaking changes were introduced via dependency update.

Pinning simple-get to 2.x and tar-fs to 1.x would allow master to continue working in Node 4.x.

Incorrectly unpack tar.gz downloaded from bitbucket

When prebuild-install download archive (via simple-get) it adds header Accept-Encoding: gzip, deflate.
Bitbucket returns tar.gz files as is (because they are already gzipped), but with the content type application/x-tar.
So, if response content type is application/x-tar and content-encoding is gzip you shouldn't unzip request body, but save it as is with tar.gz extension.
Currently you unzip request body and save plain tar content as tar.gz that is leads to errors.

npm error at require / disable require

Hi guys,

thanks for this awesome lib. I'm trying to add prebuild feature to node-rdkafka lib, but I have problem when I try to install it from prebuild. Exactly when prebuild-install require the .node file.
But the require doesn't throw error, just npm.

My question: Can I disable somehow the require step at the end of installation?

Node: 8.11.0
OS: Win7 x64

Command: npm i icebob/node-rdkafka

Error:

> @icebob/[email protected] install d:\Work\tmp\rdkafka-test\node_modules\@icebob\node-rdkafka     
> prebuild-install --verbose                                                                        
                                                                                                    
prebuild-install info begin Prebuild-install version 2.5.3                                          
prebuild-install info looking for local prebuild @ prebuilds\node-rdkafka-v2.3.2-node-v57-win32-x64.
tar.gz                                                                                              
prebuild-install info looking for cached prebuild @ C:\Users\Bobcsi\AppData\Roaming\npm-cache\_prebu
ilds\https-github.com-icebob-node-rdkafka-releases-download-v2.3.2-node-rdkafka-v2.3.2-node-v57-win3
2-x64.tar.gz                                                                                        
prebuild-install info found cached prebuild                                                         
prebuild-install info unpacking @ C:\Users\Bobcsi\AppData\Roaming\npm-cache\_prebuilds\https-github.
com-icebob-node-rdkafka-releases-download-v2.3.2-node-rdkafka-v2.3.2-node-v57-win32-x64.tar.gz      
prebuild-install info unpack resolved to d:\Work\tmp\rdkafka-test\node_modules\@icebob\node-rdkafka\
build\Release\node-librdkafka.node                                                                  
prebuild-install info unpack required d:\Work\tmp\rdkafka-test\node_modules\@icebob\node-rdkafka\bui
ld\Release\node-librdkafka.node successfully                                                        
prebuild-install info install Successfully installed prebuilt binary!                               
npm WARN [email protected] No description                                                          
npm WARN [email protected] No repository field.                                                    
                                                                                                    
npm ERR! code ELIFECYCLE                                                                            
npm ERR! errno 3221225477                                                                           
npm ERR! @icebob/[email protected] install: `prebuild-install --verbose`                           
npm ERR! Exit status 3221225477                                                                     
npm ERR!                                                                                            
npm ERR! Failed at the @icebob/[email protected] install script.                                   
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.  
                                                                                                    
npm ERR! A complete log of this run can be found in:                                                
npm ERR!     C:\Users\Bobcsi\AppData\Roaming\npm-cache\_logs\2018-04-25T20_49_58_714Z-debug.log     

I hacked prebuild-install to skip the require and it works without error. But I don't know, what is the reason of error.

Thanks!

Prebuild doesn't found the specified module and adds "\\?\" to the path

Hello,

When trying to add ZeroMQ module to my Node.js project, I receive:

The specified module could not be found
\\?\C:\test\node_modules\zeromq\build\Release\zmq.node

The file exists at the path and the issue is occuring when "prebuild-install" tries to require it

download.js Line 134:
require(resolved)

it seems that it prepends "\\?\" to the path. Is it normal?

tests not working on node 8

Just noticed that the tests doesn't pass on node 8 (I was using v8.1.3)

  • Figure out the cause
  • Update .travis.yml for node 8

Make tests windows friendlier

The tests are currently not working on windows platforms.

  • We need windows builds in a-native-module (I can build and upload them)
  • Make some tests language/platform aware e.g. This

Error: unregistered users are not allowed to access package @lgeiger/prebuild-install : @lgeiger/prebuild-install

Hi,

when trying to install zeromq in windows, I am getting the below error.
Below is the error log.

npm ERR! Windows_NT 10.0.10586
npm ERR! argv "C:\program files\nodejs\node.exe" "C:\program files\nodejs\node_modules\npm\bin\npm-cli.js" "install" "zeromq" "-g"
npm ERR! node v4.4.7
npm ERR! npm v2.15.8
npm ERR! code E403

npm ERR! unregistered users are not allowed to access package @lgeiger/prebuild-install : @lgeiger/prebuild-install
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR! https://github.com/npm/npm/issues

Please let us know how to resolve the issue

How to fail completely via flags if the remote isn't available

I'd like to test if the download works. With node-pre-gyp I was able to install the package like so and ensure it downloaded and not built.

npm install serialport@beta --fallback-to-build=false

I use this in CI to ensure the state of all the builds before publishing the npm package. It's also been able to catch npm bugs on different platforms. It has been very helpful.

Since my package.json install line now reads

    "install": "prebuild-install || node-gyp rebuild",

I need to convince prebuild-install to not download and not error, or convince node-gyp to not build if it gets run.

Is there a known way to achieve this?

Thanks!

Environmental variable options not passed through npm

I am working on a wrapper for npm where I want to be able to run npm with either a command-line flag or environmental variable to tell prebuild-install to download prebuilds from a URL other than GitHub. I see that prebuild-install uses rc and a download config option to set the download URL. Setting the environmental variable prebuild-install_download to my custom URL template works when running prebuild-install from the command-line - however, it does not work when it is run via an npm script (npm run install). From some debugging, I found that npm seems to not pass through env vars with a hyphen, and since prebuild-installs rc prefix has a hyphen, none of its env vars get passed through. Setting download=... in ~/.prebuild-installrc works, but I would prefer not to touch the file system.

Would a PR be welcomed here that added a way to pass through the download option from npm's cli args or env vars, in a non-breaking way? I am thinking the simplest way would be to use an env var without a hypen, like prebuildinstall_download, as a default for the download config option in rc.js. Or is there a better alternative that I am missing?

buildDependencies?

By using prebuild-install I've found mostly is the only one dependency I need, except when it can resolve to a prebuild image and needs to be compiled, for example nan and similar ones that are useless for prebuild images. I could set them as devDependencies, but in that case they would not be available if the prebuild is not available, so that dependencies are in a no-man-land status.

For that situation, I think a solution would be a buildDependencies where to set the dependencies needed only when the prebuild image could not be fetch (because it's not available or due to download failure), that would be installed by prebuild-install itself. This could be done being that buildDependencies an independent field object with no relationship with the other dependencies, or it could be an array with the name of the devDependencies that are in fact build ones, so this way it could be integrated with current workflows and being compatible with tools like node-check-updates and similar ones.

Thoughts?

Upgrade mkdirp to avoid warnings during install

Seeing the following warnings during install of an upstream lib:

warning keytar > prebuild-install > [email protected]: Legacy versions of mkdirp are no longer supported. Please update to mkdirp 1.x. (Note that the API surface has changed to use Promises in 1.x.)
warning keytar > prebuild-install > tar-fs > [email protected]: Legacy versions of mkdirp are no longer supported. Please update to mkdirp 1.x. (Note that the API surface has changed to use Promises in 1.x.)

Automatically install devDependencies on failure

If a prebuild file is not found or the download fails, automatically install the module devDependencies so the fallback command can be correctly executed. This way it's not required to install the dependencies needed to build the module when exists a good prebuild candidate.

[regression] modules are always compiling

After inclusion of #8 and #12, now I'm getting the modules to always get compiling and not downloading. I could see the point of force the compiling on development environment when prebuild-install is a direct dependency of the project where npm install is executed (but I disagree on this and I think --no-prebuild flag should be used instead), the problem is when using a dependency that makes uses of prebuild-install, since that dependency will gets compiled too, making prebuild-install mostly useless.

I see two solutions here:

  1. revert #12 and force users to use --no-prebuild
  2. check we are on the root project and only in that case ignore the prebuilds, but still use them in other case

I would do both because sometimes I want to test locally that the prebuild images works correctly before publishing the module to npm, so I would revert #12 and make --no-prebuild and opt-in, and also do the check because the flag would be applied to all the executions also on internal dependencies, that's what we are not interested about.

As a side note, since #12 changed the current behaviour of prebuild-install it should have been a major version, by the way. I think we could republish the module as-is with version 2.0.0 and remove 1.1.0 from npm so people can stick with 1.0.2, but I think this feature should have more discussion before adding it back...

Replace require() test with non-lazy dlopen() test

A thought. Currently, post-download, we do this test:

try {
require(resolved)
} catch (err) {
return cb(err)
}

As I understand it, this resolves symbols lazily and may thus skip potentially conflicting or missing symbols. Could we maybe do the following test instead? Would that have merit?

const constants = require('os').constants.dlopen
process.dlopen(module, 'foo.node', constants.RTLD_NOW)

@lovell

reduce dependencies

Hey, I love this effort but maybe we could replace npmlog with something lighter?

    ├─┬ [email protected]
    │ ├─┬ [email protected]
    │ │ ├── [email protected]
    │ │ └─┬ [email protected]
    │ │   ├── [email protected]
    │ │   ├── [email protected] deduped
    │ │   ├── [email protected]
    │ │   ├── [email protected]
    │ │   ├── [email protected] deduped
    │ │   ├─┬ [email protected]
    │ │   │ └── [email protected] deduped
    │ │   └── [email protected]
    │ ├── [email protected]
    │ ├─┬ [email protected]
    │ │ ├── [email protected]
    │ │ ├── [email protected] deduped
    │ │ ├── [email protected]
    │ │ ├── [email protected]
    │ │ ├── [email protected]
    │ │ ├─┬ [email protected]
    │ │ │ ├── [email protected]
    │ │ │ ├─┬ [email protected]
    │ │ │ │ └── [email protected]
    │ │ │ └── [email protected] deduped
    │ │ ├─┬ [email protected]
    │ │ │ └── [email protected]
    │ │ └─┬ [email protected]
    │ │   └── [email protected] deduped
    │ └── [email protected]

And maybe we could improve it even further, I've just installed keytar and this is my npm ls (I only have one other dependency which is dep-free)

└─┬ [email protected]
  ├── [email protected]
  └─┬ [email protected]
    ├── [email protected]
    ├── [email protected]
    ├── [email protected]
    ├── [email protected]
    ├─┬ [email protected]
    │ └── [email protected] deduped
    ├── [email protected]
    ├─┬ [email protected]
    │ └── [email protected]
    ├── [email protected]
    ├─┬ [email protected]
    │ ├─┬ [email protected]
    │ │ ├── [email protected]
    │ │ └─┬ [email protected]
    │ │   ├── [email protected]
    │ │   ├── [email protected] deduped
    │ │   ├── [email protected]
    │ │   ├── [email protected]
    │ │   ├── [email protected] deduped
    │ │   ├─┬ [email protected]
    │ │   │ └── [email protected] deduped
    │ │   └── [email protected]
    │ ├── [email protected]
    │ ├─┬ [email protected]
    │ │ ├── [email protected]
    │ │ ├── [email protected] deduped
    │ │ ├── [email protected]
    │ │ ├── [email protected]
    │ │ ├── [email protected]
    │ │ ├─┬ [email protected]
    │ │ │ ├── [email protected]
    │ │ │ ├─┬ [email protected]
    │ │ │ │ └── [email protected]
    │ │ │ └── [email protected] deduped
    │ │ ├─┬ [email protected]
    │ │ │ └── [email protected]
    │ │ └─┬ [email protected]
    │ │   └── [email protected] deduped
    │ └── [email protected]
    ├─┬ [email protected]
    │ ├─┬ [email protected]
    │ │ └── [email protected] deduped
    │ └─┬ [email protected]
    │   └── [email protected]
    ├─┬ [email protected]
    │ ├── [email protected]
    │ ├── [email protected]
    │ ├── [email protected] deduped
    │ └── [email protected]
    ├─┬ [email protected]
    │ ├─┬ [email protected]
    │ │ └── [email protected]
    │ ├── [email protected] deduped
    │ └── [email protected]
    ├─┬ [email protected]
    │ ├── [email protected]
    │ ├── [email protected] deduped
    │ ├── [email protected] deduped
    │ └─┬ [email protected]
    │   ├─┬ [email protected]
    │   │ ├─┬ [email protected]
    │   │ │ ├── [email protected]
    │   │ │ └── [email protected]
    │   │ ├── [email protected] deduped
    │   │ └─┬ [email protected]
    │   │   ├── [email protected] deduped
    │   │   ├── [email protected] deduped
    │   │   └── [email protected] deduped
    │   ├── [email protected] deduped
    │   ├── [email protected]
    │   ├── [email protected]
    │   └─┬ [email protected]
    │     ├── [email protected] deduped
    │     ├── [email protected] deduped
    │     └── [email protected] deduped
    ├─┬ [email protected]
    │ └── [email protected]
    └── [email protected]

BTW: I'd love to use prebuild-install myself (for various OSS projects) but there's no way I'd just splat there extra 80 deps. I'm currently using node-fetch and so far it's enough for my case (it's just one dep and it can follow redirects and save to file stream) - just saying as something to think about.

BTW2: I'm willing to help if you want to.

is it working with npm-scope ?

Hi,

It seems when using a package with npm score (ex @toto/mypackage) the url to fetch the prebuilt is not correclty formatted.
I did some test locally, when I remove the npm scope this is working fine.

I am using .npmr to specify the remote server and a custom server (not github)

Install issues

Hey everyone,

we've gotten reports about failed installs for atom-julia-client, which uses node-pty-prebuilt, which in turn makes use of this package.

I'm not sure if this is actually the right place for this issue -- feel free to close this if not.

Here's the issue at our repo. The failure on Windows is probably some connection issue:

prebuild-install info unpacking @ C:\Users\Farhan\.atom\.apm\_prebuilds\https-github.com-daviwil-node-pty-prebuilt-releases-download-v0.7.3-node-pty-prebuilt-v0.7.3-electron-v53-win32-x64.tar.gz
prebuild-install WARN install unexpected end of file

even though it apparently was not resolved even when retrying. The Linux issue on the other hand seems consistent:

prebuild-install WARN install ENAMETOOLONG: name too long, open '/home/user/.atom/.apm/_prebuilds/https-github.com-daviwil-node-pty-prebuilt-releases-download-v0.7.3-node-pty-prebuilt-v0.7.3-electron-v54-linux-x64.tar.gz.10302-1f746b13a169.tmp'

AFAIK no OS should have such a low file/path length limit though (the above is ~185 chars for the path and ~145 chars for the filename). I'm not sure how to debug this further though (can't reproduce either issue locally and the debug logs are apparently not available) or what a possible solution might be -- possibly making sure that the file name is much shorter or something....

Thanks!

consider bailing out on download if installing inside cloned repo

In prebuild we added the following check to force a compile when developing:

if (rc.install) {
  if (!(typeof pkg._from === 'string')) {
    // From Git directly
    rc.compile = true
  } else if (pkg._from.length > 4 && pkg._from.substr(0, 4) === 'git+') {
    // From npm install git+
    rc.compile = true
  } else {
    // From npm repository
    rc.download = rc.install
  }
}

from https://github.com/mafintosh/prebuild/blob/master/bin.js#L48-L59

Now, since this module doesn't have the code for compiling, it could basically just do process.exit(1) (instead of setting rc.compile etc) and tack on a .. || node-gyp rebuild just like the README shows to make this work.

Prebuild Install not GETting complete url for prebuilt libs

Using prebuild 9.1.0, on MacOs 10.14.6, using yarn...

prebuild itself will publish seemingly correct builds eg: safe-nodejs-v0.1.3-electron-v73-darwin-x64.tar.gz or safe_nodejs-v0.1.2-node-v72-darwin-x64.tar.gz

however, running prebuild-install fails to GET any URL resembling this. From the verbose log (with or without -r electron -t 6.0.7 set):

prebuild-install info begin Prebuild-install version 5.3.1
prebuild-install info looking for cached prebuild @ /Users/josh/.npm/_prebuilds/067418-72-darwin-x64.tar.gz
prebuild-install http request GET https://github.com/joshuef/safe-nodejs/releases/download/0.1.3/72-darwin-x64.tar.gz
prebuild-install http 404 https://github.com/joshuef/safe-nodejs/releases/download/0.1.3/72-darwin-x64.tar.gz
prebuild-install WARN install No prebuilt binaries found (target=6.0.7 runtime=electron arch=x64 libc= platform=darwin)

It seems like the application title, version, runtime and appropriate ABI are all missing from the GET url...

build scrip outputs everything on the stderr.

until 04.04.2018 my daily Atom update Skript give that output.
(script/build > dev/null)

Sourcen Updaten
Von https://github.com/atom/atom
   9a81957..8ef1a05  master     -> origin/master
   66852b3..cd30b79  1.26-releases -> origin/1.26-releases
Programm erzeugen & installieren
Packaging app for platform linux ia32 using electron v1.7.11
gtk-update-icon-cache: Cache file created successfully.
Apr 04 02:59:45 sakura sendEmail[5219]: Email was sent successfully!

since 05.04.2018 it gives

Sourcen Updaten
Von https://github.com/atom/atom
   d126d08..89945fd  master     -> origin/master
   da9995d..9c28a5b  1.26-releases -> origin/1.26-releases
 * [neuer Branch]    1.27-releases -> origin/1.27-releases
 * [neuer Branch]    weekly-2018-04-16 -> origin/weekly-2018-04-16
 * [neues Tag]       v1.26.0-beta2 -> v1.26.0-beta2
Programm erzeugen & installieren
prebuild-install info begin Prebuild-install version 2.5.1
prebuild-install info looking for local prebuild @ prebuilds/leveldown-v1.9.0-node-v57-linux-ia32.tar.gz
prebuild-install info looking for cached prebuild @ /home/ruehmann/.npm/_prebuilds/https-github.com-level-leveldown-releases-download-v1.9.0-leveldown-v1.9.0-node-v57-linux-ia32.tar.gz
prebuild-install http request GET https://github.com/level/leveldown/releases/download/v1.9.0/leveldown-v1.9.0-node-v57-linux-ia32.tar.gz
prebuild-install http 404 https://github.com/level/leveldown/releases/download/v1.9.0/leveldown-v1.9.0-node-v57-linux-ia32.tar.gz
prebuild-install WARN install No prebuilt binaries found (target=8.6.0 runtime=node arch=ia32 platform=linux)
../deps/leveldb/leveldb-1.20/db/version_set.cc: In member function ‘bool leveldb::Version::RecordReadSample(leveldb::Slice)’:
../deps/leveldb/leveldb-1.20/db/version_set.cc:436:27: warning: ‘state.leveldb::Version::RecordReadSample(leveldb::Slice)::State::stats.leveldb::Version::GetStats::seek_file’ may be used uninitialized in this function [-Wmaybe-uninitialized]
       file_to_compact_ = f;
                           ^
../deps/leveldb/leveldb-1.20/db/version_set.cc:467:9: note: ‘state.leveldb::Version::RecordReadSample(leveldb::Slice)::State::stats.leveldb::Version::GetStats::seek_file’ was declared here
   State state;
         ^
../src/onig-scanner-worker.cc: In member function ‘virtual void OnigScannerWorker::HandleOKCallback()’:
../src/onig-scanner-worker.cc:37:27: warning: ‘v8::Local<v8::Value> Nan::Callback::Call(int, v8::Local<v8::Value>*) const’ is deprecated [-Wdeprecated-declarations]
     callback->Call(2, argv);
                           ^
In file included from ../src/onig-scanner-worker.h:7:0,
                 from ../src/onig-scanner-worker.cc:1:
../../nan/nan.h:1618:3: note: declared here
   Call(int argc, v8::Local<v8::Value> argv[]) const {
   ^
../src/onig-scanner-worker.cc:43:27: warning: ‘v8::Local<v8::Value> Nan::Callback::Call(int, v8::Local<v8::Value>*) const’ is deprecated [-Wdeprecated-declarations]
     callback->Call(2, argv);
                           ^
In file included from ../src/onig-scanner-worker.h:7:0,
                 from ../src/onig-scanner-worker.cc:1:
../../nan/nan.h:1618:3: note: declared here
   Call(int argc, v8::Local<v8::Value> argv[]) const {
   ^
../src/onig-string.cc: In constructor ‘OnigString::OnigString(v8::Local<v8::String>)’:
../src/onig-string.cc:30:40: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   hasMultiByteChars = (value->Length() != utf8_length_);
                                        ^
prebuild-install info begin Prebuild-install version 2.5.1
prebuild-install info looking for local prebuild @ prebuilds/keytar-v4.2.1-node-v57-linux-ia32.tar.gz
prebuild-install info looking for cached prebuild @ /home/ruehmann/.npm/_prebuilds/https-github.com-atom-node-keytar-releases-download-v4.2.1-keytar-v4.2.1-node-v57-linux-ia32.tar.gz
prebuild-install info found cached prebuild 
prebuild-install info unpacking @ /home/ruehmann/.npm/_prebuilds/https-github.com-atom-node-keytar-releases-download-v4.2.1-keytar-v4.2.1-node-v57-linux-ia32.tar.gz
prebuild-install info unpack resolved to /home/ruehmann/atom/apm/node_modules/atom-package-manager/node_modules/keytar/build/Release/keytar.node
prebuild-install info unpack required /home/ruehmann/atom/apm/node_modules/atom-package-manager/node_modules/keytar/build/Release/keytar.node successfully
prebuild-install info install Successfully installed prebuilt binary!
(node:13430) [DEP0022] DeprecationWarning: os.tmpDir() is deprecated. Use os.tmpdir() instead.
../src/onig-scanner-worker.cc: In member function ‘virtual void OnigScannerWorker::HandleOKCallback()’:
../src/onig-scanner-worker.cc:37:27: warning: ‘v8::Local<v8::Value> Nan::Callback::Call(int, v8::Local<v8::Value>*) const’ is deprecated [-Wdeprecated-declarations]
     callback->Call(2, argv);
                           ^
In file included from ../src/onig-scanner-worker.h:7:0,
                 from ../src/onig-scanner-worker.cc:1:
../../nan/nan.h:1618:3: note: declared here
   Call(int argc, v8::Local<v8::Value> argv[]) const {
   ^
../src/onig-scanner-worker.cc:43:27: warning: ‘v8::Local<v8::Value> Nan::Callback::Call(int, v8::Local<v8::Value>*) const’ is deprecated [-Wdeprecated-declarations]
     callback->Call(2, argv);
                           ^
In file included from ../src/onig-scanner-worker.h:7:0,
                 from ../src/onig-scanner-worker.cc:1:
../../nan/nan.h:1618:3: note: declared here
   Call(int argc, v8::Local<v8::Value> argv[]) const {
   ^
../src/onig-string.cc: In constructor ‘OnigString::OnigString(v8::Local<v8::String>)’:
../src/onig-string.cc:30:40: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   hasMultiByteChars = (value->Length() != utf8_length_);
                                        ^
prebuild-install info begin Prebuild-install version 2.5.1
prebuild-install info looking for local prebuild @ prebuilds/keytar-v4.2.1-node-v48-linux-ia32.tar.gz
prebuild-install info looking for cached prebuild @ /home/ruehmann/.npm/_prebuilds/https-github.com-atom-node-keytar-releases-download-v4.2.1-keytar-v4.2.1-node-v48-linux-ia32.tar.gz
prebuild-install info found cached prebuild 
prebuild-install info unpacking @ /home/ruehmann/.npm/_prebuilds/https-github.com-atom-node-keytar-releases-download-v4.2.1-keytar-v4.2.1-node-v48-linux-ia32.tar.gz
prebuild-install info unpack resolved to /home/ruehmann/atom/apm/node_modules/atom-package-manager/node_modules/keytar/build/Release/keytar.node
prebuild-install info unpack required /home/ruehmann/atom/apm/node_modules/atom-package-manager/node_modules/keytar/build/Release/keytar.node successfully
prebuild-install info install Successfully installed prebuilt binary!
Packaging app for platform linux ia32 using electron v1.7.11
gtk-update-icon-cache: Cache file created successfully.
Apr 14 03:29:04 sakura sendEmail[5192]: Email was sent successfully!

The Atom developers said to me.

You should file this on prebuild-install, as we're just using that utility in the script wink.

Thanks very much for the report!

I am running Kubuntu 16.04 LTS (32Bit)
here is my update Script.

#!/bin/bash

ZIEL=$PWD
#BRANCH=master
BRANCH=master
#NODE=7.4.0
NODE=8.9.3
NODER=9.2.0

cd ~/
if [ -d "/usr/share/atom" ]
then
  sudo chown -R root:users /usr/share/atom/ 2> /dev/null
  sudo chmod -R g+w /usr/share/atom/ 2> /dev/null
  if [ -d "atom" ]
    then
      echo "Sourcen Updaten"
      cd atom
      ALTE_ATOM_VERSION="$(git describe --tags)" 2> /dev/null
      #git checkout $BRANCH
      GIT_PULL=$(git pull | grep -c "Already up-to-date.")
      NEUE_ATOM_VERSION="$(git describe --tags)" 2> /dev/null
        if [ $GIT_PULL -ge 1 ]
        then
          echo "Already up-to-date."
        else
          echo "Programm erzeugen & installieren"
          sudo -H n $NODE
          find ~/ -name package-lock.json -delete > /dev/null
	  script/clean > /dev/null
	  script/build > /dev/null
	  packagedAppPath=$(echo ~/atom/out/atom-*)
	  sudo node -e "require('./script/lib/install-application')('$packagedAppPath', '')" > /dev/null
	  sudo -H n $NODER
        fi
    else
      echo "Aktuelle Sourcen Clonen"
      git clone https://github.com/atom/atom.git &>/dev/null
      cd atom
      echo "Programm erzeugen & installieren"
      sudo -H n $NODE
      find ~/ -name package-lock.json -delete > /dev/null
      script/clean > /dev/null
      script/build > /dev/null
      packagedAppPath=$(echo ~/atom/out/atom-*)
      sudo node -e "require('./script/lib/install-application')('$packagedAppPath', '')" > /dev/null
      sudo -H n $NODER
  fi
fi
MESSAGE="Das Tägliche Atom Update auf $(hostname) wurde durchgeführt.
alte Versionen:
atom $ALTE_ATOM_VERSION
--------------------------------------
neue Versionen:
atom $NEUE_ATOM_VERSION
--------------------------------------

Atom github Link: https://github.com/atom/atom/commits/$BRANCH"
sendEmail -f [email protected] -t [email protected] -o message-charset=utf8 -u "Daily atom Update on $(hostname)" -m "$MESSAGE" -s mail.ruehmann.name:587 -xu [email protected] -xp "XXXXXX"
cd "$ZIEL"

exit 0

Custom binaries host not used when the package's name contains a dash

I am building an electron app unsing electron-builder. If I want to use custom binaries for a dependency, e.g. node-hid I am setting node-hid_binary_host = http://localhost:1234/ in .npmrc, and npm confirms this is set.
It seems that at some point the dashes in the variable names are replaced with underscores, so prebuild-install can not find the variable and does not use the alternate server.
By patching the getHostMirrorUrl function to replace dashes with underscores in the pkg.name I was able to get the desired behaviour, but I'm not sure this will not cause collisions at some point.

Module incorrectly resolves npm cache folder

Hello.

In your code, in the util.js file, you have the following function:

function npmCache () {
  return process.env.APPDATA ? path.join(process.env.APPDATA, 'npm-cache') : path.join(home(), '.npm')
}

From the code it is clear that it only considers case for windows and linux/mac and uses default locations for npm-cache on windows and .npm on unix.

However, npm allows to specify the configuration parameter cache in the ~/.npmrc file which tells npm the location of cache for it.

Could you make use of that variable please?

Don't require modules if runtime is electron

When runtime is electron and it happens to be the same modules version, require from node fails.
Tested with node v8.9.4 (for npm), electron v1.8.4 and serialport v6.1.1.

Possible solution: change condition in line 132 of download.js to skip this for electron runtime:
if (opts.runtime !== 'electron' && opts.platform === process.platform && opts.abi === process.versions.modules) {

Private GitHub repo

I'm trying to use prebuild install with a private GitHub repo.

Is there any way we could have a similar idea where you can add your token to .prebuildrc?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.