broccolijs/node-symlink-or-copy

Symlink files or directories, falling back to copying on Windows

node-symlink-or-copy's Issues

investigate and restore 800ef4f.

800ef4f

restore fix for #23

[WINDOWS] fallback to junctions for directories (even if symlinks are not available)

As junctions should always be available, if we symlinkOrCopy a directory, we should never need to do a deep copy recursive copy. We only need to do a copy, if we are attempting to link files but not have symlinks enabled

@raytiley thoughts?

EPERM: operation not permitted, unlink 'C:\Users\lifeart\AppData\Local\Temp\canLinkSrc.tmp'

=================================================================================

ERROR Summary:

• broccoliBuilderErrorStack: Error: EPERM: operation not permitted, unlink 'C:\Users\lifeart\AppData\Local\Temp\canLinkSrc.tmp'
  at Object.unlinkSync (fs.js:952:3)
  at testCanSymlink (C:\Users\lifeart\Documents\repos\debug-hot-load\node_modules\symlink-or-copy\index.js:31:8)
  at Object. (C:\Users\lifeart\Documents\repos\debug-hot-load\node_modules\symlink-or-copy\index.js:9:15)
  at Module._compile (internal/modules/cjs/loader.js:689:30)
  at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
  at Module.load (internal/modules/cjs/loader.js:599:32)
  at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
  at Function.Module._load (internal/modules/cjs/loader.js:530:3)
  at Module.require (internal/modules/cjs/loader.js:637:17)
  at require (internal/modules/cjs/helpers.js:22:18)
• code: [undefined]
• codeFrame: EPERM: operation not permitted, unlink 'C:\Users\lifeart\AppData\Local\Temp\canLinkSrc.tmp'
• errorMessage: debug-hot-reload/components/hot-placeholder.js: EPERM: operation not permitted, unlink 'C:\Users\lifeart\AppData\Local\Temp\canLinkSrc.tmp'
  in C:\Users\lifeart\AppData\Local\Temp\broccoli-1724wiG4basIFok7\out-136-funnel
  at broccoli-persistent-filter:Babel > [Babel: debug-hot-reload] (Babel: debug-hot-reload)
• errorType: Build Error
• location:
  • column: [undefined]
  • file: debug-hot-reload/components/hot-placeholder.js
  • line: [undefined]
  • treeDir: C:\Users\lifeart\AppData\Local\Temp\broccoli-1724wiG4basIFok7\out-136-funnel
• message: debug-hot-reload/components/hot-placeholder.js: EPERM: operation not permitted, unlink 'C:\Users\lifeart\AppData\Local\Temp\canLinkSrc.tmp'
  in C:\Users\lifeart\AppData\Local\Temp\broccoli-1724wiG4basIFok7\out-136-funnel
  at broccoli-persistent-filter:Babel > [Babel: debug-hot-reload] (Babel: debug-hot-reload)
• name: BuildError
• nodeAnnotation: Babel: debug-hot-reload
• nodeName: broccoli-persistent-filter:Babel > [Babel: debug-hot-reload]
• originalErrorMessage: EPERM: operation not permitted, unlink 'C:\Users\lifeart\AppData\Local\Temp\canLinkSrc.tmp'
• stack: Error: EPERM: operation not permitted, unlink 'C:\Users\lifeart\AppData\Local\Temp\canLinkSrc.tmp'
  at Object.unlinkSync (fs.js:952:3)
  at testCanSymlink (C:\Users\lifeart\Documents\repos\debug-hot-load\node_modules\symlink-or-copy\index.js:31:8)
  at Object. (C:\Users\lifeart\Documents\repos\debug-hot-load\node_modules\symlink-or-copy\index.js:9:15)
  at Module._compile (internal/modules/cjs/loader.js:689:30)
  at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
  at Module.load (internal/modules/cjs/loader.js:599:32)
  at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
  at Function.Module._load (internal/modules/cjs/loader.js:530:3)
  at Module.require (internal/modules/cjs/loader.js:637:17)
  at require (internal/modules/cjs/helpers.js:22:18)

=================================================================================

canSymlink is false on Linux/macOS as of 1.3.0

As of v1.3.0, this line causes canSymlink to be false on Linux and macOS. Is this supposed to be happening? It used to be true.

/cc @stefanpenner

Build errors on Windows, v1.1.4 not on npm

v1.1.4 is not on npm, so the build still breaks on Windows, probably because of missing fix that was merged after 1.1.3.
Build then breaks on 1.1.4 if \tmp folder already exists, so it has to be deleted manually before each build, error is:

EEXIST: file already exists, mkdir 'C:\Project\tmp'
Error: EEXIST: file already exists, mkdir 'C:\Project\tmp'
    at Error (native)
    at Object.fs.mkdirSync (fs.js:916:18)
    at testCanSymlink (C:\Project\node_modules\symlink-or-copy\index.js:18:6)
    at Object.<anonymous> (C:\Project\node_modules\symlink-or-copy\index.js:9:15)
    at Module._compile (module.js:541:32)
    at Object.Module._extensions..js (module.js:550:10)
    at Module.load (module.js:458:32)
    at tryModuleLoad (module.js:417:12)
    at Function.Module._load (module.js:409:3)
    at Module.require (module.js:468:17)

Use fs.copyFile() api in node 8.5+

See https://nodejs.org/api/fs.html#fs_fs_copyfile_src_dest_flags_callback

The new copyFile implementation is based on uv_fs_copyfile(), and uses much faster native file copying api's, where available. See nodejs/node#14906 (comment).

tests

reminder to everyone contributing here, we need tests for the project.

• these tests must exercise everything broccoli core could use this library for
• must run on travis
• must run on appveyor

Windows may allow file symlinks but forbid directory symlinks

I came across a small issue while using this module on Windows: As it turned out, Windows allowed making a file symlink, but forbids making a directory symlink, failing with Error: EPERM: operation not permitted.

(I haven't looked into the why exactly Windows behaves like that, although there were some changes in Windows 10 that might relate to this behaviour, see here?)

Since testCanSymlink() only tests by creating a file symlink, the module then proceeds to use symlinks and fails when trying to symlink a directory.

The problem was resolved for me by extending the testCanSymlink() function with the following lines:

  // Test symlinking a directory. For some reason, sometimes Windows allows
  // symlinking a file but not symlinking a directory...
  try {
    fs.mkdirSync(canLinkSrc);
  } catch (e) {
    return false
  }

  try {
    fs.symlinkSync(canLinkSrc, canLinkDest, 'dir')
  } catch (e) {
    fs.rmdirSync(canLinkSrc)
    return false
  }

  fs.rmdirSync(canLinkSrc)
  fs.rmdirSync(canLinkDest)

Now testCanSymlink() would return false on my system and the module would work without errors on directories.

I'd open a pull request, but I'm not sure this fix is the prefered way to resolve this issue. Alternatively the module could be expanded to track both canFileSymlink and canDirSymlink options, so that files would still be symlinked even if directory symlinks are not supported. Or, and that would be a backward compat break, we could just always use junctions for directories when on Windows, bypassing this problem all together.

Module fails when running on Windows environment from Mac OS file system

Scenario: Windows VM (on Mac OS host) in Powershell running this module. The files it is running on are shared to the VM from the Mac OS (HFS+) filesystem. The module makes the wrong decision about what to do and fails every time. Error below:

Build failed.

EISDIR, illegal operation on a directory 'Y:\main\Presentation\PWS\Scentsy-cli\Checkout\app\index.html'
Error: EISDIR, illegal operation on a directory 'Y:\main\Presentation\PWS\Scentsy-cli\Checkout\app\index.html'
at Error (native)
at Object.fs.symlinkSync (fs.js:848:18)
at symlinkWindows (Y:\main\Presentation\PWS\Scentsy-cli\Checkout\node_modules\ember-cli\node_modules\symlink-or-copy

\index.js:90:16)

The reason for the failure is because it's performing a Windows operation on HFS+ based files.

Maybe don't die on dead symlinks

I'm not sure exactly what is causing this problem, it seems to be related to angular's TreeStabilizer plugin implemented to work around the unstable outputPaths issue.

I'm fairly sure the producer of the dead symlinks is one of our own plugins (MultiCopy), but it's very difficult to verify that. Subsequently, symlink-or-copy relies on fs.realpathSync() and throws an ENOENT due to the dead simlink

For some details on how this bug manifests itself, see angular/angular#2386

If I had better tools for debugging this, I could do a better job sorting it out, but it is a regression introduced by the implementation of an incremental version of broccoli-merge-trees

Error: EEXIST, file already exists - complains about source (!) file

as verified in the code of symlinkOrCopySync:

• given source file: '/home/<...>/tmp/tree_merger-tmp_dest_dir-f1avIwvJ.tmp/vendor/js/angular-bootstrap/ui-bootstrap-tpls.min.js'
• actual source: '/home/<..>/vendor/bower/angular-bootstrap/ui-bootstrap-tpls.min.js'
• dest file: '/home/ubuntu/dev/rws-datalab-webclient/tmp/select-tmp_dest_dir-a9vQmXu3.tmp/vendor/js/angular-bootstrap/ui-bootstrap-tpls.min.js'

Built with error:
Error: EEXIST, file already exists '/home/<...>/vendor/bower/angular-bootstrap/ui-bootstrap-tpls.min.js'
  at Object.fs.symlinkSync (fs.js:741:18)
  at symlinkOrCopySync (/home/ubuntu/<...>/node_modules/broccoli-select/node_modules/symlink-or-copy/index.js:37:8) 

The dest file does not exist, and I may indeed hope the actual source file does exist.
I have all modules (including node) updated to the latest versions available.
Function is called with correct values for src and dest, and I can see that src is correctly replaced with the original location

Hence I do not understand why the EEXIST is thrown on the srcPath

Version not found: [email protected]

I can see on npm, that version 1.1.8 was updated yesterday, but when I install it, I get this error:

± |master ✓| → npm install [email protected]
npm ERR! Darwin 15.6.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "[email protected]"
npm ERR! node v5.0.0
npm ERR! npm  v2.10.0

npm ERR! version not found: [email protected]
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR!     <https://github.com/npm/npm/issues>

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/anton/projects/mui2/npm-debug.log

Has anyone else encountered this?

This line cause errors on windows without symlink

node-symlink-or-copy/index.js

Line 143 in dcd548f

options.fs.symlinkSync(srcPath, destPath, 'junction');

Looks like if no symlink support - this method should looks like copy folder sync https://www.npmjs.com/package/copy-dir

1.1.1 breaks ember apps on windows

Original issue ember-cli/ember-cli#5820

Version 1.1.5 causes Ember CLI to be failed on Windows.

Version 1.1.5 causes ember cli commands (ember s, ember t ...) to be failed on Windows. The error is:
EEXIST: file already exists, mkdir 'D:...\tmp'

Please, oh pretty please implement symlinking on Windows!

I am using Ember-CLI on Windows, Admin CMD window, and my build times are around 10 seconds for a blank CLI app (v0.2.7 and v1.13.1).

Apparently this could be very much faster if symlinking were working on Windows! Unrelenting joy would spread like wildfire around the Earth.

ExFat Support

Hi,

I have read about this issue in another repo: TryGhost/Ghost-CLI#1726
ExFat disk formatting does not support symlink, I could not find a direct way to check the file format,

Do you have suggestions to solve this?

I think that we should add a checker for in testSymlink: https://github.com/broccolijs/node-symlink-or-copy/blob/070e6ada1b699620800222b7e60b1e40627ba98e/index.js

here is a result I found on StackOverFlow:

https://stackoverflow.com/questions/46853070/get-filesystem-type-with-node-js

and code snippet suggested by ChatGPT:

const fs = require('fs');

fs.stat('/path/to/drive', (error, stats) => {
  if (error) {
    console.error(error);
    return;
  }

  console.log(stats.fstype);
});

EEXIST, file already exists when using npm linked ember addon

When using a npm linked ember addon from within another project or addon I'm getting:

$ ember serve
version: 0.1.1
Livereload server on port 35729
Serving on http://0.0.0.0:4200

EEXIST, file already exists '/home/woodzu/Ember/my-app/../my-addon/bower_components/bootstrap/dist/css/'
Error: EEXIST, file already exists '/home/woodzu/Ember/my-app/../my-addon/bower_components/bootstrap/dist/css/'
    at Object.fs.symlinkSync (fs.js:735:18)
    at symlinkOrCopySync (/home/woodzu/Ember/my-app/node_modules/ember-cli/node_modules/symlink-or-copy/index.js:37:8)
    at CustomStaticCompiler.StaticCompiler._copy (/home/woodzu/Ember/my-app/node_modules/ember-cli/node_modules/broccoli-static-compiler/index.js:52:3)
    at /home/woodzu/Ember/my-app/node_modules/ember-cli/node_modules/broccoli-static-compiler/index.js:25:12
    at $$$internal$$tryCatch (/home/woodzu/Ember/my-app/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:470:16)
    at $$$internal$$invokeCallback (/home/woodzu/Ember/my-app/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:482:17)
    at $$$internal$$publish (/home/woodzu/Ember/my-app/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:453:11)
    at $$rsvp$asap$$flush (/home/woodzu/Ember/my-app/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:1531:9)
    at process._tickCallback (node.js:419:13)

cat ../my-addon/index.js

var path = require('path');

module.exports = {

  name: 'my-addon',

  included: function(app) {

    this._super.included(app);
    var modulePath = path.relative(app.project.root, __dirname);
    var bootstrapPath = path.join(modulePath, 'bower_components/bootstrap/dist');

    app.import(path.join(bootstrapPath, 'css/bootstrap.min.css'));
    app.import(path.join(bootstrapPath, 'css/bootstrap-theme.min.css'));
  }
};

$ ember version

version: 0.1.1
node: 0.10.30
npm: 2.1.2

$ uname -a
Linux ubuntu 3.11.0-14-generic #21-Ubuntu SMP Tue Nov 12 17:04:55 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux