axosoft/nsfw

A super fast and scaleable file watcher that provides a consistent interface on Linux, OSX and Windows

node-sentinel-file-watcher

Linux	OS X	Windows

A simple file watcher library for node.

Why NSFW?

NSFW is a native abstraction for Linux, Windows, and OSX file watching services which tries to keep a consistent interface and feature set across operating systems. NSFW offers recursive file watching into deep file systems all at no additional cost to the Javascript layer. In Linux, NSFW recursively builds an inotify watch tree natively, which collects events concurrently to the javascript thread. In OSX, NSFW utilizes the FSEventsService, which recursively watches for file system changes in a specified directory. In Windows, NSFW implements a server around the ReadDirectoryChangesW method.

When NSFW has events and is not being throttled, it will group those events in the order that they occurred and report them to the Javascript layer in a single callback. This is an improvement over services that utilize Node FS.watch, which uses a callback for every file event that is triggered. Every callback FS.watch makes to the event queue is a big bonus to NSFW's performance when watching large file system operations, because NSFW will only make 1 callback with many events within a specified throttle period.

So why NSFW? Because it has a consistent and minimal footprint in the Javascript layer, manages recursive watching for you, and is super easy to use.

Usage

var nsfw = require('nsfw');

var watcher1;
return nsfw(
  'dir1',
  function(events) {
    // handle events
  })
  .then(function(watcher) {
    watcher1 = watcher;
    return watcher.start();
  })
  .then(function() {
    // we are now watching dir1 for events!

    // To stop watching
    watcher1.stop()
  });

// With options
var watcher2;
return nsfw(
  'dir2',
  function(events) {
  // handles other events
  },
  {
    debounceMS: 250,
    errorCallback(errors) {
      //handle errors
    },
    excludedPaths: ['dir2/node_modules']
  })
  .then(function(watcher) {
    watcher2 = watcher;
    return watcher.start();
  })
  .then(function() {
    // we are now watching dir2 for events!

    // we can update excludedPaths array
    return watcher2.updateExcludedPaths(['dir2/node_modules', '.git']);
  })
  .then(function() {
    // To stop watching
    watcher2.stop();
  })

Options

• debounceMS: delays notifications emitted by the library. Default 500 ms.
• errorCallback(errors): the library will call this callback when an error happens. At the moment when an error happens the service does not stop, this may change in the near future.
• excludedPaths: array with the absolute paths we want to exclude watching. You can update the excludedPaths array without restarting the service using the updateExcludedPaths function

Callback Argument

An array of events as they have happened in a directory, it's children, or to a file.

[
  {
    "action": 2, // nsfw.actions.MODIFIED
    "directory": "/home/nsfw/watchDir",
    "file": "file1.ext"
  },
  {
    "action": 0, // nsfw.actions.CREATED
    "directory": "/home/nsfw/watchDir",
    "file": "folder"
  },
  {
    "action": 1, // nsfw.actions.DELETED
    "directory": "home/nsfw/watchDir/testFolder",
    "file": "test.ext"
  },
  {
    "action": 3, // nsfw.actions.RENAMED
    "directory": "home/nsfw/watchDir",
    "oldFile": "oldname.ext",
    "newDirectory": "home/nsfw/watchDir/otherDirectory"
    "newFile": "newname.ext"
  }
]

Event are enumerated by the nsfw.actions enumeration

nsfw.actions = {
  CREATED: 0,
  DELETED: 1,
  MODIFIED: 2,
  RENAMED: 3
};

Installation

NSFW is a native node module and requires Node-Gyp to be functional before you can install it. Make sure you have completed installing all of the dependencies listed for Node-Gyp on your operating system.

nsfw's Issues

Might be reasonable to pick a new name

So I thought I should point out a little thing about the name of this particular program, and most likely at least one person on the team knows this. NSFW is more commonly known as the abbreviation for "Not Safe For Work," which means explicit content (pornography and such). Hope this helps if you guys don't want to create an accidental impression because of this, and that you got a good laugh from it. ;P (If it were intentionally named as such, though, then my gosh, Axosoft! XD)

Memory leak & poor performance when using Rust Language Server on Linux

I've traced a atom memory & performance issue when using RLS (Rust Language Server) to Atom's version of this project.

When watching a rust project directory with nsfw and running RLS, nsfw will leak memory and consume cpu.

Downstream: atom#7, rust-lang/atom-ide-rust#104

Steps to reproduce

• Install node & rustup latest stable (see https://rustup.rs, v1.35.0 at time of writing)
• Install rls rustup component add rls
• Checkout a rust project that isn't tiny, say regex d4b9419

git clone https://github.com/rust-lang/regex
cd regex
git checkout d4b9419

• Construct a minimal node project with nsfw in another directory

{
  "name": "ntmp",
  "version": "1.0.0",
  "dependencies": {
    "nsfw": "1.2.2"
  }
}

• Watch the rust project

node                                                                                                                              
> require('nsfw')('/home/alex/project/regex', events => {}).then(w => w.start())

• Run RLS in the rust project

cd regex
rls --cli

• Restart RLS & re-run to see further leakage.

With these steps it's easy enough to see nsfw consume several GB of ram & lots of CPU as RLS is running. This memory usage/leakage is worse on larger rust projects.

RLS uses cargo/rustc to compile code and generates lots of data in the ./target directory in the project which is what triggers this issue. Strangely though running cargo directly doesn't cause this memory issue. I not sure why this is RLS specific.

Remove poll thread in favor uv_async_send

Currently nsfw uses a polling thread to query the change queue. We should remove that polling thread in favor of a uv_async_t handle and a throttled callback that each file watcher can call.

Linux: delete (via trash) is not reported as delete

Steps:

• use Linux (e.g. Ubuntu)
• delete a file via the native explorer (moves to trash)
  => no event for the delete is printed

It looks like chokidar is handling this properly, a unlink is reported and an add event when restoring the file.

How to handle the MODIFIED event when copying files etc..

I'm working on a filesync app, that watches for changes, new files and so on.

I've noticed on Windows that the MODIFIED event fires multiple times when copying a file to the folder.

The same thing happens when you copy a file in the same folder.

This is kinda an issue for me, as i only need the event once per file.

How should one handle this?

Virtual memory leak on Linux?

Hi,

OS: Ubuntu 16.04
node: v8.1.1

We use NSFW for our project . We have noticed an apparent virtual memory leak, at least on Linux. Digging some more, it seems that it's related to receiving NSFW events. Note that the resident memory does not seem to increase much.

I was able to reproduce the issue, using a slightly modified version of your readme example. I have changed the watched directory to /tmp/ and removed watcher1.stop(), so that the program doesn't exit after receiving the first event.

var nsfw = require('nsfw');

var watcher1;

return nsfw(
  '/tmp',
  function (events) {
    // handle events
  })
  .then(function (watcher) {
    // only "start()" once
    if (watcher1 !== watcher) {
      watcher1 = watcher;
      return watcher.start();
    }
  });

Then I added an "event generator" script, that appends to a file 100 times a second, under the watched directory. (lower append speed also triggers leak, but not as fast):

#!/bin/bash
counter=1
while [ $counter -le 2 ]
do
	echo "a" >> /tmp/file.txt
        sleep 0.01
done

When we start the watcher program, node reports that ~1 GB of virtual memory is used. When we then start generating events using the script above, the reported virtual memory increases at a rate of about 1GB a minute (up-to 256GB if we leave it run long enough)

---

Am I doing something wrong in the code above or does this look like a bug?

AWS Storage Gateway - editing jpg in place does not result in modified event

I'm pointing NSFW at a directory in an AWS file gateway. This gateway is mounted to a ubuntu EC2 instance. I'm using sshfs to mount the EC2 instance to my local Mac.

Most of the events are spot on, however I do see inconsistencies with in-place editing of image files and I think it's related to resolution of edit history (tmp files).

Specifically, tmp edit files are being spun off from the original file and eventually being saved back under the original file's name. NSFW treats these tmp files as new files and does not resolve them back to the original file well, despite inotify showing a moved_from event with the tmp file name and a moved_to with the original file's name.

I get different issues depending on what program I use to edit the file. Photoshop, for example, never shows any rename of the tmp file to the original name. By contrast, Mac's Preview application does show a rename line from a tmp file to the original name, but the 'from' tmp file name referenced only shows in the rename action - there's no record of it being created.

In both situations there's not enough of a file trail for me to code in logic to resolve these issues within my NSFW dependent application. Hoping this can be looked into.

No events for files created inside `temp`

This patch modifies the spec to create files inside a temporary directory created with temp.

As a result, all tests fail with timeouts (after about 240000ms).

Tested against 1.0.12 (d466bd1) on OS X:

ProductName:    Mac OS X
ProductVersion: 10.11.4
BuildVersion:   15E65

Fallback to polling files in Linux

When the INotify limit is hit on a linux device, we should fallback to file polling directories that exceed the limit.

Edit: A potential name for the data structure that hosts poll nodes: Poultry PollTree

Error installing on windows 10

When I first went to install this via npm install nsfw, MSBuild threw back a big nasty build error. After a bit of research I found this article, then after running npm install -g windows-build-tools I was able to install nsfw normally.

I don't know if this is something you can integrate into the npm install? Or at the very least a README mention would be helpful.

For reference:
node v6.11.1
windows: 10.0.15063 Build 15063

Fails to build against Electron 7.0.0-beta.4

Steps:

• git clone https://github.com/Axosoft/nsfw.git
• add .yarnrc with content:

disturl "https://atom.io/download/electron"
target "7.0.0-beta.4"
runtime "electron"

• yarn

$ node-gyp rebuild
  CXX(target) Release/obj.target/nsfw/src/NSFW.o
../src/NSFW.cpp:48:32: warning: 'Call' is deprecated [-Wdeprecated-declarations]
  baton->nsfw->mErrorCallback->Call(1, argv);
                               ^
../node_modules/nan/nan.h:1739:3: note: 'Call' has been explicitly marked deprecated here
  NAN_DEPRECATED inline v8::Local<v8::Value>
  ^
../node_modules/nan/nan.h:104:40: note: expanded from macro 'NAN_DEPRECATED'
# define NAN_DEPRECATED __attribute__((deprecated))
                                       ^
../src/NSFW.cpp:66:14: error: no matching member function for call to 'Set'
    jsEvent->Set(New<v8::String>("action").ToLocalChecked(), New<v8::Number>((*events)[i]->type));
    ~~~~~~~~~^~~
/Users/bpasero/.node-gyp/7.0.0-beta.4/include/node/v8.h:3426:37: note: candidate function not viable: requires 3 arguments, but 2 were provided
  V8_WARN_UNUSED_RESULT Maybe<bool> Set(Local<Context> context,
                                    ^
/Users/bpasero/.node-gyp/7.0.0-beta.4/include/node/v8.h:3429:37: note: candidate function not viable: requires 3 arguments, but 2 were provided
  V8_WARN_UNUSED_RESULT Maybe<bool> Set(Local<Context> context, uint32_t index,
                                    ^
../src/NSFW.cpp:67:14: error: no matching member function for call to 'Set'
    jsEvent->Set(New<v8::String>("directory").ToLocalChecked(), New<v8::String>((*events)[i]->fromDirectory).ToLocalChecked());
    ~~~~~~~~~^~~
/Users/bpasero/.node-gyp/7.0.0-beta.4/include/node/v8.h:3426:37: note: candidate function not viable: requires 3 arguments, but 2 were provided
  V8_WARN_UNUSED_RESULT Maybe<bool> Set(Local<Context> context,
                                    ^
/Users/bpasero/.node-gyp/7.0.0-beta.4/include/node/v8.h:3429:37: note: candidate function not viable: requires 3 arguments, but 2 were provided
  V8_WARN_UNUSED_RESULT Maybe<bool> Set(Local<Context> context, uint32_t index,
                                    ^
../src/NSFW.cpp:70:16: error: no matching member function for call to 'Set'
      jsEvent->Set(New<v8::String>("oldFile").ToLocalChecked(), New<v8::String>((*events)[i]->fromFile).ToLocalChecked());
      ~~~~~~~~~^~~
/Users/bpasero/.node-gyp/7.0.0-beta.4/include/node/v8.h:3426:37: note: candidate function not viable: requires 3 arguments, but 2 were provided
  V8_WARN_UNUSED_RESULT Maybe<bool> Set(Local<Context> context,
                                    ^
/Users/bpasero/.node-gyp/7.0.0-beta.4/include/node/v8.h:3429:37: note: candidate function not viable: requires 3 arguments, but 2 were provided
  V8_WARN_UNUSED_RESULT Maybe<bool> Set(Local<Context> context, uint32_t index,
                                    ^
../src/NSFW.cpp:71:16: error: no matching member function for call to 'Set'
      jsEvent->Set(New<v8::String>("newDirectory").ToLocalChecked(), New<v8::String>((*events)[i]->toDirectory).ToLocalChecked());
      ~~~~~~~~~^~~
/Users/bpasero/.node-gyp/7.0.0-beta.4/include/node/v8.h:3426:37: note: candidate function not viable: requires 3 arguments, but 2 were provided
  V8_WARN_UNUSED_RESULT Maybe<bool> Set(Local<Context> context,
                                    ^
/Users/bpasero/.node-gyp/7.0.0-beta.4/include/node/v8.h:3429:37: note: candidate function not viable: requires 3 arguments, but 2 were provided
  V8_WARN_UNUSED_RESULT Maybe<bool> Set(Local<Context> context, uint32_t index,
                                    ^
../src/NSFW.cpp:72:16: error: no matching member function for call to 'Set'
      jsEvent->Set(New<v8::String>("newFile").ToLocalChecked(), New<v8::String>((*events)[i]->toFile).ToLocalChecked());
      ~~~~~~~~~^~~
/Users/bpasero/.node-gyp/7.0.0-beta.4/include/node/v8.h:3426:37: note: candidate function not viable: requires 3 arguments, but 2 were provided
  V8_WARN_UNUSED_RESULT Maybe<bool> Set(Local<Context> context,
                                    ^
/Users/bpasero/.node-gyp/7.0.0-beta.4/include/node/v8.h:3429:37: note: candidate function not viable: requires 3 arguments, but 2 were provided
  V8_WARN_UNUSED_RESULT Maybe<bool> Set(Local<Context> context, uint32_t index,
                                    ^
../src/NSFW.cpp:74:16: error: no matching member function for call to 'Set'
      jsEvent->Set(New<v8::String>("file").ToLocalChecked(), New<v8::String>((*events)[i]->fromFile).ToLocalChecked());
      ~~~~~~~~~^~~
/Users/bpasero/.node-gyp/7.0.0-beta.4/include/node/v8.h:3426:37: note: candidate function not viable: requires 3 arguments, but 2 were provided
  V8_WARN_UNUSED_RESULT Maybe<bool> Set(Local<Context> context,
                                    ^
/Users/bpasero/.node-gyp/7.0.0-beta.4/include/node/v8.h:3429:37: note: candidate function not viable: requires 3 arguments, but 2 were provided
  V8_WARN_UNUSED_RESULT Maybe<bool> Set(Local<Context> context, uint32_t index,
                                    ^
../src/NSFW.cpp:77:17: error: no matching member function for call to 'Set'
    eventArray->Set(i, jsEvent);
    ~~~~~~~~~~~~^~~
/Users/bpasero/.node-gyp/7.0.0-beta.4/include/node/v8.h:3426:37: note: candidate function not viable: requires 3 arguments, but 2 were provided
  V8_WARN_UNUSED_RESULT Maybe<bool> Set(Local<Context> context,
                                    ^
/Users/bpasero/.node-gyp/7.0.0-beta.4/include/node/v8.h:3429:37: note: candidate function not viable: requires 3 arguments, but 2 were provided
  V8_WARN_UNUSED_RESULT Maybe<bool> Set(Local<Context> context, uint32_t index,
                                    ^
../src/NSFW.cpp:84:25: warning: 'Call' is deprecated [-Wdeprecated-declarations]
  nsfw->mEventCallback->Call(1, argv);
                        ^
../node_modules/nan/nan.h:1739:3: note: 'Call' has been explicitly marked deprecated here
  NAN_DEPRECATED inline v8::Local<v8::Value>
  ^
../node_modules/nan/nan.h:104:40: note: expanded from macro 'NAN_DEPRECATED'
# define NAN_DEPRECATED __attribute__((deprecated))
                                       ^
../src/NSFW.cpp:191:15: warning: 'Call' is deprecated [-Wdeprecated-declarations]
    callback->Call(1, argv);
              ^
../node_modules/nan/nan.h:1739:3: note: 'Call' has been explicitly marked deprecated here
  NAN_DEPRECATED inline v8::Local<v8::Value>
  ^
../node_modules/nan/nan.h:104:40: note: expanded from macro 'NAN_DEPRECATED'
# define NAN_DEPRECATED __attribute__((deprecated))
                                       ^
../src/NSFW.cpp:196:33: error: no matching member function for call to 'Set'
  New(nsfw->mPersistentHandle)->Set(New("nsfw").ToLocalChecked(), info.This());
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
/Users/bpasero/.node-gyp/7.0.0-beta.4/include/node/v8.h:3426:37: note: candidate function not viable: requires 3 arguments, but 2 were provided
  V8_WARN_UNUSED_RESULT Maybe<bool> Set(Local<Context> context,
                                    ^
/Users/bpasero/.node-gyp/7.0.0-beta.4/include/node/v8.h:3429:37: note: candidate function not viable: requires 3 arguments, but 2 were provided
  V8_WARN_UNUSED_RESULT Maybe<bool> Set(Local<Context> context, uint32_t index,
                                    ^
../src/NSFW.cpp:238:15: warning: 'Call' is deprecated [-Wdeprecated-declarations]
    callback->Call(1, argv);
              ^
../node_modules/nan/nan.h:1739:3: note: 'Call' has been explicitly marked deprecated here
  NAN_DEPRECATED inline v8::Local<v8::Value>
  ^
../node_modules/nan/nan.h:104:40: note: expanded from macro 'NAN_DEPRECATED'
# define NAN_DEPRECATED __attribute__((deprecated))
                                       ^
../src/NSFW.cpp:240:15: warning: 'Call' is deprecated [-Wdeprecated-declarations]
    callback->Call(0, NULL);
              ^
../node_modules/nan/nan.h:1739:3: note: 'Call' has been explicitly marked deprecated here
  NAN_DEPRECATED inline v8::Local<v8::Value>
  ^
../node_modules/nan/nan.h:104:40: note: expanded from macro 'NAN_DEPRECATED'
# define NAN_DEPRECATED __attribute__((deprecated))
                                       ^
../src/NSFW.cpp:265:15: warning: 'Call' is deprecated [-Wdeprecated-declarations]
    callback->Call(1, argv);
              ^
../node_modules/nan/nan.h:1739:3: note: 'Call' has been explicitly marked deprecated here
  NAN_DEPRECATED inline v8::Local<v8::Value>
  ^
../node_modules/nan/nan.h:104:40: note: expanded from macro 'NAN_DEPRECATED'
# define NAN_DEPRECATED __attribute__((deprecated))
                                       ^
../src/NSFW.cpp:308:13: warning: 'Call' is deprecated [-Wdeprecated-declarations]
  callback->Call(0, NULL);
            ^
../node_modules/nan/nan.h:1739:3: note: 'Call' has been explicitly marked deprecated here
  NAN_DEPRECATED inline v8::Local<v8::Value>
  ^
../node_modules/nan/nan.h:104:40: note: expanded from macro 'NAN_DEPRECATED'
# define NAN_DEPRECATED __attribute__((deprecated))
                                       ^
7 warnings and 8 errors generated.
make: *** [Release/obj.target/nsfw/src/NSFW.o] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:262:23)
gyp ERR! stack     at ChildProcess.emit (events.js:198:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
gyp ERR! System Darwin 18.7.0
gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/bpasero/Desktop/nsfw
gyp ERR! node -v v10.16.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok 
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

Crashes on Mac OS

Details can be found here atom/github#561

Changes to directory reported as a file on Windows

When watching for changes on Windows the underlying FS api reports both files and folders.

However the consumption code in ReadLoop only expects files because it splits the path it has been given based on the last separator into directory and file. Which means when a folder changes it gets reported as a file.

I'd recommend not splitting the filename out if it is a directory being reported although this is kind of a breaking change as file would be undefined for these new type of events.

Also need to consider a directory rename. Should probably be oldDirectory and newDirectory.

You can repro this by monitoring for changes in a folder then doing mkdir test and you'll see;

{"action":0,"directory":"D:\\src\\nsfw-watcher","file":"test"}

inconsistent events for file edits vs file creation / deletion

When using nsfw to watch myProject folder that has the following structure

myProject
      |___ node_modules
      |           |__ @myProject
      |                       |___ src (symbolic link to myProject/src)
      |___ src  (regular folder)                                            

1. When I created / delete a file under myProject/src:

I noticed that all events emitted from nsfw are associated with myProject/node_modules/@myProject/src, while inotify running in terminal logged events from both the link myProject/node_modules/@myProject/src and the regular folder myProject/src

2. When I edited a file under myProject/src

I noticed that I was getting 2 events emitted from nsfw, one associated with the link myProject/node_modules/@myProject/src, and the other with the regular folder myProject/src, per edit.

feel free to disagree: I believe the events from nsfw for file creation and deletion should cover both the symbolic link and the regular folder, to be consistent with the file edits.

The testing was performed on Ubuntu.

Service shutdown unexpected any time I delete a directory or file and perform another action

I kept on getting this error log Service shutdown unexpected any time I delete a directory or file and perform another action
this my setup exactly the same from the example

import { ipcMain } from "electron";
import nsfw from "nsfw";
export default class {
  constructor(data) {
      this.localDir = data.localDir;
  }
  watch(e) {
    this.emit = e;
    return nsfw(
      this.localDir,
      events => {
        // handles other events
        console.log(events);
      },
      {
        debounceMS: 250,
        errorCallback(errors) {
          //handle errors
          console.log(errors);
        }
      }
    )
      .then(watcher => {
        console.log("Starting watcher");
        this.watcher = watcher;
        return watcher.start();
      })
      .then(() => {
        // we are now watching dir2 for events!
        console.log("Now watching", this.localDir);
      })
      .then(() => {
        // To stop watching
        console.log("Stoping... never");
        // this.watcher.stop();
      });
  }
}

Events are not fired on symlinks when the source file changes

Downstream issue: microsoft/vscode#29631

Repro steps:

1. Watch folder with structure:

   • folder
     • a (textfile)
     • b (symlink -> a)

2. Open b in vim

3. Edit and save b

A change event will fire for a only.

This does appear to be supported by chokidar.

Bug - Assertion failed: 0, file src\win\handle.c, line 71

Sometimes my tests crashed and I have the following error in the console.

Use native library code as a stand alone C++ library

We stumbled over this project when looking for a cross-platform file watcher implementation that is not GPL. This seems to be a pretty cool library but we'd like to use it in a C++ project directly.

Therefore we dug into the code base a bit and identified some necessary steps to allow this:

• decouple the C++ library from the Javascript wrapper API (NSFW class)
• compile the native code into a stand-alone C++ file-watching library (includes and static lib)
• provide a CMakeLists.txt build system (optional - mainly for convenience)

Furthermore, we'd like to suggest some changes to the C++ portion of your project. Particularly:

• modernize the code base with C++11 (or better C++14) constructs
  • f.e. RAII for memory allocation (i.e. to repair some memory leaks)
• use C++11's STL for threading and threadsafe data structures rather than OpenPA

Those changes are not too hard to implement and should not influence the Javascript wrapper at all. We'd be willing to contribute the improvements sketched above but would like to hear your opinion on those thoughts first.

Does that sound like a good idea to you?

Fails to build on node v10

[1/1] ⠈ nsfw: make: *** Waiting for unfinished jobs....
[-/1] ⠈ waiting...
[-/1] ⠈ waiting...
[-/1] ⠈ waiting...
error /tmp/trizen-jambon/code/src/vscode/build/lib/watch/node_modules/nsfw: Command failed.
Exit code: 1
Command: node-gyp rebuild
Arguments: 
Directory: /tmp/trizen-jambon/code/src/vscode/build/lib/watch/node_modules/nsfw
Output:
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | linux | x64
gyp info spawn /usr/bin/python2
gyp info spawn args [ '/usr/lib/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/tmp/trizen-jambon/code/src/vscode/build/lib/watch/node_modules/nsfw/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/usr/lib/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/jambon/.node-gyp/10.0.0/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/home/jambon/.node-gyp/10.0.0',
gyp info spawn args   '-Dnode_gyp_dir=/usr/lib/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/home/jambon/.node-gyp/10.0.0/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/tmp/trizen-jambon/code/src/vscode/build/lib/watch/node_modules/nsfw',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.' ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: Entering directory '/tmp/trizen-jambon/code/src/vscode/build/lib/watch/node_modules/nsfw/build'
  CC(target) Release/obj.target/openpa/openpa/src/opa_primitives.o
  CC(target) Release/obj.target/openpa/openpa/src/opa_queue.o
  AR(target) Release/obj.target/openpa/openpa.a
  COPY Release/openpa.a
  CXX(target) Release/obj.target/nsfw/src/NSFW.o
  CXX(target) Release/obj.target/nsfw/src/Queue.o
  CXX(target) Release/obj.target/nsfw/src/NativeInterface.o
  CXX(target) Release/obj.target/nsfw/src/Lock.o
  CXX(target) Release/obj.target/nsfw/src/linux/InotifyEventLoop.o
  CXX(target) Release/obj.target/nsfw/src/linux/InotifyTree.o
  CXX(target) Release/obj.target/nsfw/src/linux/InotifyService.o
In file included from ../../nan/nan.h:192:0,
                 from ../src/../includes/NSFW.h:5,
                 from ../src/NSFW.cpp:1:
../../nan/nan_maybe_43_inl.h: In function ‘Nan::Maybe<bool> Nan::ForceSet(v8::Local<v8::Object>, v8::Local<v8::Value>, v8::Local<v8::Value>, v8::PropertyAttribute)’:
../../nan/nan_maybe_43_inl.h:112:15: error: ‘class v8::Object’ has no member named ‘ForceSet’
   return obj->ForceSet(isolate->GetCurrentContext(), key, value, attribs);
               ^~~~~~~~
In file included from ../src/../includes/NSFW.h:5:0,
                 from ../src/NSFW.cpp:1:
../../nan/nan.h: In function ‘v8::Local<v8::Value> Nan::MakeCallback(v8::Local<v8::Object>, v8::Local<v8::Function>, int, v8::Local<v8::Value>*)’:
../../nan/nan.h:835:60: warning: ‘v8::Local<v8::Value> node::MakeCallback(v8::Isolate*, v8::Local<v8::Object>, v8::Local<v8::Function>, int, v8::Local<v8::Value>*)’ is deprecated: Use MakeCallback(..., async_context) [-Wdeprecated-declarations]
         v8::Isolate::GetCurrent(), target, func, argc, argv);
                                                            ^
In file included from ../../nan/nan.h:49:0,
                 from ../src/../includes/NSFW.h:5,
                 from ../src/NSFW.cpp:1:
/home/jambon/.node-gyp/10.0.0/include/node/node.h:172:50: note: declared here
                 NODE_EXTERN v8::Local<v8::Value> MakeCallback(
                                                  ^
/home/jambon/.node-gyp/10.0.0/include/node/node.h:88:42: note: in definition of macro ‘NODE_DEPRECATED’
     __attribute__((deprecated(message))) declarator
                                          ^~~~~~~~~~
In file included from ../src/../includes/NSFW.h:5:0,
                 from ../src/NSFW.cpp:1:
../../nan/nan.h: In function ‘v8::Local<v8::Value> Nan::MakeCallback(v8::Local<v8::Object>, v8::Local<v8::String>, int, v8::Local<v8::Value>*)’:
../../nan/nan.h:850:62: warning: ‘v8::Local<v8::Value> node::MakeCallback(v8::Isolate*, v8::Local<v8::Object>, v8::Local<v8::String>, int, v8::Local<v8::Value>*)’ is deprecated: Use MakeCallback(..., async_context) [-Wdeprecated-declarations]
         v8::Isolate::GetCurrent(), target, symbol, argc, argv);
                                                              ^
In file included from ../../nan/nan.h:49:0,
                 from ../src/../includes/NSFW.h:5,
                 from ../src/NSFW.cpp:1:
/home/jambon/.node-gyp/10.0.0/include/node/node.h:165:50: note: declared here
                 NODE_EXTERN v8::Local<v8::Value> MakeCallback(
                                                  ^
/home/jambon/.node-gyp/10.0.0/include/node/node.h:88:42: note: in definition of macro ‘NODE_DEPRECATED’
     __attribute__((deprecated(message))) declarator
                                          ^~~~~~~~~~
In file included from ../src/../includes/NSFW.h:5:0,
                 from ../src/NSFW.cpp:1:
../../nan/nan.h: In function ‘v8::Local<v8::Value> Nan::MakeCallback(v8::Local<v8::Object>, const char*, int, v8::Local<v8::Value>*)’:
../../nan/nan.h:865:62: warning: ‘v8::Local<v8::Value> node::MakeCallback(v8::Isolate*, v8::Local<v8::Object>, const char*, int, v8::Local<v8::Value>*)’ is deprecated: Use MakeCallback(..., async_context) [-Wdeprecated-declarations]
         v8::Isolate::GetCurrent(), target, method, argc, argv);
                                                              ^
In file included from ../../nan/nan.h:49:0,
                 from ../src/../includes/NSFW.h:5,
                 from ../src/NSFW.cpp:1:
/home/jambon/.node-gyp/10.0.0/include/node/node.h:158:50: note: declared here
                 NODE_EXTERN v8::Local<v8::Value> MakeCallback(
                                                  ^
/home/jambon/.node-gyp/10.0.0/include/node/node.h:88:42: note: in definition of macro ‘NODE_DEPRECATED’
     __attribute__((deprecated(message))) declarator
                                          ^~~~~~~~~~
In file included from ../src/../includes/NSFW.h:5:0,
                 from ../src/NSFW.cpp:1:
../../nan/nan.h: In member function ‘v8::Local<v8::Value> Nan::Callback::Call_(v8::Isolate*, v8::Local<v8::Object>, int, v8::Local<v8::Value>*) const’:
../../nan/nan.h:1479:5: warning: ‘v8::Local<v8::Value> node::MakeCallback(v8::Isolate*, v8::Local<v8::Object>, v8::Local<v8::Function>, int, v8::Local<v8::Value>*)’ is deprecated: Use MakeCallback(..., async_context) [-Wdeprecated-declarations]
     ));
     ^
In file included from ../../nan/nan.h:49:0,
                 from ../src/../includes/NSFW.h:5,
                 from ../src/NSFW.cpp:1:
/home/jambon/.node-gyp/10.0.0/include/node/node.h:172:50: note: declared here
                 NODE_EXTERN v8::Local<v8::Value> MakeCallback(
                                                  ^
/home/jambon/.node-gyp/10.0.0/include/node/node.h:88:42: note: in definition of macro ‘NODE_DEPRECATED’
     __attribute__((deprecated(message))) declarator
                                          ^~~~~~~~~~
../src/NSFW.cpp: In static member function ‘static Nan::NAN_METHOD_RETURN_TYPE NSFW::JSNew(Nan::NAN_METHOD_ARGS_TYPE)’:
../src/NSFW.cpp:159:49: error: no matching function for call to ‘v8::Function::NewInstance()’
     info.GetReturnValue().Set(cons->NewInstance());
                                                 ^
In file included from /home/jambon/.node-gyp/10.0.0/include/node/node.h:63:0,
                 from ../../nan/nan.h:49,
                 from ../src/../includes/NSFW.h:5,
                 from ../src/NSFW.cpp:1:
/home/jambon/.node-gyp/10.0.0/include/node/v8.h:3848:44: note: candidate: v8::MaybeLocal<v8::Object> v8::Function::NewInstance(v8::Local<v8::Context>, int, v8::Local<v8::Value>*) const
   V8_WARN_UNUSED_RESULT MaybeLocal<Object> NewInstance(
                                            ^~~~~~~~~~~
/home/jambon/.node-gyp/10.0.0/include/node/v8.h:3848:44: note:   candidate expects 3 arguments, 0 provided
/home/jambon/.node-gyp/10.0.0/include/node/v8.h:3851:44: note: candidate: v8::MaybeLocal<v8::Object> v8::Function::NewInstance(v8::Local<v8::Context>) const
   V8_WARN_UNUSED_RESULT MaybeLocal<Object> NewInstance(
                                            ^~~~~~~~~~~
/home/jambon/.node-gyp/10.0.0/include/node/v8.h:3851:44: note:   candidate expects 1 argument, 0 provided
../src/NSFW.cpp:177:54: warning: ‘v8::String::Utf8Value::Utf8Value(v8::Local<v8::Value>)’ is deprecated: Use Isolate version [-Wdeprecated-declarations]
   v8::String::Utf8Value utf8Value(info[1]->ToString());
                                                      ^
In file included from /home/jambon/.node-gyp/10.0.0/include/node/v8.h:26:0,
                 from /home/jambon/.node-gyp/10.0.0/include/node/node.h:63,
                 from ../../nan/nan.h:49,
                 from ../src/../includes/NSFW.h:5,
                 from ../src/NSFW.cpp:1:
/home/jambon/.node-gyp/10.0.0/include/node/v8.h:2822:28: note: declared here
                   explicit Utf8Value(Local<v8::Value> obj));
                            ^
/home/jambon/.node-gyp/10.0.0/include/node/v8config.h:318:3: note: in definition of macro ‘V8_DEPRECATED’
   declarator __attribute__((deprecated(message)))
   ^~~~~~~~~~
make: *** [nsfw.target.mk:122: Release/obj.target/nsfw/src/NSFW.o] Error 1
make: *** Waiting for unfinished jobs....
make: Leaving directory '/tmp/trizen-jambon/code/src/vscode/build/lib/watch/node_modules/nsfw/build'
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/lib/node_modules/node-gyp/lib/build.js:258:23)
gyp ERR! stack     at ChildProcess.emit (events.js:182:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:225:12)
gyp ERR! System Linux 4.16.6-1-ARCH
gyp ERR! command "/usr/bin/node" "/usr/bin/node-gyp" "rebuild"
gyp ERR! cwd /tmp/trizen-jambon/code/src/vscode/build/lib/watch/node_modules/nsfw
gyp ERR! node -v v10.0.0

Error: EBUSY: resource busy or locked, unlink

hi

Watching an image (windows10) directory works perfect Events are triggered and processed. I process an imagem after completion i want to delete or move it.
So here is what i do
on the event i place the filename in a queue.
The file is processed (smaller image is created, uploaded)
after that is completed i try to delete the file with:

fs.remove(filename, function(err){
resolve(err);
}, function(err) {
reject(err);
});

the following error is thrown:

{ Error: EBUSY: resource busy or locked, unlink 'c:\images\IMG_0049.JPG'
errno: -4082,
code: 'EBUSY',
syscall: 'unlink',
path: 'c:\images\IMG_0049.JPG' }

I found some notes about antivirus so i already excluded the images path from windows defender. There is no other windows antivirus installed.

same module is working on linux without a problem.

any ideas?

thank you!

I can not compile on windows ia86

Node V: 8
npm install --global --production windows-build-tools OK

Error in npm install:

https://imgur.com/a/aiBmwu4

Not working with windows share folder on macOS

Not working when the pa windows shared folder on macOS, nsfw watch cannot receive any callback when file be modified or removed. Does nsfw suupport that ?

"Service shutdown unexpectedly" when deleting the root watched folder

See microsoft/vscode#29590, the repro is pretty easy in VS Code. Just enable NSFW and then delete the root folder.

Handle windows I/O cancellation and errors better

https://msdn.microsoft.com/en-us/library/windows/desktop/aa363789(v=vs.85).aspx

When exiting ReadDirectoryChangesW check the error for ERROR_OPERATION_ABORTED and do not set error.

Perhaps setting an error code would be more informative as well for other cases.

Is development still active?

Last commit I found was on Oct 27, 2017 and then there was one merge towards the end of the year. That's almost a year. Just wondering if it's worth trying to incorporate nsfw in 08/2018. If it is, I had a couple issues I'd list.

How does this compare to Chokidar?

I wasn't able to form a conclusion from the README of how this differs from Chokidar and in what circumstances I would choose one over the other. A brief comparison would be useful.

Failed to compile

SO= Win 10 64bits
Node Version: v8.11.4
Version: nsfw 1.1.1

When renaming file, is a create event fires for the old filename (Windows)

using Node 8.0.0

What i'm doing:
Renaming a file from file2.txt -> file3.txt

What happens
2 events fire:
a rename event, and a create event for the old filename,

Events:

changes { action: 3,
  directory: 'C:\\Users\\Mikkel\\Desktop\\Watchthis\\customer',
  oldFile: 'file2.txt',
  newFile: 'file3.txt' }
-----------------
changes { action: 0,
  directory: 'C:\\Users\\Mikkel\\Desktop\\Watchthis\\customer',
  file: 'file2.txt' }

I'm pretty sure that this is not what we want. :)

How to ignore certain subdirectories?

I am using this package to watch a Node.js project which has some 50k files in node_modules. I haven't been able to find an ignore or exclude parameter in the documentation.

Any ideas how to ignore node_modules or some other dir in subdirs?

Cheers

Win10: Moving a folder that contains files to watched directory does not trigger event for files

Hi,
I have electron app where I'm watching one directory (+ subdirs).

• If I try to move there (CTRL + X & CTRL + V) folder with couple of files inside from other destination on my PC, the nsfw triggers event just for folder, not for files included within.

• If I try to copy (CTRL + C & CTRL + V) the same folder with couple of files inside, everything works as expected

This behavior was reproduced on 2 PCs with WIN10, on WIN8 works normally.

• I tried to update to newest version 1.2.2
• tried to play with debounceMS
• tried to find anything in errorCallback, no luck

Did anybody came across this issue?
Thanks

How to watch a nonexistent file?

nsfw throws 'Path must be a valid path to a file or a directory.' if the base path does not exist.

Windows shuts down from 'too many events at once'

When receiving excess of over 8k events within a short time frame, Windows can run out of buffer space. While annoying, NSFW does exit safely and can be restarted by stopping / starting it again.

Circular symlinks max out the CPU

Users are reporting in VS Code that circular symlinks are not handled gracefully microsoft/vscode#36307

Steps to Reproduce:

1. Set "files.useExperimentalFileWatcher": true and close VS Code
2. Create a symlink to root directory (must be an absolute link, not relative!):

$ mkdir loop
$ ln -s / loop/root

3. Open directory containing that symlink in VS Code:

$ code --disable-extensions loop

4. Watch processes - one of vs code processes takes high CPU

OSX duplicate create events

When watching files on OSX, NSFW cannot determine the difference between created and modified within 1 second of file creation. If you create a file, and modify it within the same second, that file will be reported as created twice.

Determine multiple file transfer has completed

Would it be possible to know if a file transfer is completed?
For example if you are running over a slow SFTP connection and you are loading multiple files at once, is there a way to know that the entire transaction has completed? I am seeing the CREATED event and the MODIFIED events coming through and I am able to put in the delay to batch the events, but the problem would come in if the transfer of all the files takes longer than the delay for the batching.

Support for monitoring multiple paths, resuming after interruption?

Hi,

Similarly to apparently many others here I'm looking at NSFW as a potential solution to use natively in C++.

• Looking at the source code, it doesn't seem like NSFW provides support for monitoring multiple paths at the same time, aside from perhaps creating multiple instances of the FileSystemWatcher (assuming the corresponding pull request eventually gets merged).
  Would that approach work, or would the multiple instances end up interfering with each other?

• Another question relates to the issue of detecting changes to a monitored path that might have occurred while the monitoring was off: Is it at all possible to "save a snapshot" of the state of the system when turning off the monitoring, so that next time it is turned on it becomes aware of all changes that have occurred since that snapshot has been saved?

I really appreciate your feedback.
Cheers!

Mkdir -p are not correctly reported on Linux

Description

When creating recursively directories with mkdir -p, the watcher only reports the first directory.

Steps to Reproduce

1. Install the CLI from #59
2. Run nsfw /tmp
3. Execute mkdir -p /tmp/foo/bar/baz

Expected behavior:

The watcher says that directories foo, foo/bar and foo/bar/baz have been created:

Created: /tmp/foo
Created: /tmp/foo/bar
Created: /tmp/foo/bar/baz

Actual behavior:

The watcher only says that the directory foo has been created:

Created: /tmp/foo

Additional Information

As far as I know, inotify is not recursive, and it's nsfw code that creates sub-watchers for directories inside the watched directory. When inotify sends the event for foo, the directories foo/bar and foo/bar/baz have already been created, so even if the watcher is able to find these directories to tell inotify to watch them, it fails to also send an event for them.

watch failed in docker mount case

when i watch a folder, then run a docker mount this folder, and do some install task in it, i found it not react the change.

How about a CLI?

e.g chokidar-cli. Would be especially useful in npm scripts…

nsfw prevents Electron v4 apps from starting on Windows

nsfw currently specifies "win_delay_load_hook": "false" in its binding.gyp file. As of Electron v4, specifying this option/value combination prevents any app that uses nsfw from running on Windows; the app simply fails to start. Relevant explanation is here: https://electronjs.org/blog/electron-4-0#win_delay_load_hook

file watch failed on Ubuntu

I use nsfw to watch a directory on Ubuntu, the nsfw watcher can't receive any event when I add or delete a file whose name starts with Chinese character.

lazy symbol binding failed: Symbol not found

When I try to start nsfw in Electron it crashes with the following message:

dyld: lazy symbol binding failed: Symbol not found: __ZN2v811HandleScope12CreateHandleEPNS_8internal10HeapObjectEPNS1_6ObjectE
  Referenced from: /project/node_modules/nsfw/build/Release/nsfw.node
  Expected in: flat namespace

dyld: Symbol not found: __ZN2v811HandleScope12CreateHandleEPNS_8internal10HeapObjectEPNS1_6ObjectE
  Referenced from: /project/node_modules/nsfw/build/Release/nsfw.node
  Expected in: flat namespace

electron: 4.0.1,
nsfw: 1.2.0,
node: v10.15.0

It compiled without problems, any idea of what could be causing this?

Keep watching after finding file

I'd like to start by saying this is an awesome module. Gotta love open-source.

I'm looking for a specific file when filesystem changes occur. This works, once. After the file is found, my code doesn't update when further changes are made in the specified folder. Does anyone have insight into what I may be doing wrong? Here is my full code:

function run() {
  let watcher2;

  nsfw("/path/to/master/folder", events => {
    if (events === undefined) return;

    let directory = events[0].directory;
    directory = directory.split("/").pop(); // gets the actual folder name

    const event = events[0];
    const file = events[0].file;

    if (!matcher([directory, file], blacklist).length) { // If a file/folder changed and it does not match the blacklist (custom)
      log(`Detected change...`);

      const filterFn = function (item) {
        if ( // Ignore some files and folders
          item.path.indexOf("node_modules") < 0 &&
          item.path.indexOf(".git") < 0
        ) return item;
      };

      let paths;

      try {
        paths = klawSync(event.directory, { // Look in the directory the change occurred in
          filter: filterFn,
          nodir: true,
          noRecurseOnFailedFilter: true
        });
      } catch (err) {
        log(`Error:\n${err}`);
      }

      for (let path of paths) {
        const file = path.path.split("/").pop();

        if (file === "file.ext") {
          log(path.path); // Display file
          // this is where my code stops
        }
      }
    }

    /*

      NSFW actions:
        0 = created
        1 = deleted
        2 = modified
        3 = renamed

    */

  },
  {
    debounceMS: 250,
    errorCallback(errors) {
      log(`Errors:\n${errors}`);
    }
  }).then(watcher => {
    watcher2 = watcher; return watcher.start();
  }).then(() => {
    log(`Running`);
  });
}

how to stop watching

hi, is there any way to stop watching files with this module? docs are lacking or perhaps i'm not seeing them

Renaming a file does not returns "RENAMED" (3) action

When renaming a file, no RENAMED (3) action is returned.
Instead a MODIFIED (2) action and a DELETED (1) action is returned.

events:  [
  {
    action: 2,
    directory: '/Users/andersjramsay/dev/testing_project/ctf_test_project/src',
    file: 'initialFOO.ts'
  },
  {
    action: 1,
    directory: '/Users/andersjramsay/dev/testing_project/ctf_test_project/src',
    file: 'initial.ts'
  }
]

I created a minimal repo which reproduces the above: https://github.com/andersr/filewatcher-test
Running on OSX, using Node v12.4.0

Does this module support remote shares?

Hello,

I just came up to this package while looking for a native inotify module due to performance issues with chokidar, and anything which relies on fs.watch. The idea of packing the events is really what I was looking for. The only current limitatiion I have so far is that currently in chokidar I can configure some watchers to poll (which relies on fs.watchFile) when the destination is a samba or nfs share. From what I've seen so far this module does not support that option, is that correct?

Thanks!

macOS: undoing a folder delete does not report folder as added

Steps:

• be on macOS X
• start the watcher on a folder that includes other folders
• delete the folder via finder
• undo this operation

=> the watcher reports the folder as MODIFIED after the undo
=> I would expect the folder event to be reported as CREATED
=> I would also expect CREATED events for each child inside

Output:

[ { action: 1,
    directory: '/Users/bpasero/Desktop/test-js',
    file: 'test' } ]
[ { action: 2,
    directory: '/Users/bpasero/Desktop/test-js',
    file: '.DS_Store' },
  { action: 2,
    directory: '/Users/bpasero/Desktop/test-js',
    file: 'test' } ]

Related issue in VS Code: microsoft/vscode#29766

Linux: file move does not contain enough information to find out the target

When moving a file on Linux using the native file manager from one folder to another, the events that NSFW emits are:

{ action: 3,
    directory: '/home/ticino/development/vscode',
    oldFile: 'CODE_OF_CONDUCT.md',
    newFile: 'CODE_OF_CONDUCT.md' 
}

That however is not sufficient information to know where the file was moved to so it is impossible for a listener to properly react on this change. I would expect the newFile to contain the information to which folder the file was moved to.

Consider shipping individual lodash modules to reduce lib size

The full version of lodash takes 4.9mb uncompressed but nsfw is only using isInteger and isUndefined. It would be better to ship the individual modules, eg. https://www.npmjs.com/package/lodash.isinteger

Uncaught Error: Module did not self-register.

I'm trying to use nsfw my my NWJS project, but i keep getting this error:

Uncaught Error: Module did not self-register.
at Object.Module._extensions..node (module.js:640:18)
at Module.load (module.js:512:32)
at tryModuleLoad (module.js:471:12)
at Function.Module._load (module.js:463:3)
at Module.require (module.js:522:17)
at require (internal/module.js:20:19)
at Object. (node_modules\nsfw\lib\src\index.js:3:16)
at Module._compile (module.js:595:32)
at Object.Module._extensions..js (module.js:610:10)
at Module.load (module.js:512:32)