yuanchuan / node-watch Goto Github PK
View Code? Open in Web Editor NEWA wrapper and enhancements for fs.watch
Home Page: https://npm.im/node-watch
License: MIT License
A wrapper and enhancements for fs.watch
Home Page: https://npm.im/node-watch
License: MIT License
When a Watcher for home
(recursive) receives an event about home/c/sub
being removed (file or directory), it calls close(fullPath)
.
The logic in Watcher.prototype.close
is mostly correct. It understands fullPath
and only closes fs.FSWatcher
objects that correlate to the fullPath
. And, if the removed thing was just a file, then it means there is no FSWatcher
object for a directory, and nothing happens.
So far so good.
Then, the last line in the close()
function is:
this._isClosed = true;
process.nextTick(emitClose, this);
After this, the internal FSWatcher stays open (which is good, because the directory was not really removed, only a file inside of it). But, the Watcher now thinks it is closed!
This problem only affects Linux, because on macOS and Windows, the recursive watcher works natively, and this logic is never used.
I found this problem because of a combination with a problem of Node.js internally.
The internal problem is nodejs/node#25301. In a nut shell: When removing directory home/c
, there are three events:
In my bug fix for #69, I separate the internal delay from the external delay. That means the callback
in Watcher.prototype.add
runs immediately when an event happens.
The first event is for home/c/
which thinks home/c/c
was removed. It checks if home/c/c
exists, sees that it doesn't exist, and calls close(fullPath='home/c/c')
, which sets _isClosed = true
.
The second event ...
The third event is for home/
which thinks home/c
was removed. This is the event we want. But... the callback
checks isClosed()
and then returns early. There is no emit('change')
.
This caused the unit test for watch for directories / should watch directories inside a directory
to fail, because no event for home/c
is emitted.
So before I can fix #69, I will try to reproduce and fix this one first :)
Hey!
Great tool by the way.
So this is my script:
'use strict';
// Imports
const watch = require('node-watch');
// Start the watcher
const watcher = watch('**/*');
console.log("Starting server watch...");
// Wait for changes...
watcher.on('change', file => {
console.log("foo");
});
// Was there an error?
watcher.on('error', err => console.error(err));
This is the script I'm trying to run.
Is watcher supposed to be left open? At the moment, with this script, the watcher doesn't actually "watch" it just closes right at the end of the script. Is there any option to keep the watcher open?
It looks like the recursive
option for fs.watch
is available only on macos and Windows:
if (!(common.isOSX || common.isWindows))
common.skip('recursive option is darwin/windows specific');
If we know this, why not just check the platform quickly, instead of doing an elaborate runtime check? The nodejs tests use these helpers:
exports.isWindows = process.platform === 'win32';
exports.isOSX = process.platform === 'darwin';
The code for hasNativeRecursive is pretty extensive:
Is there a reason why the quick checks aren't used?
Thanks!
I've been considering switching over from chokidar to this due to a few issues, and having 0 dependencies is actually somewhat appealing to me.
But I'm missing a few features:
Currently I have a few dirty workarounds for some of the things above, but it'd be nice if node-watch can have these built-in.
@yuanchuan I can't even get this running on windows. I tried the examples from the readme. Nothing happends.
I tried on Mac, and it works. However I noticed
The watch function returns a fs.FSWatcher like object as the same as fs.watch
let watcher = watch('file', () => {});
watcher.on('close', () => {
// this never gets called
console.log('close event');
});
watcher.close();
I have tried creating a symlink like this:
── example
│ ├── sym.jade -> ../../temp/symlink.jade
│ └── test.jade
and then I watched example/
however, instead of test.jade, when I modified sym.jade, watch.js didn't trigger file changed.
my watch.js is like this:
watch('example', { recusive: true, followSymLinks: true, maxSymLevel: 3 },
filter(/\.jade$/, function(filename) {
console.log(filename, ' changed.');
}));
Hi. Is there any possibility to write something like that:
watch('myDir/*/', filter(/\.js$/, function(filename) {
//
}));
It means that i've got many folders in myDir and i want to watch for js-files in them. Can i use a pattern in myDir value?
When you use recursive: true and watch a file together with some folders, it seems the lstat is throwing an error..
Is it somehow possible to know, when deleting a directory, the list of files and subdirectories deleted with it, or at least if it was a file or a directory?
The : should be a ; on the following line in the second FAQ question:
fn(filename):
On Mac OS X El Capitan I listen for a directory "upload" recursively.
Using finder (also tried to do it via console), I copied "abbrev" folder with some files in it to "upload/node_modules" directory.
node-watch only detected that upload/node_modules/abbrev changed (no file changes inside abbrev directory)
But sometimes node-watch also detects files inside copied directory
The hardcoded delay of 200ms in has-native-recursive.js means that on Linux, there are is no watcher existing during the first 200ms after calling watch(dir, options)
.
This is okay, but I think it would be good for users of node-watch to not have to hardcode this number in their own code.
Instead, perhaps we can emit a ready
event once this check has finished and consumers can expect events to fire.
Is there any option can be used to configure the dirs I don't want to watch like node_modules
?
During #73, I was adding on('ready')
to various tests, and they worked everywhere, except for the two tests that use composeWatcher
.
Waiting for ready
on a composed watcher times out, because I forgot to add support for it in #71. I did not realise...
I can fix this, but wanted to focus on #69 first, so I'm writing it down here for later :)
not always emit "change" because is.file return "undefined" - exception: wrong file path
replace
normalizeCall(**fname**, options, cb, watcher);
with
normalizeCall(**fpath**, options, cb, watcher);
It is not possible to Watch changes to 'files' or 'directoriies' in the onewire directory used by owfs. FS has the same problem, but files can be read and written to without problmes (in fs).
Any explanation?
Hi,
I want to know is there a way I can pass config from json file ?
I wrote json file however not sure how to pass the regular expression from json file.
When I use below code it is not working
"watchConfig": {
"recursive": true,
"filter": "/\\.tests.ts$/"
},
It seems in watch.js on like 23 throws an error in situtation where files are moving at light-speed:
if (fs.existsSync(fpath)) {
memo.push(fpath, method);
return stat(fpath)['is' + shortcuts[method]]();
We get ENOENT errors even though the existsSynch succeed. It happens very seldom but it does happen. When it does happen, the application crashes complete. Should a simple try{} catch(){} around the watch function capture this and not affect watch from continuing?
Is there a way to avoid following symbolic links from the directory being watched?
Hi @yuanchuan,
You might be planning to already, but would be nice to release the recent commits soon for use in qunitjs/qunit.
I've already landed use of [email protected]
, but would feel more comfortable with the adoption if these recent changes are also applied there before releasing the next version of QUnit.
Thanks for everything!
It would be great to have options like select
and reject
.
select
is a regular expression or an array of file extensions to be watched (by default any)
reject
is a regular expression or an array of file extensions to be not watched (by default none).
Example Codes:
watch('myCustomDir',{
filter: function(filename){
return /\.js$/.test(filename);
}
},function(filename){
// do something else ...
})
What I mean is to watch all the js file in the custom directory, but the function just return with nothing.
I walk through the source code, and found these lines at the beginning of the watch function:
var skip = watcher.closed || !options.filter(fpath) || (
is.sym(fpath) && !(options.followSymLinks && options.maxSymLevel--)
);
if (skip) return;
Which means: The path given must suit the filter, or it will return directly.
But should the filter here only affect only on the changed files/directories ? In other words, removing the !options.filter(fpath)
condition may be a better choice here ?
The following code
watch('foo.bar',function(fn){
console.log(fn,'first listener');
});
watch('foo.bar',function(fn){
console.log(fn,'second listener');
});
Will result:
foo.bar first listener
second listener not firing
Hi, on my system I'm doing a quick fix to allow for unwatching and figured you might be interested in what I'm doing. I'm just returning the fs.watch so the user can perform a close on that. We might as well have the function return something.
If you get ENOSPC error, but you actually have free disk space - it means that your OS watcher limit is too low and you probably want to recursively watch a big tree of files.
Follow this description to increase the limit:
https://confluence.jetbrains.com/display/IDEADEV/Inotify+Watches+Limit
@yuanchuan my suggestion would be to notice this somewhere in README to avoid confusion.
It seems on Linux, once a new file is created, changes to it are not reported correctly.
I encountered this issue when attempting to use node-watch
inside QUnit (for its CLI watch mode), as some of our unit tests weren't passing.
I think there needs to be a way to remove existing recursive watch and I do not see anything like this in the API. Please advise.
NodeJS's fs.watch
supports a { persistent: false }
flag which can be used to stop the watcher stalling the event loop (and stopping the program from exiting).
node-watch should support passing persistent:false through its options to fs.watch
The following snippet will give me double change events if I change a file in the files
directory:
const watcher = watch([".", "files/"], { recursive: true });
Is this by design or a bug? Both the patterns will match on a file in the files
directory, but I would expect that duplicate events are removed.
Sample case:
------------ test script -----------
events = require 'events'
event = new events.EventEmitter()
watch = require 'watch'
fileExists = (stat) ->
return false unless stat
return stat.dev or stat.mode
BASEDIR = "./root"
watch.watchTree BASEDIR, { ignoreDotFiles: on }, (f, curr, prev) ->
if typeof(f) is 'object'
for path of f
event.emit '#set', path
return
event.emit '#del', f if fileExists(prev) and not fileExists(curr)
event.emit '#set', f
return event.on '#del', (file) -> console.log "del #{file}"
event.on '#set', (file) -> console.log "set #{file}"
------------ test script end --------
How to reproduce:
./root/FOO/BAR/BAZ/BUZ
./root/FOO/BAR/BAZ
./root/FOO/BAR
./root/FOO
set ./root
set root/FOO
set root/FOO
Problem: seems like some events are getting triggered twice. Furthermore, watch didn't pick up the subfolders.
Thanks
JM
I get the following error if I delete a directory that is within the directory I am watching. It crashes the node server. I'm on Windows 8 64bit.
events.js:71
throw arguments[1]; // Unhandled 'error'event
Error: watch EPERM
at errnoException (fs.js:806:11)
at FSEvent._handle.onchange (fs.js:824:26)
It happens on both hard delete and deleting to recycle bin, and whether the directory is empty or not.
When using pure fs.watch it does not happen, although oddly fs.watch reports a rename event, not a delete or remove.
Hi,
The watcher callback is triggered even if the .close() method is called. To be exact the callback is triggered after the configured delay.
Not sure if you schedule this callback with a timeOut or something else, but we expect .close() method to clean the timer when triggered.
We detected the problem on our Config package (reported by one user).
SlimIO/Config@5228597
You should be able to reproduce with a case like that:
const watch = require('node-watch');
const config = { delay: 500 };
const watcher = watch('yourfile', config, function() {
console.log('triggered');
});
// Write your file here to trigger the watcher
watcher.close();
// Wait 500ms or more here
// Watcher callback triggered even if closed...
Best Regards,
Thomas
Do you have any plans to implement command line interface for the library? I've been searching for a good cli tool for watching files matching a certain pattern.
If you don't have plans for this, would you accept a cli as pull request?
I im having ffmpeg that is generating upon node start in tmp folder m3u8 files that holds indexes for ts segment files..so i need to watch m3u8 file to file change and then extract indexes from that file.
node-watch works excellent in terms of low cpu usage and instantly it get triggered from OS when m3u8 file changes...that works flawlessly.
But problem is when i start program ffmpeg needs sometime 2seconds to generate m3u8 file sometimes 10seconds...so time creation of m3u8 file is random.
And when i call on start node.js application and run watch for file 1.m3u8 i get this error:
Error: /tmp//function (req, res, next) {
app.handle(req, res, next);
}_.m3u8 does not exist.
So i need if you could rewrite code in watch.js line 371 to return to watch function and to call it again in for example one second...when file will be generated...or if you have better idea you can add it to this line...
here is original code that throws error when file not exists...maybe adding check if file exists and then settimeout to call function again?
if (!is.exists(fpath)) {
watcher.emit('error',
new Error(fpath + ' does not exist.')
);
}
So in short description problem is that watcher exits with error when file does not exists...if you could fix this to if file not exists call watcher again in one second...if it exists continuo to execute code like now
Sample case:
------------ test script -----------
events = require 'events'
event = new events.EventEmitter()
watch = require 'watch'
fileExists = (stat) ->
return false unless stat
return stat.dev or stat.mode
BASEDIR = "./root"
watch.watchTree BASEDIR, { ignoreDotFiles: on }, (f, curr, prev) ->
if typeof(f) is 'object'
for path of f
event.emit '#set', path
return
event.emit '#del', f if fileExists(prev) and not fileExists(curr)
event.emit '#set', f
return event.on '#del', (file) -> console.log "del #{file}"
event.on '#set', (file) -> console.log "set #{file}"
------------ test script end --------
How to reproduce:
set ./root
set ./root/FOO
set ./root
set root/FOO
Problem: watch is trimming "./", but it's not even consistent. It would be better to keep whatever the user supplied.
Thanks
JM
the file change event doesn't be triggered in recursive directory if having filter. eg. dir is /root, change under /root is triggered, but change under /root/doc is not triggered.
`
options =
recursive: true
filter: (name) ->
path.extname(name) is '.jade'
watch dir, options, (eventType, filename) =>
console.log "#{eventType}: #{filename}"
`
There's a specific node package called "blade" that uses a single circular reference in
node_modules/blade/meteor/node_modules/blade/
where the last folder is
-> ../../
this causes watch to endlessly recurse until the stack is blown. There should probably be an option to ignore symlinks, stop at a certain depth, or (better yet) recognize circular symlinks.
Using an async function as callback doesn't works anymore, the function never get called (no warnings/errors either).
This used to work with v0.5.5
node v8.5.0
Hi
How i can run the watch when new folder is added in that watched folder or any folder is removed in it
Test platform: Windows 7.
When delete a file in nodejs 0.12.7 , a TypeError is thrown:
path.js:233
throw new TypeError('Arguments to path.join must be strings');
^
TypeError: Arguments to path.join must be strings
at Object.win32.join (path.js:233:13)
at FSWatcher.<anonymous> (E:\work\game_bible_cms_proj\document\tools\node_mo
dules\node-watch\lib\watch.js:257:26)
at FSWatcher.emit (events.js:110:17)
at FSEvent.FSWatcher._handle.onchange (fs.js:1159:12)
This error happend because the filename arguments is not always provided, see the doc here.
So the path.join(fpath, fname)
will raise that error. It usually occurs when deleting a file.
However, it works fine on node 4.5.0 and node 6.6.0. It may need a fallback logic or friendly warning, or simply drop the support for low version nodejs.
When we use this library (which is great), we still have to add some code to fs.stat
the file once an inotify
event occurs.
We must do this because larger files when copied come across the TCP pipe in smaller buffers until the file is completely written. We created some timeout events to make this work. Perhaps some attention can be put into addressing the idea of creating a custom EventEmitter or something of that nature when a file is completely written.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.