Files in new directories not detected on Linux

[yuanchuan]

Files in new directories not detected on Linux

[Krinkle]

When investigating the bug, I think I understand the problem better now. I'll try to explain it.

---

1. The code runs on Linux an calls watch('/home', { recursive: true });

Here, watch() will check is.directory() and call watcher.watchDirectory().

watcher.watchDirectory() will call hasNativeRecursive().

hasNativeRecursive() tests if native fs.watch() supports recursion. After setTimeout(, 200ms) it will callback with false.

Then, watcher.watchDirectory() will continue and call getSubDirectories(), and then make more calls watcher.watchDirectory() until all sub directories are watched. This happens very quickly, but because fs.readdir() is asynchronous, let us say it will take 1ms.

After 201ms, all directories are being watched. Now, step 2.

---

2. The code calls tree.newFile('home/new/file1');. This will synchronously make directory home/new and then synchronously create file file1 inside it.

After the directory and file have been synchronously created, it is now time for Node.js to process the callback for FSWatcher events. There is an event for the FSWatcher object for /home in our code, which says that /home/new is created.

There is no event about /home/new/file1 being created, because we do not have an FSWatcher for /home/new.

Our code debounces the FSWatcher#change callback by 200ms delay.

---

3. The code makes change 1 to tree.modify('home/new/file1');, after 20ms.

There is no event about /home/new/file1 being changed, because we do not have an FSWatcher for /home/new yet.

---

4. The code makes change 2 to tree.modify('home/new/file1');, after 50ms.

Still no FSWatcher to see the event...

---

5. The code makes change 3 to tree.modify('home/new/file1');, after 150ms.

Still no FSWatcher...

---

6. The debounce for FSWatcher#change callback is reached.

The callback is executed, and our code will:

• Check is.directory.
• Call self.watchDirectory() for the new directory, to create an FSWatcher.
• Emit change to the user of node-watch.

---

I propose that we move the debounce logic from the internal FSWatcher#change callback to only the emit('change') for the user of node-watch.

This way, node-watch will create the FSWatcher as soon as it sees the directory created. That means it will see change 1 -2 -3. It also means it will still not see the creating of the file, but I think that is okay. This is only a problem when they are created synchronously (like in tests). In reality, when this is used by programs that help humans develop code (like watching unit tests), they cannot synchronously create a directory and file at the same time.

[Krinkle]

The test case at #70 is passing for my locally with macOS, as well as on Travis with macOS. But, it is failing with Linux on Travis.

I am looking in the code to try and understand this. I see the following:

• Watch home (on Linux)
• Create directory home/new and create home/new/foo.
• Native fs.watch reports home/new as a new directory.
• node-watch waits until delay.
• After the delay is complete, a new watcher is created for home/new.

But, at this time, the file home/new/foo has already been created and changes to it were not detected. When using a filter for a file extension (e.g. \.js$) this means the file will never be found unless it is modified again after the delay.

I would like to help to solve this. What solution would you prefer? Perhaps we can separate the internal delay and external delay. So, that the new watcher is created a soon as possible.

[yuanchuan]

Forgive me, I don't quite follow my own code. It seems you understand better than me now and that's great!

I have just lost by laptop tonight (so sad) and I couldn't make some tests for some time.

I think I can trust you to make better decisions. Is it okay to add you as a collaborator ?

[Krinkle]

@yuanchuan Oh no, I'm so sorry to hear that!

I'll be poking around this issue a bit over the holidays but don't feel any pressure on this, it's not urgent :)

I'd love to help out as a collaborator, but I would also prefer to receive review on my patches. It's a principle I follow in most of my projects to make sure at least two people share and understand how a specific part of the project works.