e-dant/watcher

Filesystem watcher. Works anywhere. Simple, efficient and friendly.

Watcher

Quick Start

#include "wtr/watcher.hpp"
#include <iostream>
#include <string>

using namespace std;
using namespace wtr;

// The event type, and every field within it, has
// string conversions and stream operators. All
// kinds of strings -- Narrow, wide and weird ones.
// If we don't want particular formatting, we can
// json-serialize and show the event like this:
//   some_stream << event
// Here, we'll apply our own formatting.
auto show(event e) {
  cout << to<string>(e.effect_type) + ' '
        + to<string>(e.path_type)   + ' '
        + to<string>(e.path_name)
        + (e.associated ? " -> " + to<string>(e.associated->path_name) : "")
       << endl;
}

auto main() -> int {
  // Watch the current directory asynchronously,
  // calling the provided function on each event.
  auto watcher = watch(".", show);

  // Do some work. (We'll just wait for a newline.)
  getchar();

  // The watcher would close itself around here,
  // though we can check and close it ourselves.
  return watcher.close() ? 0 : 1;
}

# Sigh
PLATFORM_EXTRAS=$(test "$(uname)" = Darwin && echo '-isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -framework CoreFoundation -framework CoreServices')
# Build
eval c++ -std=c++17 -Iinclude src/wtr/tiny_watcher/main.cpp -o watcher $PLATFORM_EXTRAS
# Run
./watcher

modify file /home/e-dant/dev/watcher/.git/refs/heads/next.lock
rename file /home/e-dant/dev/watcher/.git/refs/heads/next.lock -> /home/e-dant/dev/watcher/.git/refs/heads/next
create file /home/e-dant/dev/watcher/.git/HEAD.lock

Enjoy!

---

Tell Me More

A filesystem event watcher which is

1. Friendly

I try to keep the 1623 lines that make up the runtime of Watcher relatively simple and the API practical:

auto w = watch(path, [](event ev) { cout << ev; });

wtr.watcher ~

2. Modular

Watcher may be used as a library, a program, or both. If you aren't looking to create something with the library, no worries. Just use ours and you've got yourself a filesystem watcher which prints filesystem events as JSON. Neat. Here's how:

# The main branch is the (latest) release branch.
git clone https://github.com/e-dant/watcher.git && cd watcher
# Via Nix
nix run | grep -oE 'cmake-is-tough'
# With the build script
tool/build --no-build-test --no-run && cd out/this/Release # Build the release version for the host platform.
./wtr.watcher | grep -oE 'needle-in-a-haystack/.+"' # Use it, pipe it, whatever. (This is an .exe on Windows.)

3. Efficient

You can watch an entire filesystem with this project. In almost all cases, we use a near-zero amount of resources and makes efficient use of the cache. We regularly test that the overhead of detecting and sending an event to the user is an order of magnitude less than the filesystem operations being measured.

4. Well Tested

We run this project through unit tests against all available sanitiziers. This code tries hard to be thread, memory, bounds, type and resource-safe. What we lack from the language, we try to make up for with testing. For some practical definition of safety, this project probably fits.

5. Dependency Minimal

Watcher depends on the C++ Standard Library. For efficiency, we use System APIs when possible on Linux, Darwin and Windows. For testing and debugging, we use Snitch and Sanitizers.

6. Portable

Watcher is runnable almost anywhere. The only requirement is a filesystem.

---

Usage

Project Content

The important pieces are the (header-only) library and the (optional) CLI program.

• Library: include/wtr/watcher.hpp. Include this to use Watcher in your project.
• Program: src/wtr/watcher/main.cpp. Build this to use Watcher from the command line.

A directory tree is in the notes below.

The Library

Copy the include directory into your project. Include watcher like this:

#include "wtr/watcher.hpp"

The event and watch headers are short and approachable. (You only ever need to include wtr/watcher.hpp.)

There are two things the user needs:

• The watch function
• The event object

watch takes a path, which is a string-like thing, and a callback, with is a function-like thing. Passing watch a character array and a lambda would work well.

Typical use looks like this:

auto watcher = watch(path, [](event ev) { cout << ev; });

watch will happily continue watching until you stop it or it hits an unrecoverable error.

The event object is used to pass information about filesystem events to the (user-supplied) callback given to watch.

The event object will contain:

• associated, another event, associated with this one, such as a renamed-to path. (This is a recursive structure.)
• path_name, which is an absolute path to the event.
• path_type, the type of path. One of:
  • dir
  • file
  • hard_link
  • sym_link
  • watcher
  • other
• effect_type, "what happened". One of:
  • rename
  • modify
  • create
  • destroy
  • owner
  • other
• effect_time, the time of the event in nanoseconds since epoch.

The watcher type is special.

Events with this type will include messages from the watcher. You may recieve error messages or important status updates, such as when it first becomes alive and when it dies.

The last event will always be a destroy event from the watcher. You can parse it like this:

bool is_last = ev.path_type == path_type::watcher
            && ev.effect_type == effect_type::destroy;

Happy hacking.

Your Project

This project tries to make it easy for you to work with filesystem events. I think good tools are easy to use. If this project is not ergonomic, file an issue.

Here is a snapshot of the output taken while preparing this commit, right before writing this paragraph.

{
  "1666393024210001000": {
    "path_name": "./watcher/.git/logs/HEAD",
    "effect_type": "modify",
    "path_type": "file"
  },
  "1666393024210026000": {
    "path_name": "./watcher/.git/logs/refs/heads/next",
    "effect_type": "modify",
    "path_type": "file"
  },
  "1666393024210032000": {
    "path_name": "./watcher/.git/refs/heads/next.lock",
    "effect_type": "create",
    "path_type": "other"
  }
}

Which is pretty cool.

A capable program is here.

Consume

This project is accessible through:

• Conan: Includes the header
• Nix: Provides isolation, determinism, includes header, cli, test and benchmark targets
• Bazel: Provides isolation, include header and cli targets
• tool/build: Includes header, cli, test and benchmark targets
• CMake: Includes header, cli, test and benchmark targets
• Just copying the header file

Conan

See the package here.

Nix

nix build # To just build
nix run # Build the default target, then run without arguments
nix run . -- / | jq # Build and run, watch the root directory, pipe it to jq
nix develop # Enter an isolated development shell with everything needed to explore this project

Bazel

bazel build cli # Build, but don't run, the cli
bazel build hdr # Ditto, for the single-header
bazel run cli # Run the cli program without arguments

tool/build

tool/build
cd out/this/Release

# watches the current directory forever
./wtr.watcher
# watches some path for 10 seconds
./wtr.watcher 'your/favorite/path' -s 10

This will take care of some platform-specifics, building the release, debug, and sanitizer variants, and running some tests.

CMake

cmake -S . -B out
cmake --build out --config Release
cd out

# watches the current directory forever
./wtr.watcher
# watches some path for 10 seconds
./wtr.watcher 'your/favorite/path' -s 10

Bugs & Limitations

Safety and C++

I was comfortable with C++ when I first wrote this. I later rewrote this project in Rust as an experiment. There are benefits and drawbacks to Rust. Some things were a bit safer to express, other things were definitely not. The necessity of doing pointer math on some variably-sized opaque types from the kernel, for example, is not safer to express in Rust. Other things are safer, but this project doesn't benefit much from them.

Rust really shines in usability and expression. That might be enough of a reason to use it. Among other things, we could work with async traits and algebraic types for great good.

I'm not sure if there is a language that can "just" make the majority of the code in this project safe by definition.

The guts of this project, the adapters, talk to the kernel. They are bound to use unsafe, ill-typed, caveat-rich system-level interfaces.

The public API is just around 100 lines, is well-typed, well-tested, and human-verifiable. Not much happens there.

Creating an FFI by exposing the adapters with a C ABI might be worthwhile. Most languages should be able to hook into that.

The safety of the platform adapters necessarily depends on each platform's documentation for their interfaces. Like with all system-level interfaces, as long as we ensure the correct pre-and-post-conditions, and those conditions are well-defined, we should be fine.

Platform-specific adapter selection

Among the platform-specific implementations, the FSEvents API is used on Darwin and the ReadDirectoryChanges API is used on Windows. There is some extra work we do to select the best adapter on Linux. The fanotify adapter is used when the kernel version is greater than 5.9, the containing process has root priveleges, and the necessary system calls are otherwise allowed. The system calls associated with fanotify may be disallowed when inside a container or cgroup, despite the necessary priviledges and kernel version. The inotify adapter is used otherwise. You can find the selection code for Linux here.

The namespaces for our adapters are inline. When the (internal) detail::...::watch() function is invoked, it resolves to one (and only one) platform-specifc implementation of the watch() function. One symbol, many platforms, where the platforms are inline namespaces.

Exceptions to Efficient Scanning

Efficiency takes a hit when we bring out the warthog, our platform-independent adapter. This adapter is used on platforms that lack better alternatives, such as (not Darwin) BSD and Solaris (because warthog beats kqueue).

Watcher is still relatively efficient when it has no alternative better than warthog. As a thumb-rule, scanning more than one-hundred-thousand paths with warthog might stutter.

I'll keep my eyes open for better kernel APIs on BSD.

Ready State

There is no reliable way to communicate when a watcher is ready to send events to the callback.

For a few thousand paths, this may take a few milliseconds. For tens-of-thousands of paths, consider waiting a few seconds.

Unsupported events

None of the platform-specific implementations provide information on what attributes were changed from. This makes supporting those events dependant on storing this information ourselves. Storing maps of paths to stat structures, diffing them on attribute changes, is a non-insignificant memory commitment.

The owner and attribute events are unsupported because I'm not sure how to support those events efficienty.

Unsupported filesystems

Special filesystems, including /proc and /sys, cannot be watched with inotify, fanotify or the warthog. Future work may involve dispatching ebpf programs for the kernel to use. This would allow us to monitor for modify events on some of those special filesystem.

Resource limitations

The number of watched files is limited when inotify is used.

OS APIs Used

Linux

• inotify
• fanotify
• epoll
• eventfd

Darwin

• FSEvents
• dispatch

Windows

• ReadDirectoryChangesW
• IoCompletionPort

Minimum C++ Version

For the header-only library and the tiny-watcher, C++17 and up should be fine.

We might use C++20 coroutines someday.

Cache Efficiency

$ tool/gen-event/dir &
$ tool/gen-event/file &
$ valgrind --tool=cachegrind wtr.watcher ~ -s 30

I   refs:      797,368,564
I1  misses:          6,807
LLi misses:          2,799
I1  miss rate:        0.00%
LLi miss rate:        0.00%

D   refs:      338,544,669  (224,680,988 rd   + 113,863,681 wr)
D1  misses:         35,331  (     24,823 rd   +      10,508 wr)
LLd misses:         11,884  (      8,121 rd   +       3,763 wr)
D1  miss rate:         0.0% (        0.0%     +         0.0%  )

LLd miss rate:         0.0% (        0.0%     +         0.0%  )
LL refs:            42,138  (     31,630 rd   +      10,508 wr)
LL misses:          14,683  (     10,920 rd   +       3,763 wr)
LL miss rate:          0.0% (        0.0%     +         0.0%  )

Namespaces and the Directory Tree

Namespaces and symbols closely follow the directories in the devel/include folder. Inline namespaces are in directories with the - affix.

For example, wtr::watch is inside the file devel/include/wtr/watcher-/watch.hpp. The namespace watcher in wtr::watcher::watch is anonymous by this convention.

More in depth: the function ::detail::wtr::watcher::adapter::watch() is defined inside one (and only one!) of the files devel/include/detail/wtr/watcher/adapter/*/watch.hpp, where * is decided at compile-time (depending on the host's operating system).

All of the headers in devel/include are amalgamated into include/wtr/watcher.hpp and an include guard is added to the top. The include guard doesn't change with the release version. In the future, it might.

watcher
├── src
│  └── wtr
│     ├── watcher
│     │  └── main.cpp
│     └── tiny_watcher
│        └── main.cpp
├── out
├── include
│  └── wtr
│     └── watcher.hpp
└── devel
   ├── src
   │  └── wtr
   └── include
      ├── wtr
      │  ├── watcher.hpp
      │  └── watcher-
      │     ├── watch.hpp
      │     └── event.hpp
      └── detail
         └── wtr
            └── watcher
               ├── semabin.hpp
               └── adapter
                  ├── windows
                  │  └── watch.hpp
                  ├── warthog
                  │  └── watch.hpp
                  ├── linux
                  │  ├── watch.hpp
                  │  ├── sysres.hpp
                  │  ├── inotify
                  │  │  └── watch.hpp
                  │  └── fanotify
                  │     └── watch.hpp
                  └── darwin
                     └── watch.hpp

You can run tool/tree to view this tree locally.

Comparison with Similar Projects

https://github.com/notify-rs/notify:
  lines of code: 2799
  lines of tests: 475
  lines of docs: 1071
  implementation languages: rust
  interface languages: rust
  supported platforms: linux, windows, darwin, bsd
  kernel apis: inotify, readdirectorychanges, fsevents, kqueue
  non-blocking: yes
  dependencies: none
  tests: yes
  static analysis: yes (borrow checked, memory and concurrency safe language)

https://github.com/e-dant/watcher:
  lines of code: 1623
  lines of tests: 881
  lines of docs: 2103
  implementation languages: cpp
  interface languages: cpp, shells
  supported platforms: linux, darwin, windows, bsd
  kernel apis: inotify, fanotify, fsevents, readdirectorychanges
  non-blocking: yes
  dependencies: none
  tests: yes
  static analysis: yes

https://github.com/facebook/watchman.git:
  lines of code: 37435
  lines of tests: unknown
  lines of docs: unknown
  implementation languages: cpp, c
  interface languages: cpp, js, java, python, ruby, rust, shells
  supported platforms: linux, darwin, windows, maybe bsd
  kernel apis: inotify, fsevents, readdirectorychanges
  non-blocking: yes
  dependencies: none
  tests: yes (many)
  static analysis: yes (all available)

https://github.com/p-ranav/fswatch:
  lines of code: 245
  lines of tests: 19
  lines of docs: 114
  implementation languages: cpp
  interface languages: cpp, shells
  supported platforms: linux, darwin, windows, bsd
  kernel apis: inotify
  non-blocking: maybe
  dependencies: none
  tests: some
  static analysis: none

https://github.com/tywkeene/go-fsevents:
  lines of code: 413
  lines of tests: 401
  lines of docs: 384
  implementation languages: go
  interface languages: go
  supported platforms: linux
  kernel apis: inotify
  non-blocking: yes
  dependencies: yes
  tests: yes
  static analysis: none (gc language)

https://github.com/radovskyb/watcher:
  lines of code: 552
  lines of tests: 767
  lines of docs: 399
  implementation languages: go
  interface languages: go
  supported platforms: linux, darwin, windows
  kernel apis: none
  non-blocking: no
  dependencies: none
  tests: yes
  static analysis: none

https://github.com/parcel-bundler/watcher:
  lines of code: 2862
  lines of tests: 474
  lines of docs: 791
  implementation languages: cpp
  interface languages: js
  supported platforms: linux, darwin, windows, maybe bsd
  kernel apis: fsevents, inotify, readdirectorychanges
  non-blocking: yes
  dependencies: none
  tests: some (js bindings)
  static analysis: none (interpreted language)

https://github.com/atom/watcher:
  lines of code: 7789
  lines of tests: 1864
  lines of docs: 1334
  implementation languages: cpp
  interface languages: js
  supported platforms: linux, darwin, windows, maybe bsd
  kernel apis: inotify, fsevents, readdirectorychanges
  non-blocking: yes
  dependencies: none
  tests: some (js bindings)
  static analysis: none

https://github.com/paulmillr/chokidar:
  lines of code: 1544
  lines of tests: 1823
  lines of docs: 1377
  implementation languages: js
  interface languages: js
  supported platforms: linux, darwin, windows, bsd
  kernel apis: fsevents
  non-blocking: maybe
  dependencies: yes
  tests: yes (many)
  static analysis: none (interpreted language)

https://github.com/Axosoft/nsfw:
  lines of code: 2536
  lines of tests: 1085
  lines of docs: 148
  implementation languages: cpp
  interface languages: js
  supported platforms: linux, darwin, windows, maybe bsd
  kernel apis: fsevents
  non-blocking: maybe
  dependencies: yes (many)
  tests: yes (js bindings)
  static analysis: none

https://github.com/canton7/SyncTrayzor:
  lines of code: 17102
  lines of tests: 0
  lines of docs: 2303
  implementation languages: c#
  interface languages: c#
  supported platforms: windows
  kernel apis: unknown
  non-blocking: yes
  dependencies: unknown
  tests: none
  static analysis: none (managed language)

https://github.com/g0t4/Rx-FileSystemWatcher:
  lines of code: 360
  lines of tests: 0
  lines of docs: 46
  implementation languages: c#
  interface languages: c#
  supported platforms: windows
  kernel apis: unknown
  non-blocking: yes
  dependencies: unknown
  tests: yes
  static analysis: none (managed language)

watcher's Issues

[adapter/linux] use an `fanotify` adapter if the user is root

Live Test Cases

Leave comments here for test cases.

I'm looking for interesting ways to attempt to break the watchers.

Our current test suite creates many thousands of files and/or directories. This is done manually while waiting for an error.

The test suite should be expanded and automated.

[test/windows] test and build on both clang and msvc

Which filesystem events correspond to which event::effect_type values?

Hi @e-dant, I am a bit unsure as to what event::effect_type values are set for which filesystem events. This might be due to my lack of experience with filesystem events, or due to my debian bookworm setup (I am using Debian Bookworm with KDE Plasma 5.27.5). Could you clarify a few points, please?

1. When I copy a file into the watched directory with cp new_file watched_dir/ command, 2 events are triggered event::effect_type::create first and then event::effect_type::modify. Is this intended or should only 1 event be set?
2. When I rename a file in the watched directory with mv old_filename new_filename command, only the event::effect_type::rename with the old path is set. How should I get the new filename? Shouldn't a event::effect_type::create be set?
3. Using mv new_file watched_dir/ command to move a file into the watched directory does not trigger any events at all. Shouldn't this set event::effect_type::create event?
4. Using mv filename .. in the watched directory triggers a event::effect_type::rename and not event::effect_type::destroy. Is this intended?
5. When I modify the watched file content event::effect_type::modify is set twice after file modifications are saved. Is this intended?
6. When modifying a file with nano, vim, or kate editors a temporary hidden file (.filename.txt.swp or .filename.txt.kate-swp for kate) is created. This file also triggers watch events, it's not hard to filter them out by checking the event::path_name filename content, but shouldn't these files not trigger any watch events?

Stopping the watcher

Thank you for the library. I have a one comment:

• I would remove the usage of static as you cannot create multiple watchers (the fact that it is an inline function does not change the nature of the static variable)

https://github.com/e-dant/watcher/blob/release/sinclude/watcher/watcher.hpp#L838

  static auto callback_hook = callback; // here
  const auto callback_adapter =
      [](ConstFSEventStreamRef, /* stream_ref
                                   (required) */
         auto*,                 /* callback_info (required) */

You can simply pass the hook in the lambda:

  const auto callback_adapter =
      [&callback_hook = callback](ConstFSEventStreamRef, /* stream_ref
                                   (required) */
         auto*,                 /* callback_info (required) */

• Secondly, I do not see an "API" to stop the watcher. In my code I need to be able to stop the watcher on command. It looks like the main README talks about it but I don't see a way.

`namespace literal` should be removed or (massively) reduced

There is only one function that matters (and only one function that should be accessible) to the user: water::watcher::run.

The header include/watcher/event.hpp may have some types worth exposing as well.

The only items in namespace literal should be the ones mentioned above.

[api] explore an `asio` interface

We should investigate boot::asio in these places:

• The event loops, which currently use epoll (Linux), iocp (Windows) and dispatch_queue (Darwin).
• The user API, which is currently a callback-style interface
• The watch thread: Currently, every invocation of wtr::watch() spawns a new thread. This may be undesirable to some users. While we do test the efficiency of a large number of parallel watchers, and those tests are sufficiently performant, spawning a new thread for each watcher just seems a bit... well, there are better ways to do this.

compilation error on MSVC since 0.8.1

Since 0.8.1, MSVC 19.33 causes compile error.
https://godbolt.org/z/qj6W84KP7

example.cpp
C:\Users\ContainerAdministrator\AppData\Local\Temp\compiler-explorer-compiler2023313-18468-1p7mtrd.iwwy\raw.githubusercontent.com/e-dant/watcher/release/0.8.3/include/wtr/watcher.hpp(191): error C3431: 'kind': a scoped enumeration cannot be redeclared as an unscoped enumeration
C:\Users\ContainerAdministrator\AppData\Local\Temp\compiler-explorer-compiler2023313-18468-1p7mtrd.iwwy\raw.githubusercontent.com/e-dant/watcher/release/0.8.3/include/wtr/watcher.hpp(177): error C3431: 'what': a scoped enumeration cannot be redeclared as an unscoped enumeration
Compiler returned: 2

I do not know the reason for this compile error.
Is there any workaround?

Windows runner no longer executes executables properly

At some point in the last two weeks, it looks like the GitHub hosted Windows runner changed.

For some reason I do not understand, Windows is now failing to execute the test artifacts: https://github.com/e-dant/watcher/actions/runs/6426774299/job/17451441610

It is not clear to me what the error is, or what changed that may have affected this since https://github.com/e-dant/watcher/actions/runs/6140888668

Portability thread

If anyone find themselves in a spot where they need to create compile definitions or patches to get this project working the way it should, please drop a comment here. I'm happy to address the issues.

Some possible requests you can make:

• Create version definitions within the project.

• Create a basic test program (without running the full test suite) to see if the project compiles and/or runs on some platform.

• Fix X thing on Y compiler.

Etc, etc.

Whatever you need.

`include` Directory Conventions

Decide which one you want:

• The "stuttering" include, in which case you should move README.md and .tellfile into include/watcher
• The "shallow" include, in which case you should bring include/watcher/* up one level

[test] Fuzz

iOS CI Secrets

Looking for thoughts and solutions.

This project runs on iOS. I want to integrate iOS into our CI pipeline.

The only method I'm currently aware of is attaching a certificate and provisioning profile to this repository's secrets.

The only such secrets I have are tied to me personally.

That would be fine for now.

The problem I see is complications for future contributors access rights to this repository.

API Changes

This library has become somewhat popular. I expect at least one other user is actively using this project. For them/those users, we need to publish upcoming API changes.

They are not fully fleshed out yet, however, at least these two things may be affected:

1. When we implemented heuristics, the template parameter on watch for delay_ms may be removed.
2. To fully support rename events, either the event.where will be changed to a tuple or another field will be added to the event object.

This will happen before the 1.0 release.

Please provide a way to disable fanotify

I am excited with the 0.5.3 version upgrade that wraps not only inotify but also fanotify.

On the other hand, fanotify is a relatively new feature that will not compile on slightly older kernels.
The 0.5.3 implementation of fanotify uses FAN_REPORT_DFID_NAME, so it seems to require Kernel 5.9 or later to compile.
https://man7.org/linux/man-pages/man2/fanotify_init.2.html

Is it possible to add the following condition for include/watcher/adapter/linux/watch.hpp and include/watcher/adapter/linux/fanotify/watch.hpp?

• Check for the existence of fanotify.h
• Check if FAN_REPORT_DFID_NAME is defined

[test/windows] figure out a way to use any sanitizer at all

Clean Includes

We need to document what we use from each header and remove unused includes, at least in include/watcher/* and src/watcher/* files.

Watch proc with ebpf

One of the only filesystems we can't watch is the virtual /proc filesystem on Linux.

There is a way to create an ebpf program for the kernel to run which lets us know about modification events on those paths.

Error handling docs

The messaging syntax (including errors) should be detailed in the readme.

Messages come through event. The where field looks like this:

[s, e]/[sys, self]/[description]

Where s means "successful" (such as s/self/live@path) and e means "error" (such as e/sys/read).

self means we failed. sys means the operating system failed.

The description is usually a function name, an api, or a path. It's something extra.

These events always have the kind field as watcher (as opposed to file, dir, etc).

The what field varies. When a watcher first starts watching, what is create. When a watcher dies, it's destroy. Error messages are usually other.

We need to put this in the readme.

Containers

Necessary (or, at the least, very helpful) for testing and development are containers suitable for building and running Watcher.

I have some containers laying around for this purpose. They should be made available here.

`README.md`: Words

"watch will happily keep continue watching"... there was an attempt to use words, but it looks like they got mangled like symbols in a binary.

[api] finalize the api

Performance Tests

We need performance benchmarks against the other filesystem watchers. I'm thinking:

• chokidar
• fswatch
• watchman
• notify-rs

We have performance tests, but they're not comparative.

is C++17 standard no longer supported?

Hello @toge and @e-dant,

I wanted to use this project as a header-only library via conan-center, but found out that the minimum C++ standard is set to C++20 and not C++17.

in v0.2.2 changelog, it is stated that:

C++20 Concepts are removed for being very interesting but not beneficial enough to an API with a very small surface.

However, in e28bd3c commit wtr.watcher.cmake the minimum standard was changed from 17 to 20. Is this only for the standalone program or is it also important for the header-only library files?

Handle rename event properly

Is `/permissive-` needed on MSVC in 0.9.0?

In 0.9.0, watcher uses "alternative operator representations".

For example, following code uses and operator.
https://github.com/e-dant/watcher/blob/release/0.9.0/include/wtr/watcher.hpp#L111-L112

MSVC doesn't understand "alternatvie operator representations" in default.
With /permissive- compiler flag, MSVC can understand it.

I think there are two way to solve it.

1. add /permissive- compiler flag to CMakeLists.txt.
2. stop using "alternative operator expressions".

I think both are effective.
Can you please decide on a solution?

watcher.hpp includes C++20 standard library <bit>

Hi @e-dant, thanks for your work on issue #32, however, pure C++17 support is still not available.

wtr/watcher.hpp includes the bit library header, which was introduced in C++20 and is preventing compilation on older systems (for example Debian buster).

Is this header needed?

macOS CI runner occasionally fails due to missing CMake binary

Every so often, we get an error like this: https://github.com/e-dant/watcher/actions/runs/5920128540/job/16051048104#step:5:33

I tried to address this in 38d93a4 by adding a GitHub action to install cmake. Maybe that cmake action isn't working, or maybe something else is off. The path which make is looking for cmake in hints that the brew version of cmake isn't always there. I'm not sure why make keeps looking there.

Maybe the CMake cache needs to be reset? Maybe we should use Nix and just not worry about these kinds of issues?

Contributing

I need to create docs about contributions. Here's the flow we should generally use:

Push to next -> await ci/GitHub actions -> run tool/hone for the single header -> run tool/release

`tool/contain`: Needs a `README.md` and a `.tellfile`

All directories should have, at least, a README.md and a .tellfile. Even if they're short.

`sinclude/watcher/watcher.hpp`: Needs a `#pragma once` and a section in the `README.md`

Compiler error: `chrono:2320:48: note: undefined function '_S_fractional_width' cannot be used in a constant expression`

Recently, on both the release and next branches, our CodeQL builds have been coming up with this error:

https://github.com/e-dant/watcher/actions/runs/6684346814/job/18161503474#step:5:74

We also found this error on a Clang build for Linux in development:

https://github.com/e-dant/watcher/actions/runs/6684014904/job/18160816032#step:4:58

This looks and feels like a kind of issue unrelated to our codebase. I am not sure why the issue has only just started to crop up.

Using Nix is always an option to make our builds reproducible and mostly deterministic. Nix was mentioned in a previous build issue: #33

`README.md` Priorities

The readme offers Watcher's design goals, sure, but it doesn't offer the non-goals.

We should add a section on the things that Watcher is not interested in solving (at least at the moment). These are the OS-level caveats to certain system APIs. For inotify, that's events potentially disappearing or being duplicated. For std::filesystem, there are some places where UB might happen, although I haven't looked into exactly when they occur.

These are important caveats that deserve a place in the readme but require more research.

Broken Sanitizers

There are some problems with the sanitizers on Linux and Windows that we need to fix.

On Linux hosts, the address and memory sanitizers seem to be broken.

The address sanitizer fails here.

Which, as best I can tell, is a bug in LLVM's asan. Others seem to have similar issues, although their workarounds don't work locally, and CI time adds up, so I haven't deployed their workarounds.

The memory sanitizer is also in a bad state, saying that some standard library string functions are being used uninitialized from Catch.

The sanitizers on windows, I'm assuming, have different link options. The documentation is unhelpful because it should work as-is and seems to only support one sanitizer, asan.

Kernel Features Tracker

I've read the source code and I was disappointed not to find any usage of kernel's primitive to watch for filesystem changed (except for Darwin).

Why don't you use kernel primitives (like inotify on linux, for example)?

IMHO, it'll be a lot more efficient than computing a giant hash table of path to monitor.

Readme/Quick Start: Inaccurate 'Compile & Run'

On Apple platforms, the user needs to link against two system libraries.

On most (all?) platforms, c++ should point to the platform's default C++ compiler.

We should change g++ to c++ and put something like $(uname | grep darwin && echo '-framework CoreFoundation -framework CoreServices') in the compile command from "Quick Start".