Giter Site home page Giter Site logo

gorakhargosh / watchdog Goto Github PK

View Code? Open in Web Editor NEW
6.3K 134.0 683.0 2.05 MB

Python library and shell utilities to monitor filesystem events.

Home Page: http://packages.python.org/watchdog/

License: Apache License 2.0

Python 90.68% C 9.05% Batchfile 0.27%

watchdog's Introduction

Watchdog

Build Status CirrusCI Status

Python API and shell utilities to monitor file system events.

Works on 3.8+.

Example API Usage

A simple program that uses watchdog to monitor directories specified as command-line arguments and logs events generated:

import sys
import time
import logging
from watchdog.observers import Observer
from watchdog.events import LoggingEventHandler

if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO,
                        format='%(asctime)s - %(message)s',
                        datefmt='%Y-%m-%d %H:%M:%S')
    path = sys.argv[1] if len(sys.argv) > 1 else '.'
    logging.info(f'start watching directory {path!r}')
    event_handler = LoggingEventHandler()
    observer = Observer()
    observer.schedule(event_handler, path, recursive=True)
    observer.start()
    try:
        while True:
            time.sleep(1)
    finally:
        observer.stop()
        observer.join()

Shell Utilities

Watchdog comes with an optional utility script called watchmedo. Please type watchmedo --help at the shell prompt to know more about this tool.

Here is how you can log the current directory recursively for events related only to *.py and *.txt files while ignoring all directory events:

watchmedo log \
    --patterns="*.py;*.txt" \
    --ignore-directories \
    --recursive \
    --verbose \
    .

You can use the shell-command subcommand to execute shell commands in response to events:

watchmedo shell-command \
    --patterns="*.py;*.txt" \
    --recursive \
    --command='echo "${watch_src_path}"' \
    .

Please see the help information for these commands by typing:

watchmedo [command] --help

About watchmedo Tricks

watchmedo can read tricks.yaml files and execute tricks within them in response to file system events. Tricks are actually event handlers that subclass watchdog.tricks.Trick and are written by plugin authors. Trick classes are augmented with a few additional features that regular event handlers don't need.

An example tricks.yaml file:

tricks:
- watchdog.tricks.LoggerTrick:
    patterns: ["*.py", "*.js"]
- watchmedo_webtricks.GoogleClosureTrick:
    patterns: ['*.js']
    hash_names: true
    mappings_format: json                  # json|yaml|python
    mappings_module: app/javascript_mappings
    suffix: .min.js
    compilation_level: advanced            # simple|advanced
    source_directory: app/static/js/
    destination_directory: app/public/js/
    files:
      index-page:
      - app/static/js/vendor/jquery*.js
      - app/static/js/base.js
      - app/static/js/index-page.js
      about-page:
      - app/static/js/vendor/jquery*.js
      - app/static/js/base.js
      - app/static/js/about-page/**/*.js

The directory containing the tricks.yaml file will be monitored. Each trick class is initialized with its corresponding keys in the tricks.yaml file as arguments and events are fed to an instance of this class as they arrive.

Installation

Install from PyPI using pip:

$ python -m pip install -U watchdog

# or to install the watchmedo utility:
$ python -m pip install -U "watchdog[watchmedo]"

Install from source:

$ python -m pip install -e .

# or to install the watchmedo utility:
$ python -m pip install -e ".[watchmedo]"

Documentation

You can browse the latest release documentation online.

Contribute

Fork the repository on GitHub and send a pull request, or file an issue ticket at the issue tracker. For general help and questions use stackoverflow with tag python-watchdog.

Create and activate your virtual environment, then:

python -m pip install pytest pytest-cov
python -m pip install -e ".[watchmedo]"
python -m pytest tests

If you are making a substantial change, add an entry to the "Unreleased" section of the changelog.

Supported Platforms

  • Linux 2.6 (inotify)
  • macOS (FSEvents, kqueue)
  • FreeBSD/BSD (kqueue)
  • Windows (ReadDirectoryChangesW with I/O completion ports; ReadDirectoryChangesW worker threads)
  • OS-independent (polling the disk for directory snapshots and comparing them periodically; slow and not recommended)

Note that when using watchdog with kqueue, you need the number of file descriptors allowed to be opened by programs running on your system to be increased to more than the number of files that you will be monitoring. The easiest way to do that is to edit your ~/.profile file and add a line similar to:

ulimit -n 1024

This is an inherent problem with kqueue because it uses file descriptors to monitor files. That plus the enormous amount of bookkeeping that watchdog needs to do in order to monitor file descriptors just makes this a painful way to monitor files and directories. In essence, kqueue is not a very scalable way to monitor a deeply nested directory of files and directories with a large number of files.

About using watchdog with editors like Vim

Vim does not modify files unless directed to do so. It creates backup files and then swaps them in to replace the files you are editing on the disk. This means that if you use Vim to edit your files, the on-modified events for those files will not be triggered by watchdog. You may need to configure Vim appropriately to disable this feature.

About using watchdog with CIFS

When you want to watch changes in CIFS, you need to explicitly tell watchdog to use PollingObserver, that is, instead of letting watchdog decide an appropriate observer like in the example above, do:

from watchdog.observers.polling import PollingObserver as Observer

Dependencies

  1. Python 3.8 or above.
  2. XCode (only on macOS when installing from sources)
  3. PyYAML (only for watchmedo)

Licensing

Watchdog is licensed under the terms of the Apache License, version 2.0.

Copyright 2011 Yesudeep Mangalapilly.

Copyright 2012 Google, Inc & contributors.

Project source code is available at Github. Please report bugs and file enhancement requests at the issue tracker.

Why Watchdog?

Too many people tried to do the same thing and none did what I needed Python to do:

watchdog's People

Contributors

ajordat avatar altendky avatar apieum avatar bobotig avatar ccp-aporia avatar danilobellini avatar dependabot[bot] avatar evilham avatar gorakhargosh avatar hathawsh avatar ilayrosenberg avatar kreichgauer avatar kurtmckee avatar maybe-sybr avatar msabramo avatar ormod avatar palfrey avatar peritus avatar pilt avatar rrzaripov avatar samschott avatar sholsapp avatar taleinat avatar tamland avatar timgates42 avatar tony avatar txbm avatar wbond avatar yesudeep avatar z00m1n avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

watchdog's Issues

FSEvents: Cannot watch the same path with two different observers

When you schedule a watch on the same path with two different observers, the fsevents C extension runs into an error. This is because _watchdog_fsevents is a singleton and watchdog_fsevents_add_watch returns null if a watch has already been scheduled.

Test case:

>>> from watchdog.observers import Observer
>>> from watchdog.events import LoggingEventHandler
>>> obs = Observer()
>>> other_obs = Observer()
>>> handler = LoggingEventHandler()
>>> other_handler = LoggingEventHandler()
>>> path = '/tmp'
>>> obs.schedule(handler, path)
<ObservedWatch: path=/tmp, is_recursive=False>
>>> other_obs.schedule(other_handler, path)
<ObservedWatch: path=/tmp, is_recursive=False>
Exception in thread Thread-8:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "[...]/watchdog/src/watchdog/observers/fsevents.py", line 140, in run
    self.pathnames)
SystemError: error return without exception set

>>> 

Another cross-platform file system monitor in Python

Hi,

I just wanted to ping you since I'm the founder/developer of File Conveyor (code on github), for which I developed something similar to watchdog, called FSMonitor.

It's also cross-platform (but only supports inotify and FSEvents). It's far less documented or developed than watchdog though. It dates back to April 15, 2009, i.e. long before watchdog existed. The fact that I did an extremely poor job of making File Conveyor's components separately visible is probably why you never found it.

Recently, File Conveyor has become very active again. Today, I stumbled upon watchdog. After the first stable release (1.0) of File Conveyor, I'd like to switch to watchdog, because I obviously can gain from this far more actively developed component :)

I opened a ticket in my issue queue at wimleers/fileconveyor#77.

"TypeError: walk() got an unexpected keyword argument 'followlinks'' with Python2.5

It seems that watchdog didn't test with python2.5, I got an error like this, the followlinks parameter only appears in versions above python2.6.

Traceback (most recent call last):
  File "/Users/victorlin/zzzenv/bin/nowsync_client", line 8, in 
    load_entry_point('nowsync==0.3b', 'console_scripts', 'nowsync_client')()
  File "/Users/victorlin/zzzenv/lib/python2.5/site-packages/nowsync-0.3b-py2.5.egg/nowsync/scripts/commands.py", line 16, in run_client
    client.run()
  File "/Users/victorlin/zzzenv/lib/python2.5/site-packages/nowsync-0.3b-py2.5.egg/nowsync/client/client.py", line 170, in run
    observer.schedule(self, path=self.sync_path, recursive=True)
  File "/Users/victorlin/zzzenv/lib/python2.5/site-packages/watchdog-0.5.4-py2.5.egg/watchdog/observers/api.py", line 357, in schedule
    timeout=self.timeout)
  File "/Users/victorlin/zzzenv/lib/python2.5/site-packages/watchdog-0.5.4-py2.5.egg/watchdog/observers/polling.py", line 66, in __init__
    self._snapshot = DirectorySnapshot(watch.path, watch.is_recursive)
  File "/Users/victorlin/zzzenv/lib/python2.5/site-packages/watchdog-0.5.4-py2.5.egg/watchdog/utils/dirsnapshot.py", line 232, in __init__
    for root, directories, files in path_walk(self._path, recursive):
  File "build/bdist.macosx-10.6-i386/egg/pathtools/path.py", line 96, in walk
TypeError: walk() got an unexpected keyword argument 'followlinks'

EventEmitter threads never terminate

Since the FSEventsEmitter run loop is blocking and doesn't have a timeout, FSEventsEmitter threads never exit, not even after you called unschedule_all or similar on a BaseObserver instance.

The easiest way to observe that is to put an emitter.join() after emitter.stop() in both BaseObserver._remove_emitter and BaseObserver._clear_emitters -- this causes the program to block indefinitely whenever an Emitter is removed from an Observer.

incorrect path for file modification event after directory move

Illustrated by way of interleaved shell and simplified watchmedo log output:

$ mkdir first

on_created(event=<DirCreatedEvent: src_path=/tmp/watchdog-bug/first>)

$ touch first/file

on_created(event=<FileCreatedEvent: src_path=/tmp/watchdog-bug/first/file>)
on_modified(event=<FileModifiedEvent: src_path=/tmp/watchdog-bug/first/file>)

$ mv first second

# (suspiciously quiet...)

$ touch second/file

on_modified(event=<FileModifiedEvent: src_path=/tmp/watchdog-bug/first/file>)

note that the final event has the path "first/file" - which no longer exists. It should be "second/file".

And I'm not sure if the log trick is supposed to track directory moves, but it doesn't look like it is.

This is all using linux, so it could well be inotify-specific.

Too many open files

Running on OSX 10.6, Python 2.6.1:

$ watchmedo log --patterns="*" --recursive --command='echo "${watch_src_path}"' /Volumes/Data/Documents

Leads to:

Traceback (most recent call last):
File "/Volumes/Data/.virtualenvs/playground/bin/watchmedo", line 8, in
File "/Volumes/Data/.virtualenvs/playground/lib/python2.6/site-packages/watchdog/watchmedo.py", line 452, in main
File "/Volumes/Data/.virtualenvs/playground/lib/python2.6/site-packages/argh/helpers.py", line 304, in dispatch
File "/Volumes/Data/.virtualenvs/playground/lib/python2.6/site-packages/argh/helpers.py", line 205, in dispatch
File "/Volumes/Data/.virtualenvs/playground/lib/python2.6/site-packages/argh/helpers.py", line 281, in _execute_command
File "/Volumes/Data/.virtualenvs/playground/lib/python2.6/site-packages/argh/helpers.py", line 264, in _call
File "/Volumes/Data/.virtualenvs/playground/lib/python2.6/site-packages/watchdog/watchmedo.py", line 356, in log
File "/Volumes/Data/.virtualenvs/playground/lib/python2.6/site-packages/watchdog/watchmedo.py", line 126, in observe_with
File "/Volumes/Data/.virtualenvs/playground/lib/python2.6/site-packages/watchdog/observers/api.py", line 346, in schedule
File "/Volumes/Data/.virtualenvs/playground/lib/python2.6/site-packages/watchdog/observers/api.py", line 90, in init
File "/Volumes/Data/.virtualenvs/playground/lib/python2.6/site-packages/pathtools/path.py", line 183, in absolute_path
File "/Volumes/Data/.virtualenvs/playground/bin/../lib/python2.6/posixpath.py", line 337, in abspath
OSError: [Errno 24] Too many open files

non-recursive `watchmedo log` using FSEvents fails on OS X.

Error for watchmedo log . using python 2.7:

python[25252:1403] In '__CFRunLoopLock', file /SourceCache/CF/CF-550.42/RunLoop.subproj/CFRunLoop.c, line 438, during lock, spin lock 0x101500a40 has value 0x6, which is neither locked nor unlocked. The memory has been smashed.

Simple example does not work as advertised on python 2.

The default logging level (in 2.5 and 2.6 at least) is WARNING/30 but LoggingEventHandler uses INFO/20 to log events. This means that the simple example in the documentation and in the README never actually logs a message. Those examples should be changed to set the logging level to INFO/20 in order to show newcomers that the code is actually working. Alternatively LoggingEventHandler could take a logging level as a parameter.

Issues with Django

Trying to use gunicorn as an alternative to the Django development server, using Watchdog to monitor my Django project files and reload gunicorn when they change.

I've tried to be as aggressive as possible in limiting the amount of files watchdog has to watch.

watchmedo shell-command
--patterns=".rst;.rst.inc;.py;.py.swp;.rst.swp;.rst.inc.swp"
--ignore-patterns=".;#;~;.py.;static-media/;uploads/.svn;.png;.gif;.js;.css;*.jpg;"
--ignore-directories
--recursive
--command='kill -HUP cat /tmp/gunicorn.pid'
/path/to/project

Tried for hours to get this to work properly. I think what's also causing issues is that my project uses Subversion and I need a way to block .svn directories from being watched as they have files like "settings.py.svn-base" which are probably being watched (because of the ".py" match. I've tried to block this via the ".py." ignore pattern but not sure if that works?

Watchdog starts successfully but as soon as I open a file I get:

Exception in thread Thread-2:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 530, in bootstrap_inner
self.run()
File "/Users/ryanblunden/.virtualenvs/excela/lib/python2.7/site-packages/watchdog/observers/api.py", line 195, in run
self.queue_events(self.timeout)
File "/Users/ryanblunden/.virtualenvs/excela/lib/python2.7/site-packages/watchdog/observers/kqueue.py", line 703, in queue_events
new_snapshot)
File "/Users/ryanblunden/.virtualenvs/excela/lib/python2.7/site-packages/watchdog/observers/kqueue.py", line 546, in _queue_dirs_modified
self.queue_event(FileCreatedEvent(file_created))
File "/Users/ryanblunden/.virtualenvs/excela/lib/python2.7/site-packages/watchdog/observers/kqueue.py", line 520, in queue_event
self._register_kevent(event.src_path, event.is_directory)
File "/Users/ryanblunden/.virtualenvs/excela/lib/python2.7/site-packages/watchdog/observers/kqueue.py", line 470, in _register_kevent
self._descriptors.add(path, is_directory)
File "/Users/ryanblunden/.virtualenvs/excela/lib/python2.7/site-packages/watchdog/observers/kqueue.py", line 257, in add
self._add_descriptor(KeventDescriptor(path, is_directory))
File "/Users/ryanblunden/.virtualenvs/excela/lib/python2.7/site-packages/watchdog/observers/kqueue.py", line 342, in __init

self._fd = os.open(path, WATCHDOG_OS_OPEN_FLAGS)
OSError: [Errno 24] Too many open files: '/Users/ryanblunden/Projects/investorcentre.excela.com.au/excela/static-media/media/admin/img/gis/move_vertex_off.png'

libyaml is not required

Hi,

there is no need to install libyaml. It is optional for PyYAML and only brings a performance boost for parsing, but that's not really required for config files. I would change the wording of the README.

Regards,
Oliver

Use of Flask theme for project unrelated to Flask

…is permitted, as the themes are open sourced. But the author, @mitsuhiko, prefers if the unmodified themes are used only for projects related to Flask.

We kindly ask you to only use these themes in an unmodified manner just
for Flask and Flask-related products, not for unrelated projects. If you
like the visual style and want to use it for your own projects, please
consider making some larger changes to the themes (such as changing
font faces, sizes, colors or margins).

DirectorySnapshotDiff can't detect modification from vim

Don't know how, but it seems that if a file is modified by vim, and its inode id will be changed. With current implementation of DirectorySnapshotDiff, it think the file is modified, only if the inode id is same, which might not be correct in this situation.

To fix the bug, I modify watchdog.utils.dirsnapshot from line 97

        # Detect all the modifications.
        for path, stat_info in dirsnap.stat_snapshot.items():
            if path in ref_dirsnap.stat_snapshot:
                ref_stat_info = ref_dirsnap.stat_info(path)
                if stat_info.st_ino == ref_stat_info.st_ino and stat_info.st_mtime != ref_stat_info.st_mtime:
                    if stat.S_ISDIR(stat_info.st_mode):
                        self._dirs_modified.append(path)
                    else:
                        self._files_modified.append(path)

Remove the inode id condition:

        # Detect all the modifications.
        for path, stat_info in dirsnap.stat_snapshot.items():
            if path in ref_dirsnap.stat_snapshot:
                ref_stat_info = ref_dirsnap.stat_info(path)
                if stat_info.st_mtime != ref_stat_info.st_mtime:
                    if stat.S_ISDIR(stat_info.st_mode):
                        self._dirs_modified.append(path)
                    else:
                        self._files_modified.append(path)

and it can detect modification from vim now.

Optimize FSEvents and Kqueue snapshotting

Currently, for any event both these emitters snapshot the entire directory tree
to compare for changes. If we know which directory the change occurred in
we can modify the DirectorySnapshot class to update itself with only that
part of the tree and return a diff of events within that subtree. This should
save quite a bit of I/O.

ctypes.ArgumentError: argument 7: <type 'exceptions.TypeError'>: wrong type

On Windows XP when I try to run watchdog example, I get this:

Traceback (most recent call last):
  File "observer.py", line 8, in <module>
    observer.schedule(event_handler, path='.', recursive=True)
  File "c:\python25\lib\site-packages\watchdog-0.5.3-py2.5.egg\watchdog\observers\api.py", line 357, in schedule
    timeout=self.timeout)
  File "c:\python25\lib\site-packages\watchdog-0.5.3-py2.5.egg\watchdog\observers\read_directory_changes.py", line 69, i
n __init__
    WATCHDOG_FILE_FLAGS)
  File "C:\Python25\lib\site-packages\watchdog-0.5.3-py2.5.egg\watchdog\observers\winapi_common.py", line 109, in get_di
rectory_handle
    '')
ctypes.ArgumentError: argument 7: <type 'exceptions.TypeError'>: wrong type
Exception exceptions.AttributeError: '_shutdown' in <module 'threading' from 'C:\Python25\lib\threading.pyc'> ignored

Monitoring non-directory files

It took me some time to figure out that Observer.schedule only accepts path names pointing to a directory. The API Reference clearly states that it has to be a directory path, but the Quick Start guide could be more clear about it, and the Observer classes should probably raise an exception if you pass a non-directory file path.

Ignore Subdirectories

I'm evaluating watchdog for a little pet-project of mine..
Is there a platform independent way for watchdog to ignore subdirectories by setting some flag?

AttributeError: '_Event' object has no attribute 'is_set'

This I got before

Traceback (most recent call last):
  File "observer.py", line 7, in <module>
    observer = Observer()
  File "c:\python25\lib\site-packages\watchdog-0.5.3-py2.5.egg\watchdog\observers\read_directory_changes.py", line 129,
in __init__
    timeout=timeout)
  File "c:\python25\lib\site-packages\watchdog-0.5.3-py2.5.egg\watchdog\observers\api.py", line 277, in __init__
    EventDispatcher.__init__(self, timeout)
  File "c:\python25\lib\site-packages\watchdog-0.5.3-py2.5.egg\watchdog\observers\api.py", line 208, in __init__
    DaemonThread.__init__(self)
  File "C:\Python25\lib\site-packages\watchdog-0.5.3-py2.5.egg\watchdog\utils\__init__.py", line 94, in __init__
    if not has_attribute(self._stopped_event.is_set, 'is_set'):
AttributeError: '_Event' object has no attribute 'is_set'
Exception exceptions.AttributeError: '_shutdown' in <module 'threading' from 'C:\Python25\lib\threading.pyc'> ignored

Then I changed line 94 in watchdog\utils\__init__.py into this, removing .is_set:
if not has_attribute(self._stopped_event, 'is_set'):

Got another problem https://github.com/gorakhargosh/watchdog/issues/#issue/33

Enhancement: Event after closing a writable file

I haven't found a way to get notified about completed writes. As a test, I was listening to any event and downloaded a file with wget to see the events:

On linux, there's a FileCreatedEvent upon creation and one FileModifiedEvent upon completion. On OS X instead, there is a DirModifiedEvent, a FileCreatedEvent and a lot of FileModifiedEvents.

The script I used is based on your example and lives here: http://dpaste.com/hold/529603/

If there is no technical reason, I'd love to see an event like that so that I can do something with files after they have been moved.

No notifications for permission changes

Watchdog currently does not report any changes to file permissions. I will probably work on a patch for this, which I could contribute back if there is interest.

Suspend of workstation breaks the running process and exits with a system exception

Hi Yesudeep,

the application provides very solid run, but when a suspend of workstation is performed the process fails with the following exception.

Exception in thread Thread-4:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "/usr/local/lib/python2.6/dist-packages/watchdog-0.5.4-py2.6.egg/watchdog/observers/api.py", line 192, in run
    self.queue_events(self.timeout)
  File "/usr/local/lib/python2.6/dist-packages/watchdog-0.5.4-py2.6.egg/watchdog/observers/inotify.py", line 731, in queue_events
    inotify_events = self._inotify.read_events()
  File "/usr/local/lib/python2.6/dist-packages/watchdog-0.5.4-py2.6.egg/watchdog/observers/inotify.py", line 516, in read_events
    event_buffer = os.read(self._inotify_fd, event_buffer_size)
OSError: [Errno 4] Interrupted system call

Is there a way to keep this process running even the suspend is performed? Can it be solved on a level of watchdog or does it have to be managed by additional user defined try and catch?

Thank you for this excellent piece of software!

depends on brownie.datastructures, which is not yet published

The version of watchdog on pypi imports brownie.datastructures but the version of brownie on pypi doesn't yet include the datastructures module. I tried checking brownie out from github because it has a datastructures module in it, but setup.py doesn't package it:

/usr/src/brownie$ python setup.py build
running build
running build_py
copying brownie/caching.py -> build/lib/brownie
copying brownie/proxies.py -> build/lib/brownie
copying brownie/init.py -> build/lib/brownie
copying brownie/importing.py -> build/lib/brownie
copying brownie/itools.py -> build/lib/brownie
copying brownie/abstract.py -> build/lib/brownie
copying brownie/functional.py -> build/lib/brownie
copying brownie/pycompat.py -> build/lib/brownie
copying brownie/parallel.py -> build/lib/brownie
copying brownie/tests/caching.py -> build/lib/brownie/tests
copying brownie/tests/proxies.py -> build/lib/brownie/tests
copying brownie/tests/init.py -> build/lib/brownie/tests
copying brownie/tests/importing.py -> build/lib/brownie/tests
copying brownie/tests/itools.py -> build/lib/brownie/tests
copying brownie/tests/abstract.py -> build/lib/brownie/tests
copying brownie/tests/parallel.py -> build/lib/brownie/tests

Works fine once or twice then nothing happens

I'm on OSX and it's using the kqueue observer. I start it something like this:

watchmedo shell-command --patterns="*.py" --recursive  \
  --command="./manage.py test" apps/

It starts, I change a file and save and the shell command runs; and I notice my unit test failed. I change the file again and save but this time nothing happens with watchmedo.

Sometimes it works a second time and even a third I've seen but never more than that.

Uncatchable OSError raised in Watchdog thread when watched path is deleted

Start watching a path which already exists, eg.

from watchdog import LoggingEventHandler
from watchdog import Observer

event_handler = LoggingEventHandler()
observer = Observer()
observer.schedule(event_handler, path="/path/to/watch")
observer.start()
observer.join()

Delete the path being watched ...

$ rmdir /path/to/watch

Receive uncatchable exception:

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/threading.py", line 522, in __bootstrap_inner
    self.run()
  File "/Library/Python/2.6/site-packages/watchdog-0.5.4-py2.6.egg/watchdog/observers/api.py", line 192, in run
    self.queue_events(self.timeout)
  File "/Library/Python/2.6/site-packages/watchdog-0.5.4-py2.6.egg/watchdog/observers/kqueue.py", line 686, in queue_events
    self.watch.is_recursive)
  File "/Library/Python/2.6/site-packages/watchdog-0.5.4-py2.6.egg/watchdog/utils/dirsnapshot.py", line 227, in __init__
    stat_info = os.stat(self._path)
OSError: [Errno 2] No such file or directory: '/path/to/watch'

nose, unittest, and watchdog - OSError on exit

Watchdog 0.5.4 emits an OSError exception on program exit from a unit test if a directory that was scheduled is deleted, even if that directory was unscheduled before deletion and before program exit - but only if that unit test is run by nose (http://readthedocs.org/docs/nose/en/latest/), not run in isolation.

Let's take a look at some example code

http://pastie.org/2515178

When the unit test above is run in isolation, no OSError exception is raised

http://pastie.org/2515203

But when the unit test above is run by nose, an OSError exception is raised

http://pastie.org/2515200

It looks like when run by nose, watchdog attempts to remove a watch that has already been removed - inotify_rm_watch(...) returns -1 on an invalid watch descriptor or file descriptor.

PatternMatchingEventHandler fires even when pattern fails to match

In PatternMatchingEventHandler you make the dispatch conditional like so:

if filter_paths(...):

The problem being that filter_paths returns a generator which will be True even if it has no elements to yield. I think the simplest fix would be to change it to:

if list(filter_paths(...)):

Or something like that.

1.7.6 produces Trace/BPT trap

Installed GIT on OS X Server 10.5 but commands like

git config --global user.name "My Name"
dyld: unknown required load command 0x80000022
Trace/BPT trap

git -h
dyld: unknown required load command 0x80000022
Trace/BPT trap

Help!

Doesn't detect file modification with vim on OSX 10.7

At first, FSEvents-support works beautifully on OSX now. Thank you!

Unfortunately, watchdot cannot detect vim saving modifications to an existing file on a mac.

I traced it here:
https://github.com/gorakhargosh/watchdog/blob/master/src/watchdog/utils/dirsnapshot.py#L95

I have no idea why, but it seems that when vim saves a file, it gets a new inode number and watchdog cannot categorize the event and ignores it.

An example stat info from ref_dirsnap:

posix.stat_result(st_mode=33188, st_ino=51180728, st_dev=234881027L, st_nlink=1, st_uid=501, st_gid=20, st_size=2756, st_atime=1313212427, st_mtime=1313212426, st_ctime=1313212426)

The same stat info from dirsnap:

posix.stat_result(st_mode=33188, st_ino=51180736, st_dev=234881027L, st_nlink=1, st_uid=501, st_gid=20, st_size=2756, st_atime=1313212514, st_mtime=1313212514, st_ctime=1313212514)

Some other editors, e.g. TextMate work correctly (st_ino doesn't change and modification is detected correctly).

0.4.0 not installable as an egg because of symlink README.md -> README

Hey.

I tried building a sdist egg from current master and install it in my buildout, but this breaks:

Installing watchdog.
Installing 'watchdog'.

We have no distributions for watchdog that satisfies 'watchdog==0.4.0dev0'.
Getting distribution for 'watchdog==0.4.0dev0'.
Running easy_install:
/usr/bin/python "-S" "-c" "import sys,os;p = sys.path[:];import site;sys.path[:] = p;[sys.modules.pop(k) for k, v in sys.modules.items() if hasattr(v, '__path__') and len(v.__path__)==1 and not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))];from setuptools.command.easy_install import main;main()" "-mUNxd" "/Users/filip/.buildout/eggs/tmpBeFEr0" "-q" "/Users/filip/.buildout/downloads/dist/watchdog-0.4.0dev0.tar.gz"
path=/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python

error: /var/folders/am/amvgZGWsGOiqK+2kOSmf4k+++TI/-Tmp-/easy_install-ZWnBuP/watchdog-0.4.0dev0/README.md: No such file or directory
An error occurred when trying to install watchdog 0.4.0dev0. Look above this message for any errors that were output by easy_install.
While:
  Installing watchdog.
  Getting distribution for 'watchdog==0.4.0dev0'.
Error: Couldn't install: watchdog 0.4.0dev0

This is because of the fact that there is a symlink in the generated tar (README.md points to README) and setuptools (and distribute too) has a bug unpacking symlinks from tars http://bugs.python.org/setuptools/issue89 (i really need to undust my patch for this setuptools bug - missing 100% test coverage atm) and send it to tarek.

schedule() after unschedule_all() fails on OSX

>>> from watchdog.observers import Observer
>>> from watchdog.events import LoggingEventHandler
>>> ob = Observer()
>>> ob.start()
>>> event_handler = LoggingEventHandler()
>>> ob.schedule(event_handler, '.')

>>> ob.unschedule_all()
>>> event_handler = LoggingEventHandler()
>>> ob.schedule(event_handler, '.')

>>> Exception in thread Thread-3:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/threading.py", line 522, in __bootstrap_inner
    self.run()
  File "/Users/parente/envs/rst/src/watchdog/src/watchdog/observers/fsevents.py", line 145, in run
    pathnames)
SystemError: error return without exception set

[linux] issue with files moved from/to watched path

Hi there,

Maybe I'm missing something but I think there are two issues with file moves in watchdog:

  • issue 1: when I move a file from the watched directory to a path outside it, no event is triggered
  • issue 2: I've got a KeyError when I move a file or directory from an external path fixed

To reproduce:

import time
import logging

logging.basicConfig(level=logging.DEBUG)

from watchdog.events import LoggingEventHandler
from watchdog.observers import Observer

handler = LoggingEventHandler()
observer = Observer()
observer.schedule(handler, path='/home/izi/test_watchdog', recursive=True)
observer.start()

try:
    while True:
        time.sleep(0.1)
except KeyboardInterrupt:
    observer.stop()

And from command line:

$ python watch.py
$ cd /home/izi/test_watchdog

Create a file (btw, you'll notice that two events are generated but I think this is normal, not sure):

$ touch foo.txt
on_created <FileCreatedEvent: src_path=/home/izi/test_watchdog/foo.txt>
INFO:root:Created file: /home/izi/test_watchdog/foo.txt
on_modified <FileModifiedEvent: src_path=/home/izi/test_watchdog/foo.txt>
INFO:root:Modified file: /home/izi/test_watchdog/foo.txt

issue 1: no event generated:

$ mv foo.txt ../

issue 2: error raised:

$ mv ../foo.txt .
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "/home/izi/mo/lib/python2.6/site-packages/watchdog/observers/api.py", line 192, in run
    self.queue_events(self.timeout)
  File "/home/izi/mo/lib/python2.6/site-packages/watchdog/observers/inotify.py", line 731, in queue_events
    inotify_events = self._inotify.read_events()
  File "/home/izi/mo/lib/python2.6/site-packages/watchdog/observers/inotify.py", line 529, in read_events
    move_src_path = self.source_for_move(inotify_event)
  File "/home/izi/mo/lib/python2.6/site-packages/watchdog/observers/inotify.py", line 475, in source_for_move
    return self._moved_from_events[destination_event.cookie].src_path
KeyError: 2546

Informations on my system:

$ uname -a
Linux izi-desktop 2.6.35-28-generic #50-Ubuntu SMP Fri Mar 18 19:00:26 UTC 2011 i686 GNU/Linux
$ python --version
Python 2.6.6
$ python -c 'import watchdog.version;print watchdog.version.VERSION_STRING'
0.5.4

That's it, I hope my bug report is clear enough feel free to ping me (izimobil at gmail dot com) if you need more infos !

David

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.