a quick review of file watchers

7 min read Original article ↗

File watchers. I always forget about those and never use then, but I constantly feel like I need them. So I made this list to stop searching everywhere for those things which are surprisingly hard to find in a search engine.

  1. a quick review of file watchers
  2. Generic
    1. caretaker
    2. chokidar
    3. direvent
    4. entr
    5. fluffy
    6. fswatch
    7. gamin
    8. incron
    9. inoticoming
    10. inotify-hookable
    11. inotify-tools
    12. notify-rs
    13. peat
    14. systemd .path units
    15. watchexec
    16. watchman
  3. Web development
    1. grip (markdown)
    2. livereload
    3. frameworks
  4. Unit tests
    1. autotest
    2. conttest
    3. tdaemon
    4. Test::Continuous
  5. File synchronization
    1. gitwatch
    2. inosync
    3. lsyncd
  6. Intrusion detection
    1. fail2ban
    2. iwatch
    3. sshguard
  7. Other
    1. inotify-info
    2. kfmon (kobo launcher)
    3. timetrack

Generic

Those tools an watch files or trees of files and execute whatever.

caretaker

https://github.com/grego/caretaker

  • 2020
  • Rust
  • MIT
  • No Debian package
  • requires a TOML config file

chokidar

https://github.com/kimmobrunfeldt/chokidar-cli

  • 2015-2019
  • Javascript
  • MIT
  • No Debian package

direvent

https://www.gnu.org.ua/software/direvent/

  • 2013-2019, GNU project since 2014
  • C
  • GPL-3
  • Debian package, since 2015 (stretch), out of date (5.1 vs 5.2)
  • requires a config file to operate

entr

http://eradman.com/entrproject/

  • 2012-2019
  • C
  • ISC
  • Debian package since 2014 (jessie), up to date
  • finds file from stdin
  • requires one watch per file, which quickly runs over the builtin limits, so requires sysctl hacks to actually work on even a medium-sized project
  • has special hacks to reload browser
  • can clear screen between calls

fluffy

https://github.com/tinkershack/fluffy

  • 2018 (no official release)
  • C
  • Unlicense
  • No Debian package
  • Linux-specific
  • Streams events to standard output
  • also a library

fswatch

http://emcrisostomo.github.io/fswatch/

  • 2014-2018
  • C++
  • GPL-3
  • Debian package since 2017 (buster), up to date
  • outputs changesets using a specific syntax, so requires more commandline voodoo

gamin

https://people.gnome.org/~veillard/gamin/

incron

https://github.com/ar-/incron

  • 2006-2012, 2014-2015
  • C++
  • GPL-2
  • Debian package since 2007
  • Watches files and launches command in a cron-like (and rather obscure) syntax
  • no event deduplication

inoticoming

https://tracker.debian.org/pkg/inoticoming

  • 2007-2012
  • C
  • GPL-2
  • Debian package is upstream
  • watches directory, fires command
  • yet another wrapper for inotify
  • built for reprepro
  • no event deduplication

inotify-hookable

https://metacpan.org/pod/App::Inotify::Hookable

  • 2012-2016
  • Debian package since 2013 (jessie)
  • Perl
  • GPL-1+
  • yet another wrapper for inotify
  • no event deduplication, but can "buffer" multiple events together with a timeout

https://github.com/rvoicilas/inotify-tools/

notify-rs

https://github.com/notify-rs/notify

peat

https://github.com/sjl/peat

  • 2012-2016
  • Python
  • GPL-3
  • Not in Debian
  • relatively short Python script
  • runs a command when files changed, specified on stdin

systemd .path units

https://www.freedesktop.org/software/systemd/man/systemd.path.html

  • 2010-2019
  • C
  • GPL-2
  • Debian package since 2010
  • activates a system or user "unit" on inotify changes

Update: I tried to make this work for calibre but somehow it didn't work:

# this doesn't actually work. either it doesn't notices changes from git, or it
# doesn't notify calibre-server.service, or it does and that doesn't trigger a
# restart, but the thing doesn't restart as i would expect
[Path]
PathModified=/srv/books/metadata.db
PathModified=/srv/books
PathChanged=/srv/books/metadata.db
PathChanged=/srv/books

[Unit]
Description=calibre content server
After=network.target

... ie. it doesn't restart the service on changes to any of those files.

watchexec

https://github.com/watchexec/watchexec

  • 2016-2019
  • Rust
  • Apache-2.0
  • No Debian package (bug 946546)
  • supports .gitignore files and filename patterns
  • merges multiple events
  • fast polling
  • can clear screen between calls
  • waits for command to complete to launch again

  • simple commandline, ie. this worked for me:

     watchexec -w ./feed2exec/ -e py -d 2000 -v -p tox

    it automatically picked up my .gitignore directory, which elegantly avoided the loops I have had in watchman because of the files generated by tox

watchman

http://facebook.github.io/watchman/

  • 2013-2019
  • C++, Python
  • Apache-2
  • Facebook
  • Debian package, since 2019 (bullseye)
  • can watch multiple directories
  • waits for settling

  • somewhat complex client/server architecture, sends output to logfiles, although watchman-make provides a nice little wrapper:

     watchman-make -p '**/*.py' --run tox

    Unfortunately this doesn't work for my Python project, as it picks up setup.py writing the _version.py file as a change, and constantely runs the tests, because there's no way to ignore files in watchman-make.

Web development

grip (markdown)

https://github.com/joeyespo/grip

  • 2012-2018
  • Python
  • MIT
  • Debian package since 2016 (stretch)
  • rebuilds markdown files and servers them over HTTP with autoreload
  • relies on GitHub's API for rendering (???)

livereload

https://github.com/lepture/python-livereload

  • 2012-2019
  • Python
  • BSD-3
  • Debian package since 2013 (jessie), up to date
  • watches a directory and triggers a web browser livereload extension
  • also supports arbitrary commands through custom Python scripts
  • integration with Python frameworks (Django, Flask, Bottle)

frameworks

Many frameworks automatically reload files when they change. A few examples:

Unit tests

autotest

https://metacpan.org/pod/distribution/App-autotest/scripts/autotest

  • 2012-2015
  • Perl
  • Artistic license
  • Rerun tests on file change
  • Perl only
  • No Debian package

conttest

https://github.com/eigenhombre/continuous-testing-helper

  • 2012-2018 (no official release)
  • Python
  • No license
  • No Debian package
  • Reruns tests (but also any command) on file change in the current directory
  • Part of a continuous testing strategy on the desktop

tdaemon

https://github.com/brunobord/tdaemon

  • 2008-2014 (no official release)
  • Python
  • MIT
  • No Debian package
  • inspiration for the above
  • restricted to a subset of Python test frameworks

Test::Continuous

https://metacpan.org/pod/Test::Continuous

  • 2008-2015
  • Perl
  • Artistic license
  • Run tests on file change
  • Perl only
  • No Debian package

File synchronization

I will not go through a list of all the file synchronization tools here. Most of them have some sort of "wake-up" system to notify file changes, but they are considered out of scope here unless they can also launch custom commands. But just for the sake of the argument, I am aware of:

gitwatch

https://github.com/gitwatch/gitwatch

  • 2012-2020
  • GPLv3
  • relies on inotifywait
  • just commits and pushes to git after a delay

inosync

https://github.com/hollow/inosync

  • 2008-2010 (archived upstream)
  • Python
  • BSD
  • Debian package since 2009 (jessie), in RFA
  • combination of rsync and inotify
  • similar to lsyncd

lsyncd

https://github.com/axkibe/lsyncd

  • 2012-2018
  • Lua, C
  • GPL-2
  • Debian package since 2010 (jessie), up to date
  • spawns rsync on file changes
  • Lua configuration can be leveraged to do other things than sync

Intrusion detection

Here again, there are many filesystem integrity checkers and intrusion detection systems (IDS), but they are not relevant here unless they can also execute arbitrary commands. But, again, here's an idea of the other stuff that is out there that might be a better fit than trying to fit a square peg in this round hole:

fail2ban

http://www.fail2ban.org/

  • 2004-2018
  • Python
  • GPL-2
  • Debian package since 2005, out of date
  • parses logfiles and executes commands
  • handles multiple services and patterns and can escalate

iwatch

http://iwatch.sourceforge.net/

  • 2006-2009
  • Perl
  • GPL-2+
  • Debian package since 2006
  • emails on file changes
  • can also execute commands, but primarily designed as an intrusion detection system
  • XML configuration (???)

sshguard

https://sshguard.net/

  • 2007-2019
  • C
  • ISC
  • Debian package since 2007, out of date
  • similar to fail2ban

Other

inotify-info

https://github.com/mikesart/inotify-info

  • 2021-...
  • C++
  • MIT
  • Debian package
  • simply lists active inotify watches

kfmon (kobo launcher)

https://github.com/NiLuJe/kfmon

  • 2016-2019
  • C
  • GPL-3
  • No Debian package
  • Launches programs when icons are activated on a Kobo reader

I've probably missed half a million such programs so this might be expanded with time and your comments.

Update: I've added a bunch of tools. I was relunctant to add all those old inotify wrappers because I don't find them as interesting as the newer ones - they're really hard to use! - but I guess it's worth mentioning them even if just to criticise them. ;)

timetrack

https://github.com/joshmcguigan/timetrack

Created . Edited .