Snippet 0x03: Recursively watch folders on Java (using the Java 7 WatchService)

The “new” Java 7 WatchService provides a mechanism to easily monitor a single folder for file system events. Java uses the underlying OS mechanisms to realize that (inotifiy on Linux, and ReadDirectoryChanges* on Windows). What the WatchService cannot do, however, is monitor a folder recursively – meaning monitoring all sub-folders and register new folders if created. This tiny code snippet article shows you how.

1. Full code

The full working code (embedded in a my open source file sync software Syncany) is available on GitHub. Feel free to use it, or to leave feedback in the comments.

2. A simpler approach

The basic idea is simple: To monitor an entire file tree, we first must walk the file tree and register a watch on every single folder. Once the initial watches are set, new watches must be added once the watches register any changes.

In my first attempt to implement this (back in 2011 with jpathwatch), I tried to interpret the events to add new folders and remove deleted/moved folders. This turned out to be very, very difficult, because quick changes on the file systems are hard to trace – especially if you move around things very quickly. Files vanish, file watches get cancelled while you’re trying to set watches for their subfolders, etc.

Long story short: This time, I settled for a simpler approach:

1. Walk the file tree and register watches in all folders and subfolders (walkTreeAndSetWatches())
2. When an event occurs, wait 3 seconds events to settle (resetWaitSettlementTimer())
3. When the settlement time has passed, walk the tree again, cancel stale watches and fire a watch event

3. One thread to monitor them all

The central watch thread basically runs this algorithm. It first walks the file tree using walkTreeAndSetWatches() (basically using Java 7’s Files.walkFileTree()) and then enters a loop polling for events. In case you haven’t worked with the file tree walker before, here is what it looks like in the wild:

The walkFileTree method makes it very easy to traverse a folder with all it subfolders

private synchronized void walkTreeAndSetWatches() {
  try {
    Files.walkFileTree(root, new FileVisitor<Path>() {
      @Override
      public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
        if (ignorePaths.contains(dir)) {
          return FileVisitResult.SKIP_SUBTREE;
        }
        else {
          registerWatch(dir);
          return FileVisitResult.CONTINUE;
        }
      }

      // Return FileVisitResult.CONTINUE for other methods
      // ...
    });
  }
  catch (IOException e) {
    // Don't care
  }
}

It applies a classic visitor pattern, firing callback methods for every node it touches. Depending on what you return (CONTINUE, SKIP_SUBTREE, SKIP_SIBLINGS or TERMINATE), the rest of the algorithm behaves accordingly. I use this functionality just to ignore certain sub trees – in the case of Syncany that’s just the .syncany sub tree.

After initially walking the tree and registering the watches for the sub folders, the poll-wait-fire-loop is started. Inside the loop, the watchService.take() method is blocking so it’s important that this happens in a separate thread:

Starting the watch thread and the watch service

public void start() throws IOException {
  watchService = FileSystems.getDefault().newWatchService();

  watchThread = new Thread(new Runnable() {
    @Override
    public void run() {
      running.set(true);
      walkTreeAndSetWatches();

      while (running.get()) {
        try {
          WatchKey watchKey = watchService.take();
          watchKey.pollEvents(); // Take events, but don't care what they are!

          watchKey.reset();
          resetWaitSettlementTimer();
        }
        catch (InterruptedException | ClosedWatchServiceException e) {
          running.set(false);
        }
      }
    }
  }, "Watcher");
  
  watchThread.start();
}

When an event happens, we actually don’t process events at all. Instead, we just throw them away by not collecting the events returned by watchKey.pollEvents(). We just reset the watch key (= make it listen to events again) and start the above mentioned 3-second-timer:

The settlement timer prevents passing too many events to the application

private synchronized void resetWaitSettlementTimer() {
  if (timer != null) {
    timer.cancel();
    timer = null;
  }

  timer = new Timer("WatchTimer");
  timer.schedule(new TimerTask() {
    @Override
    public void run() {
      walkTreeAndSetWatches();
      unregisterStaleWatches();

      fireListenerEvents();
    }
  }, 3000);
}

This method is called for every event the watch service registers – meaning that if the 3 seconds haven’t run out yet, but another event occurs, this method is called more than once time. If, for instance, you move a file, wait a second, then move it again, and then wait >3 seconds, this method will be called twice, but the timer task will only be fired once.

This is a basic mechanism to avoid passing too many events to the application – believe me, it’s worth waiting at least 3 seconds. Other applications wait even longer: As far as I know, SparkleShare even waits 5 seconds.

All that’s left now is to actually fire the even to our listener(s). In the case of Syncany, I only need one listener, so this is pretty easy:

Firing an event to the connected listener

private synchronized void fireListenerEvents() {
  if (listener != null) {
    listener.watchEventsOccurred();
  }
}

A. About this post

I’m trying a new section for my blog. I call it Code Snippets. It’ll be very short, code-focused posts of things I recently discovered or find fascinating or helpful. I hope this helps.