Update (May 4, 2017). This post has inspired OmniGroup to provide official support for NGINX installed via the nginx-extras APT package. After using OmniFocus 2.9 for several weeks, I have removed the previously suggested workaround from the solution. The rest of the article may still be of some interest as a primer of debugging a proprietary application.

One of the less talked about features of OmniFocus is its support for using a private server for storing and syncing your data. While the complimentary OmniSync Server remains a great option for most users, I for one welcome this advanced alternative. Being a happy user of OmniFocus for Mac for almost a year, I finally splurged on the iOS version in January. This motivated me to look into what it’d take to self-host my OmniFocus data.

In order for OmniFocus sync to work, your server must implement the WebDAV protocol. The Omni Group has published three separate guides describing the configuration process of Apache HTTP Server, Server.app, and WebDAVNav as personal WebDAV servers. I run my servers on Debian or Ubuntu and avoid Apache, so I wasn’t attracted to either of those options. Still, seeing Apache on the list gave me hope that there might be an easy NGINX alternative. Unfortunately, a quick search turned up pretty grim results. Not only WebDAV support provided by ngx_http_dav_module was incomplete, but The Omni Group themselves confirmed they found no easy way to make sync work with NGINX.

@maclemon Not at all. We tinkered with a test nginx server a little bit, but couldn't easily get it running well for WebDAV.

— OmniFocus (@OmniFocus) July 2, 2013

After a moment of despair, I decided to treat this as a challenge and try to get to the bottom of the problem. If you’re here for just the solution, feel free to skip to the last section of this article. Otherwise, I invite you to follow along and go down this metaphorical rabbit hole with me. As much as I could, I tried to turn this experience into a lesson on debugging an undocumented protocol.

Down the rabbit hole

According to the scarce information found on forums and Twitter, the NGINX’s built-in implementation of WebDAV was missing support for PROPFIND method responsible for metainformation retrieval of stored objects. Any discussion on the topic eventually linked to a 3rd party module called nginx_dav_ext_module by Roman Arutyunyan on GitHub. At a first glance, it looked like a dead end: no license, a bunch of unresolved issues, and I wasn’t too keen to jump and on the “build NGINX from source” train. Just imagining having to rebuild the server each time a new security patch would come out caused me to shudder. Luckily, additional research revealed that the nginx-extras package from APT comes with this 3rd party module included.

Praising the prudence of Ubuntu developers, I quickly set up a WebDAV virtual host on a local Ubuntu server running in VirtualBox and pointed OmniFocus to it. Synchronization attempt failed with a cryptic error message complaining about a missing directory.

I tried to manually create the specified path, but that resulted in a different error. Surprisingly, the NGINX logs showed no trace of a WebDAV error. It became evident that OmniFocus was merely expecting a different response to one of the requests.

To find out what the expected response was, I needed to consult a reference implementation. I was too lazy to install Apache, so I searched APT for “webdav” and immediately found the python-webdav package that looked promising. It ships with a binary called davserver that can be run as davserver --host 0.0.0.0 --port 8080 -B http://127.0.0.1:8080 -D /tmp/webdav -v -l INFO -n, which starts serving files from /tmp/webdav over WebDAV on port 8080.

Finally, OmniFocus reported a successful sync. By comparing the logs produced by NGINX and davserver, I soon noticed a discrepancy in the response status codes associated with a PROPFIND request to /OmniFocus.ofocus/.

# NGINX
10.0.2.2 - - [29/Dec/2016:19:44:25 +0000] "PROPFIND /OmniFocus.ofocus/ HTTP/1.1" 207 264 "-"

# davserver
10.0.2.2 - - [29/Dec/2016 20:03:44] "PROPFIND /OmniFocus.ofocus/ HTTP/1.1" 404 -

When querying a non-existent resource, NGINX was responding with 207 (Multi-Status) compared to 404 returned by davserver.

Note: At this point, I briefly considered a possibility of just running python-webdav behind NGINX. To a great surprise, I discovered that, in reality, the synchronization didn’t work. The data dir remained empty even after OmniFocus reported a successful sync. Further investigation showed that renaming (moving) directories is broken in davserver and apparently results in removing the source path.

Since WebDAV is a protocol on top of HTTP, the codes themselves are just a piece of the puzzle, so I had to compare the response payloads as well. One of the easiest ways to do something like this on Linux is using tcpdump. For example, tcpdump -i enp0s3 -A tcp port 8080 will print all packets communicated through port 8080 on interface enp0s3 in ASCII. The output is not pretty, but readable. Armed with tcpdump, I could finally see that in both cases OmniFocus sent an identical request to the server.

PROPFIND /OmniFocus.ofocus/ HTTP/1.1
Host: localhost:8080
Content-Type: text/xml; charset="utf-8"
Depth: 1
Connection: keep-alive
Accept: text/xml,application/xml
X-Caused-By: XMLSyncTriggerManualSync
User-Agent: OmniFocus-Mac/109.16.0.271654/v2.7.2 Darwin/10.12.2 (MacBookPro11%2C3) (aac-laptop.local)
Content-Length: 192
Accept-Language: en-us
Accept-Encoding: gzip, deflate

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<propfind xmlns="DAV:">
  <prop>
    <resourcetype/>
    <getcontentlength/>
    <getlastmodified/>
    <getetag/>
  </prop>
</propfind>

The responses differed though.

NGINX

HTTP/1.1 207 Multi-Status
Server: nginx/1.10.0 (Ubuntu)
Date: Thu, 29 Dec 2016 00:29:17 GMT
Transfer-Encoding: chunked
Connection: keep-alive

<?xml version="1.0" encoding="utf-8" ?>
<D:multistatus xmlns:D="DAV:">
<D:response>
<D:href>/data/OmniFocus.ofocus/</D:href>
<D:propstat>
<D:prop>
</D:prop>
<D:status>HTTP/1.1 404 Not Found</D:status>
</D:propstat>
</D:response>
</D:multistatus>

davserver

HTTP/1.0 404 Not Found
Server: DAV/0.9.8 Python/2.7.12
Date: Thu, 29 Dec 2016 00:35:23 GMT
Connection: close
Accept-Ranges: bytes
DAV: 1
Content-Length: 0

Where NGINX reported the file status via an XML-formatted response body, davserver was returning no payload. This gave me an idea to try to “fix” the missing file scenario via NGINX configuration directives. In pseudocode, I aimed for something like:

if request method equals PROPFIND and request file is missing
then respond with 404 and empty body
else yield control to webdav module

While NGINX supports “if” operator and has facilities to check for file existence, currently there is no “and” operator, so people on the Internet suggest a clever trick. First, you add an “if” for each argument of a logical conjunction. When truthful, concatenate a value to a string variable. Eventually, a final “if” checks if the resulting string indicates that each conditional was truthful. It’s easier to demonstrate than to explain.

# if request method equals PROPFIND…
if ($request_method = PROPFIND) {
  set $hack_propfind "T";
}

# …and request file is missing…
if (!-e $request_filename) {
  set $hack_propfind "${hack_propfind}T";
}

# …then respond with 404 and empty body
if ($hack_propfind = TT) {
  add_header DAV "1" always;
  return 404;
}

I wasn’t sure this hack would work, because NGINX’s “ifs” are tricky, but it did! Immediately after adding this block to the server config, OmniFocus reported a successful sync. No other issues surfaced during my further use of this configuration.

So, which implementation of PROPFIND is correct? Based on my limited understanding of the standard, at the very least, the NGINX’s interpretation is not wrong. The specification implies that the server is free to use a multi-status response type when a property is not found.

If there is an error retrieving a property then a proper error result must be included in the response. A request to retrieve the value of a property which does not exist is an error and must be noted, if the response uses a multistatus XML element, with a response XML element which contains a 404 (Not Found) status value.

— 9.1 PROPFIND METHOD, “RFC 2518, HTTP Extensions for Web Distributed Authoring and Versioning (WebDAV)”

Nevertheless, the fact that responding with 404 to this type of request helped me work around the problem doesn’t necessarily prove that OmniFocus fails to interpret a properly formatted response. There might be other factors contributing to this issue. In the end, only the OmniFocus developers can fully answer this question.

Solution

Disclaimer: I take no responsibility for any damage or information loss caused by this configuration. The users are encouraged to perform their own testing and properly secure their setup with HTTP authentication and TLS encryption (HTTPS).

Starting with OmniFocus 2.9.1 for Mac and OmniFocus 2.19.1 for iPhone, the workaround described above is not required. To get started with setting up your sync server, install the nginx-extras package from APT. It provides a version of NGINX with a WebDAV extension required for OmniFocus Sync to work.

sudo apt-get install nginx-extras

Next, edit your server config (usually placed in /etc/nginx/sites-available/webdav.mylittlewebsite.com) to look like this:

server {
  listen 80;
  server_name webdav.mylittlewebsite.com;
  
  # … authentication and TLS settings go here …

  location / {
    root /srv/www/webdav;
    client_body_temp_path /tmp;
    dav_methods PUT DELETE MKCOL COPY MOVE;
    dav_ext_methods PROPFIND OPTIONS;
    dav_access group:rw all:r;
  }
}

Don’t forget to symlink the created config to /etc/nginx/sites-enabled/.

ln -s /etc/nginx/sites-available/webdav.mylittlewebsite.com /etc/nginx/sites-enabled/webdav.mylittlewebsite.com

That’s it! I’ve been using this configuration for a full month now and had zero problems syncing OmniFocus between my MacBook and iOS devices.

While OmniFocus encrypts your data locally before uploading it to the server, you absolutely must configure TLS (HTTPS) and HTTP basic (or digest) authentication in NGINX. This task is outside the scope of this article and is left as an exercise to the reader. Also, note that by default OmniFocus will use your basic authentication password as the encryption passphrase. I recommend that you “unlink” passwords and use a distinct encryption passphrase. This way, even if your server is compromised by a malevolent actor, they won’t be able to decrypt the data using a cracked basic auth password.