Writing Collaborative Documentation with DokuWiki and GitHub

I've found myself thinking quite a bit about collaborative documents and wikis recently. I wrote a bit a few months ago about my new love for GitHub and while I'd heard of people throwing out the idea of collaborative writing using GitHub I hadn't seen that actually done too well (or much at all really) in practice. After all, at it's core, GitHub is a versioning system with a focus on software development with a bit of project management/bug tracking thrown in. Could one shoehorn the idea of collaborative writing into the platform? Sure. And I found at least a few examples of how that could work. Prose.io is a service that authenticates with your GitHub account and lets you edit documents within repositories using a great web-based editor. It's a great option if you're trying to avoid having to teach the basics of Git to someone (although I think when using a desktop client like GitHub for Mac this isn't as big a hurdle as one would assume). I had also bookmarked this Atlantic article on the potential for replacing wikis with GitHub repositories. The article points to the Tacofancy repository where over 40 contributors have put together a deconstructed view of taco recipes breaking down the food into it's various components (base layers like meat, seasonings, toppings, etc). But in the end these are all just folders full of Markdown files and while that's great in terms of data storage and portability, it doesn't offer a lot of what you'd expect a frontend site to provide. For example the search is going to show results based on line numbers, which is great for code but really confusing to look at when you search a string and raw markdown formatted text is the result rather than excerpts from the generated pages themselves. And I got to thinking about this even more so as Ryan began taking a hard look at the documentation for Domain of One's Own. This documentation has already taken 2 transformations since the inception of the project. We began by using a self-hosted MediaWiki install and utilizing Wiki Embed for the times where we wanted to embed documentation into other sites. There's nothing inherently wrong with that process, but MediaWiki comes with a lot of baggage. It's a bear to support, development of the platform seems downright stagnant, and damn if it couldn't use a fresh coat of paint in the way of a better default theme. When we began the full rollout of Domain of One's Own this year we switched to using the embedded knowledgebase that WHMCS has, in part as an experiment to see whether that would be better, and in part with the thinking that having documentation closer to the support area of UMW Domains was a smart thing. The knowledge base does get referenced and searched for keywords when a user goes to submit a support ticket which sounds like an awesome feature until you realize that no student submits a support ticket, they just shoot an email to you. The interface looked good on the front end but editing wasn't a huge step up from MediaWiki an in some ways it was even worse at how it handled WYSIWYG/html conversion. So when Jim brought up the idea of switching back to MediaWiki in the Spring, Ryan and I began testing a few other solutions to see if there might be something better out there.

I wanted so much for GitHub to be the solution to all of this. After all it's the latest craze, versioning is built right in, there was support for static file generation using Jekyll which gave the impression of possible front-end site work. Ryan looked at ReadTheDocs as one approach to generating a documentation site from raw files in GitHub. I looked at Octopress as another possibility. Ultimately all of those failed because the sites had to be generated on someone's computer every time a commit was pushed, which wasn't going to work (The basic Jekyll support built into GitHub Pages doesn't support external plugins like searching so it was a nonstarter). I thought the GitHub wiki feature might be an option as well, it runs a piece of software developed by GitHub called Gollum which supports Markdown files with more extensible interlinking between pages. The nice thing about GitHub wikis is that they serve as their own repository. You can add ".wiki.git" to the end of the repository URL and if they have a wiki enabled you can clone it, make edits, and commit them just like normal. That balanced with the front-end editor enabled with Gollum seemed promising, but a lot of other very basic features of standard wiki software like table of contents generation were completely absent. It also seemed like anything that normally played well with regular GitHub repositories (like Prose.io) wasn't going to recognize the wiki as a true project of its own. Back to the drawing board. > If you have a favorite piece of software for wikis that is not MediaWiki please hit me with it. Ideally self-host but whatever. — Tim Owens (@timmmmyboy) November 21, 2013

I put the call out on Twitter for suggestions and one of the first responses was Scott Leslie recommending I look at Dokuwiki. Dokuwiki was one that I had heard of before, in fact our own faculty member Marie McAllister who Jim interviewed for the latest DTLT Today episode had used Dokuwiki this past Spring. What I didn't realize was that Dokuwiki didn't use a database at all, the pages were stored as raw text files. I immediately realized if I could make Dokuwiki work as frontend software there'd be no reason I couldn't sync up the folder with all the raw text files to GitHub as well. I started hacking on it and we've got something pretty good at this point, in fact better than I ever could have expected. The DokuWiki install lives at http://docs.umwdomains.com/ where I've already started migrating some of the documentation we had already wrote. Dokuwiki has a pretty robust plugin architecture and what seems to be a decent community around it so finding solutions to smaller issues wasn't a problem at all. Want to embed video? There's a plugin for that. Markdown support in addition to standard wikitext? You got it. Autogenerate menu structures from folder trees? Yep. It seems like any time I think "I'd like to do something like this" I can find a plugin that gets me there. And installing/activating plugins is pretty darn close to being as seamless as Wordpress, just copy the URL to the plugin, paste it into the admin area and click install and it's up and running. Anyone whose had to get extensions working in MediaWiki will realize what a big deal that alone is. I even managed to find a plugin that would let me embed content from various separate wiki pages into one curated page which is ideal for building out custom documentation for a course. You can see this in action on the Wordpress page where all subpages of that category are being inserted in full on the main page with table of contents. This allows us to really granularly break up the content so when particular processes change we can make those changes in one location and the multiple pages that reference it will remain up-to-date. Dokuwiki stores all pages in maininstalldir/data/pages so the next step was to initialize that directory as a git repository synced up with our DTLT GitHub account. I've got that repository up at https://github.com/DTLT/dooo. Now since the files stored by DokuWiki are .txt instead of .md GitHub won't display them with formatting, however that's not a deal breaker for me. I had found a plugin for DokuWiki that allowed Markdown file support but new pages didn't automatically get the extension and I figured I'd rather have the ability to use the syntax with the supported file format than switch file extensions or have a combination of two file extension types. The last piece of the puzzle was sync between the two which turned out to be a pretty easy process of writing a small script that would pull any changes from GitHub, add all revisions that had been done locally on DokuWiki, commit everything, and push back to GitHub. You'll need to have an ssh key setup for your repository so that the script can run without needing a password and then just set a cron job for how often the sync should run (I chose every 5 minutes). Here's that script: [gist id=7593909] So now we have the ability to invite collaborators to use GitHub and fork our documentation as well as contribute by offering a pull request with changes and we can easily accept those changes which will get pushed back to DokuWiki. Meanwhile even though we still have GitHub, the DokuWiki offers a much cleaner interface and more advanced features for building out a robust wiki for documentation. I've taken the approach of editing the files from the GitHub repository locally on my laptop using my favorite Markdown editor, Mou, and pushing those changes to GitHub but anyone else could just as easily feel comfortable editing the pages directly in the DokuWiki interface and both work just fine. I'm still finding my sea legs with regards to getting the wiki formatted in a way that I think makes sense and looks good, but I think that will be an iterative (and yes, collaborative) process over the next few weeks. No doubt though, I think we've found a sweet spot to building out documentation that is openly available for reuse and collaboration but also meets the needs of our department and the project at hand.