1. Documenting Documentation

My focus on these pages is capturing, organizing, maintaining, and sharing knowledge. As I described in my site introduction notes, these pages are the “rubber duck” that I use to bounce my understanding of topics off of.

I’ve been delighted to receive a number of requests for information on building and maintaining such a site. I can’t think of a better way to give back to our collective community than sharing some tips of sharing information.

1.1. Project Goals

These note pages are the culmination of many little personal projects I’ve tinkered with since the early 1990s. My primary goals for this project are:

  • Minimal cost outlay on my part.

  • Minimal effort to build, secure, update, and maintain documentation on my part.

  • Based on open source technologies where feasible.

  • Reasonably performant.

  • Future-proof local and open file formats.

  • Compatible with my myriad other web projects.

1.2. The short version of how these pages are built and served

If you’re just after the details of how these pages are built, here you go:

  1. The raw notes source files are written using a plain text editor.

  2. Raw notes are composed in Restructured Text (.rst) format.

  3. RST format notes files are compiled into plain HTML and support files using the Sphinx static web site generator.

  4. Compiled HTML and support files are uploaded to a virtual server hosted on Amazon’s Web Services (AWS).

  5. The virtual server runs Debian Linux.

  6. Web pages are served by the nginx web server with local caching.

  7. My domain (DNS) is hosted on Cloudflare.

  8. I use Cloudflare’s free tier caching service to reduce the load on my virtual server. Although my AWS server hosts my content, copies are loaded into Cloudflare’s content delivery network (CDN) to improve global performance and reduce my bandwidth costs.

With that out of the way, here’s a bit more detail on the thinking that went into each decision.

1.3. Writing the content

I’m using Microsoft Visual Studio Code (VSC) since I’m already using it as a code editor. VSC supports a wide range of file formats, and surprisingly for Microsoft, it’s free and open source.

1.4. Choosing a source file format

Over the years, I’ve been burned by using products that were later discontinued, or simply superseded by new technology and techniques. I wanted a “future proof” solution for building for my documents and serving content before investing any significant effort. I really wanted to avoid vendor lock in and obsolescence. I wanted an open file format that could be edited with any editor now and into the foreseeable future.

There are two leading formats in widespread use that meet this criteria:

  1. Markdown (.md)

  2. ReStructured Text (.rst)

.rst offers a few nice features not available in .md format and it’s the preferred source file format for Sphinx (see below), so I have used that format to date (2018-2022). To get a feel for the .rst file format, you can just click on the View page source button at the top of this page. In practice, I’m finding that Markdown-based editors have improved significantly and Markdown has become the preferred documentation file format for web-based documentation. I’m considering migrating to the .md file format going forward. Fortunately, Sphinx can use both source formats, so an incremental switch shouldn’t be a problem – another reason to stick with open file formats. There seem to be very robust tools for the .rst to .md format conversion should I decide to convert all the content, but everything should work with any combination of source formats.

1.5. The Sphinx static web site generator

I’ve dabbled with various full-blown content management systems (CMS) over the years, and while many are excellent, all require constant attention to secure and maintain. My goals for simplicity, security, and low cost dictated that any sort of dynamic content and dependencies on resource-hogging server technologies wasn’t going to work. Fortunately, there are many excellent static site generators available. These compile various source file formats into static HTML files that can literally be hosted anywhere with a basic web server. You literally copy the generated HTML and other static content to your preferred web hosting solution. This simplicity does impose one major restriction: You can only serve static content. This means no user interaction, accounts, forums, or other means of interaction. I prefer python for the casual coding I do, so was biased towards a python-based static site generator. I found that Sphinx provides a powerful documentation system for python, and is used for many popular documentation sites (e.g., ReadTheDocs) so I selected to use it to generate my content. Sphinx uses the .rst file format by default, so that influenced my choice of source file formats. Any static web site generator will work just fine. I’m just biased towards Sphinx right now.

1.6. Hosting content

Having generated a bunch of content, the next question is inevitably how to serve it to the public. Here again, technology has burned me more than once. Effort I spent submitting forum posts to online communities was simply thrown away when the owning company decided to discontinue a service or simply go out of business. I still shudder to think how much time and effort I spent writing posts and replies on Google+. I also don’t like Medium or any of the big online content firms that profit from my efforts. This led me to a self-hosted solution.

1.7. My current approach

1.7.1. Amazon Web Services (AWS)

I already run a small AWS t2.micro web instance on Amazon’s cloud for my personal email server and small projects. The lightweight requirements for hosting these project pages don’t add significantly to my server load. #### Debian linux Here again, personal preference made this an easy choice. I’ve been running a debian linux server of some sort in my home since the early 1990s, and online since 2003.

1.7.2. nginx web server

The server demands for a static site minimal. I selected nginx simply because I have experience with it and adding my notes pages was trivial.

1.7.3. Cloudflare caching

Todo

More on Cloudflare and caching.

1.7.4. Having said all that…

All of this is much more complicated than is really necessary to serve these pages. Again, any hosting solution that lets you upload your own pages will work for such a site. The next incarnation of these pages will likely be built on the following combination:

  1. Amazon S3 to store raw files.

  2. Amazon CloudFront to serve static web content.

Todo

More on CloudFront and S3.

Contact and feedback

You can find me on Reddit where I lurk in many of the tech subreddits.

Last modified Mar 7, 2022. Last build on Apr 22, 2022.