Workflow And Tools To Maintain a Hyde Blog (featuring Jenkins)
Having switched my blog from Wordpress to Hyde, my blogging workflow now looks something like this:
- Write the blog post in Markdown format
- Run it through a Markdown previewer to make sure it’s formatted the way I intended
- Attach the metadata to the finished post
- Create the appropriate directory structures if it doesn’t exists (I’m now organizing the post with each month having their own directories, like /blog/2012/01/) and copy the file into that particular folder
- Push it to Github to backup my work
- Run it through
hyde gento have the site generated - Once the site is generated, rsync the directory into the server
Now, writing a blog post is already a laborious process in by itself and having to go through the menial and boring workflow like this is just going to sap and kill all the joy I derive from writing (and now I hold deep respect towards people who writes great blogging software and CMSes). So to smooth things over, an automated and pain-free solution is required. And it comes in two parts.
Preparing content for Hyde
Coming back to my workflow, this would be step 1 - 4 of the whole process. I’ve written a Python script that does just all that.
With this script, I’m now able to write my blog post as a plain text file, run it through this script, type in the title and tags I want to attach to this file and viola! The script will create the /blog/year/month directory structure and the file name of the post will be a proper URL slug derived from the title of the post, e.g. /2012/01/recovering-from-a-long-outage.html.
Automating deployment with Jenkins
If there’s a reason to stick to a blogging engine or CMS for your own site, this would be it: the time required to generate a Hyde blog is proportionate to the number of pages you have. There’s about 1600 posts in my blog and Hyde would need 9 minutes to generate the whole site. Nannying the generation and rsync script after a long night of writing == not fun, if not brutal. In my case, I get extremely cranky if my intense urge for my fluffy pillow and soft bed is hindered by a script.
To automate the build and deployment process, I’ve setup Jenkins on my server that would listen for any changes on Github and do the whole thing for me even in my sleep. Bliss.
As much as I find Jenkins really pleasant and easy to use (and I recommend that you check it out), I’ve fell through a lot of pitfalls in getting the configuration right. So to save you from the trouble of going through the knocks I’ve been through, here’s a checklist of setting up a fresh Jenkins instance that works on Ubuntu:
- Installing Jenkins for the first time: Instructions to install it on Ubuntu is on their website
- Host Jenkins on your domain: Decide on where you want to host Jenkins and do a proxy forwarding through Jenkin’s HTTP port - read it up on how to do it with Apache or Nginx
- Plugins to start off with: Jenkins Git plugin, Github plugin, Github Authentication plugin
- Enable Github authentication: In the Jenkins configuration screen, do the following:
- Check the “Enable security” checkbox
- Check the Github Authentication Plugin radio button
- Visit Github’s OAuth Application page and register a new application with the following setting:
- Main URL is the URL of your Jenkins instance
- Callback URL should map to
/securityRealm/finishLogin/on your Jenkins’ URL
- Get the Client ID and Client Secret from Github and paste it to its respective fields under “Global Github OAuth Settings”
- Under “Authorization”, check the “Github Commiter Authorization Strategy” radio button
- Under “Github Authorization Settings”, add your Github username to the “Admin User Names” field as a safety measure against locking yourself out from Jenkins :)
- Check the “Grant READ permissions for /github-webhook” checkbox
- Other configuration settings
- Under Git plugin, set “Global Config user.name Value” and “Global Config user.email Value”
- Under Shell, set “Shell executable” to
/bin/bash- the lack of features in/bin/shmakes young men like me cry
- Things to do while you’re SSH’ing into your server (and yes, some configuration from the commandline is necessary, unfortunately)
- Once you logged in, execute
sudo su - jenkins(i.e. perform everything from this point onwards as the jenkins user) - Perform
ssh-keygenand attach your public RSA key from this user to your Github account (see Github’s help for details) - Perform
ssh -T git@github.comto get Github’s host key - Logout as jenkins
- Execute
sudo apt-get install virtualenvwrapper pip
- Once you logged in, execute
- Creating Jenkins project to autodeploy your Hyde blog
- Add a Post-receive hooks on your Github repository to /github-webhook of your Jenkins URL
- Click on “New Job” on Jenkins’ sidebar
- Set the following configuration:
- Project name
- GitHub project - this will be the Github project where you host your blog files
- Source Code Management - check the “Git” radio button and set the “Repository URL”
- Build Triggers - check the “Build when a change is pushed to Github” checkbox
- Build - Click on “Add build step > Execute shell”, here’s the script I’ve set:
VENV_ROOT=.env
if [ ! -d "$VENV_ROOT" ]; then
echo "**> creating virtualenv"
virtualenv --no-site-packages "$VENV_ROOT"
fi
source "$VENV_ROOT/bin/activate"
pip install -r requirements.txt
# Start our deployment here
fab publishBash
Wrapping up
Phew! I have to say the whole process is indeed overwhelming as this whole experiment took me a whole day to get everything right.
If you’ve read up to this point, I really hope that you’d find this valuable… and that this would shave off hours in tinkering with the configuration. :)
Now if there’s a feature request in Hyde to selectively generate certain pages…
